Debunking Security Rating Myths

Security Ratings are still a relatively new phenomenon. As a result, many security and risk professionals are still familiarizing themselves with how ratings work, the data used to compute ratings, and how ratings are put into action. We expect this education to continue: consumer credit scores are always changing and after many years, people are still constantly coming up to speed with the multitude of factors that affect their score.

While security and risk professionals familiarize themselves with the emergence of security ratings, it is vital to recognize information that is misleading - or meaningless. Below are some common myths about security ratings that everyone should be aware of.

#1 Ratings claiming to have X% more data

Security Ratings are a big data application that assess a company’s security performance from the outside-in. Rating firms need to leverage as many data sources as possible in order to be accurate. Moreover, the data consumed by rating firms needs to be vetted for accuracy before being added into the calculation of a security rating.

When a firm claims to leverage a larger percentage of data than other rating services, ask yourself:

• How could they know this? No business is disclosing the exact amount of data they collect.

• The more important questions to ask are; where is the data coming from? How proven are these data sources? Is the data relevant to the security hygiene of my company?

• What does relative data volume even mean if the rating isn’t actionable, especially when it comes to managing third party vendors?

#2 Creating Data In-House Is Better Than Acquiring It Elsewhere

Some rating firms claim that they have “proprietary” data collection methods, and that their ratings are inherently more accurate since the bulk of the underlying data is gathered internally. Businesses should be wary of ratings that are comprised almost entirely of internally generated data.

Ask yourself:

• Is the data as comprehensive as other providers? There are many threat intelligence providers that have advanced sinkholing infrastructures, honeypots, and mature data collection methods that span 10 years or more. Rating firms that are trying to reinvent the wheel and replicate these processes in house are likely falling short in terms of quality.

• Who is checking this data for false-positives and other inaccuracies? Rating firms who partner with threat intelligence providers will have their eyes peeled for inaccuracies and inconsistencies.

This shouldn’t be a case of either/or. Ideally, a ratings provider should do both; collect unique data through proprietary methods that matter, and then collect the highest quality data available in the marketplace. It might be that others have exclusive rights to these data sets which has prevented other rating providers from getting access. A firm touting the fact that proprietary data is always superior could be “taking a negative and spinning it into a positive”.

#3 All risk vectors are equal

Not all cyber threats pose the same risks to organizations. Accordingly, risks that cannot be immediately verified should not be factored into a security rating. Some threats, such as botnets (actual compromised machines) and peer-to-peer file sharing have been directly correlated to data loss. Rating factors should be weighted appropriately so that organizations can identify and prioritize security threats that put them at risk.

Ask yourself:

• Which cyber threats pose the greatest risk to my organization? How organizations deal with the most damaging cyber threats should be the basis of an accurate security rating.

• Is there a reason as to why a certain risk affects my rating? Rating services should explain why some risks are calculated into their ratings, and why others are not.

Don’t Fall For Security Rating Myths

At the end of the day, ratings should be put into action. Any of the above claims warrant a double take. Security and risk professionals are well-versed in buying GRC and security products, yet security ratings remain a new concept for most. As different security rating options emerge, it is vital to know what is valuable and what is hyperbole. Organizations need security ratings that are proven to help them manage cyber risk.

See how Chris Porter, CISO for Fannie Mae has been using BitSight Security Ratings to monitor the security posture for hundreds of third parties. To learn more about the data BitSight uses to calculate Security Ratings, click here.