<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1175921925807459&amp;ev=PageView&amp;noscript=1">
Big Data

Security Ratings: Quality over Quantity (but here are the numbers)

Debbie Umbach | April 16, 2015

Poor information security can lead to serious, public data breaches for companies and their customers. That's why BitSight Security Ratings are used by companies to evaluate and mitigate information risk. This risk applies to a company's vendors and M&A targets as well as its own internal security performance. Quantifying this risk is also key to the nascent cyber insurance market. The accuracy of these ratings is essential to ensure the right business decisions are made, and BitSight has been working since early 2011 to develop ratings that are supported by high quality data.  

History

The company, initially backed by the National Science Foundation, was founded on the premise of transforming how organizations evaluate risk and security performance through providing an external, objective view of their security posture.  After spending nearly three years gathering and analyzing hundreds of data sources, using rigorous methodologies to learn what worked and didn’t work, and gaining feedback from early customers, the company launched the first commercially available security rating platform in September 2013.  Nearly three years and 350 customers later, BitSight has strong customer validation for ratings quality. Not only does BitSight confirm events observed by other tools and internal monitoring, it has also captured events and diligence data identifying issues that other controls have missed.

Ratings Overview

BitSight Security Ratings range between 250 and 900, with higher ratings indicating better performance. These ratings are comprised of risk vectors, which include security events (observed compromises on a company’s network) and diligence risk vectors (steps a company has taken to prevent attacks). For each risk vector, an overall letter grade (A-F) is assigned, indicating the company’s performance relative to others. The grade takes into account factors such as frequency, severity, and duration (for events) as well as record quality, evaluated based on industry-standard criteria (for diligence).

It’s all about the Quality

Rating quality is based upon the accuracy of the risk vectors that comprise them. Security events, which make up 60% of a security rating, are especially important. Tens of millions of new security events are observed around the globe on a daily basis, but many of these are simply noise or false positives. What really matters for security ratings is evidence of actual compromise, e.g. a botnet that has invaded your network and may be sending sensitive personally identifiable information (PII) to a command and control center. Based on information from AnubisNetworks, a BitSight subsidiary, and numerous other global data sources, BitSight is able to detect evidence of actual attacks and measure information such as frequency, duration, and confidence. We do this through correlation and cross-checking against internally-developed sources, external vendors, and publicly accessible data.

Accurate security ratings give organizations confidence and ensure credibility when talking to peers, executives, third party vendors, insureds, or acquisition targets about security and risk performance. This is why BitSight places so much emphasis on ensuring security ratings -- and the algorithms and data behind them -- are accurate and actionable.

So what about the Quantity?

While it’s all about the quality, many people still want to understand how much data BitSight crunches. We don’t highlight it, but we are indeed a “big data” company. First let’s start with the top and then dive into the detail. Here we are by the numbers:

Number of data items*

Description

1

The top-level rating of a company ranging from 250-900. Tracked historically so you can see how the company is doing over time.

4

The number of categories for tracking risk vectors:

  • Events: observed compromises on a company’s network
  • Diligence: steps a company has taken to prevent attacks
  • User Behavior: observed actions, such as file sharing, taken by a particular user on an enterprise network
  • Publicly Disclosed Breaches: gathered from various news sources and FOIA requests

13

The total number of risk vectors we track:

  • Events: Botnets, Malware, Spam, Unsolicited Communications, Potentially Exploited
  • Diligence: SSL, SPF, DKIM, DNSSEC, Open Ports, Application Security
  • User Behavior: File Sharing
  • Publicly Disclosed Breaches

639

Risk Indicators observed (sub-categories of risk vectors):

  • Botnet infections: Banker, Backdoor, Carufax, etc.
  • Malware Servers: professor-xx.com, mebiuswebxx.com, etc.
  • Spam Propagation: Darkmailer, Malformed Email, etc.
  • SSL: Warn: RSA public key is less than 2048 bits
  • SPF: Bad: SPF record is improperly formatted
  • DKIM: Bad: Malformed public key
  • DNSSEC: Warn: DSA public key is less than 2048 bits
  • Open Ports: 7 (echo), 11 (systat), 23 (telnet), etc.
  • Application Security: HTTP Headers such as Expires and Content-Security-Policy
  • File Sharing: Pirated or Trojanized applications, music, and adult content
  • Etc. etc.

> 19,000,000,000 **

Daily volume of historical events processed.

> 20,000,000,000

Daily average volume of new records processed to determine which critical measurements include into the ratings.

* These numbers continue to grow month-over-month, year-over-year.

** We process all 19 billion curated events daily because we're always mapping IP addresses to new organizations, and we must determine if any of the historical events were relevant to the new organizations.

Conclusion

With our powerful, intelligent processing engine, BitSight is able to synthesize the vast amount of information we gather from around the globe in real-time into high quality, easy-to-understand security ratings. It takes time and brainpower to generate an influential metric that enables people to make significant business decisions. As Steve Jobs put it so well, “Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains.”

Suggested Posts

Security Ratings: Quality over Quantity (but here are the numbers)

Poor information security can lead to serious, public data breaches for companies and their customers. That's why BitSight Security Ratings are used by companies to evaluate and mitigate information risk. This risk applies to a company's ...

READ MORE »

Data Driven Security Podcast: Measurement & Security Performance

On June 22, 2014, BitSight CTO and Cofounder Stephen Boyer (@SWBoyer) joined Bob Rudis (@hrbrmstr) and Jay Jacobs (@jayjacobs) on their Data Driven Security Podcast series.  This conversation was long in the works, and something we were...

READ MORE »

Security Ratings: A Big Data Approach to Mitigating and Measuring Risk

In the past year data breaches have become a fixture on the news cycle. The major breaches across multiple industries have also caught the attention of business leaders, with a recent study noting that CEOs and senior executives rank cyber...

READ MORE »

Subscribe to get security news and updates in your inbox.