Practitioner’s Corner: Reading the Tea Leaves: Interpreting the Bitsight Rating and Risk Vectors

On the surface, the Bitsight Rating and associated Risk Vectors look self-explanatory. However, taking them at face value doesn't tell the full cybersecurity story of your organization or your vendors. As with any complex dataset, interpreting the presented information takes a bit of domain knowledge and some leaps of intuition. The investigative skill involved can feel like arcana, almost like reading tea leaves.

Risk vs. Exposure

The Bitsight Rating provides a single, objective rating that's normalized across all organizations, taking into account footprint and size. More than just an arbitrary number, Bitsight's Rating is correlated to both likelihood of a security incident and to becoming a victim of ransomware.

Stepping back for a moment, the Rating was designed as a complement to other ratings systems such as:

• Consumer credit, including Equifax, Experian, TransUnion, and Schufa
• Capital markets and credit ratings, including Moody's, Fitch, S&P Global, and JCR
• Consumer goods, including Consumer Reports, Michelin, Zagat, TripAdvisor, and Yelp

As such, the Rating assesses an organization's performance over time, correlated to the probability of a negative outcome in the future.

To put this in perspective, if you miss a car payment for June, but make up for it in July—paying what's owed for both June and July—you may be caught up but your credit rating will drop and take a while to recover. You've shown that you either have a cash flow or a payment discipline problem, albeit slight in this case. If you miss additional payments for the car or other loans, or take on excessive potential debt by applying for multiple credit cards—even if you don't run them up—your rating continues to drop because you're a credit risk.

And that's the operative term—the Bitsight Rating is a measure of risk, of which vulnerabilities are just one factor. This subtlety is often lost on security operations, who monitor and manage immediate liability—mainly exploitable flaws in systems and software, including open ports, out-of-date or unpatched operating systems and applications, and malware. These are point-in-time issues, or exposure, and restricted to technology; risk is a measure of cybersecurity governance, policy, practices, and procedures, of which exposure is but one consideration.

For example, a Conficker infection is not an immediate threat. It's fifteen years old and none of the Windows vulnerabilities it exploits should be unpatched at this point in time. As a result, Conficker is no longer an exposure concern. However, all modern endpoint detection and response (EDR, aka host firewalls or anti-malware) tools should be able to detect and neutralize it, so the presence of Conficker tells us that the organization has poor defenses. If the infection persists for longer than a day, it tells us they have gaps in their detection and response practices and may lack log management or a SIEM.

Risk Vectors are more than vulnerabilities and open ports

Digging deeper into the rating model, Bitsight captures 25 Risk Vectors and their underlying findings. Most contribute to the overall Rating, although some are simply informational—they don't tell us anything about the organization's practices but are helpful in informing the organization about external threats—and others are being evaluated for eventual inclusion in the Rating.

Each of the Risk Vectors has associated findings, and each finding has a grade of GOOD, FAIR, NEUTRAL, WARN, or BAD, based on severity, which percolate up to a cumulative grade of A–F.

Summary of the Bitsight Findings, Grades, Risk Vectors, Rating, and Organization Tree

With the details out of the way, the question is what do you do with all this information? While the Risk Vectors in isolation tell you something about the practices—and to some extent the exposure—of particular control areas, the real power is interpreting them in combination as they relate to the type of organization you're assessing.

For example, take a vendor who provides software-as-a-service (SaaS). Their business is writing software and making it available on a web platform. Such an organization must be rigorous in writing software, including not only the functional aspects but also security, and managing the assets on which the software runs. These practices map to the following Bitsight Risk Vectors:

• Web Application Headers: Complete and properly formed security headers, including Content-Security-Policy headers and Strict-Transport-Security
• Server Software: Up-to-date services (web, email, etc), libraries, and plug-ins
• Patching Cadence: How quickly organizations patch detectable vulnerabilities
• Open Ports: System hardening and firewall configuration
• TLS/SSL Configurations: Configuration of network encryption
• TLS/SSL Certificates: Management of TLS/SSL certificates
• Malware Servers: A server owned by the organization hosting malware that may infect visitors

These Risk Vectors tell us something about their practices and security hygiene, both from a software development and operations perspective. Poor grades in some or many of the Risk Vectors should trigger a reassessment of the vendor or even an audit.

Whether the organization had any security incidents is also a consideration, particularly if it involved their platform or any back-end systems that may give an intruder access to customer-hosted data or source code for the SaaS application.

The astute reader will note that Bitsight finds additional assets beyond those that support the SaaS application. When a poor grade is surfaced, you should look at the assets included in the associated findings, prioritizing those that run the SaaS application. This may require a bit of investigation: looking at the name of the systems, performing DNS queries, and even reaching out to the vendor to find out which systems are relevant to the application.

But even those assets outside of the application are relevant. An organization is responsible for its extended ecosystem, whether it's a cloud-hosted infrastructure-as-as-service (IaaS) vendor or a marketing website developed and operated by a third-party. The organization's policies should specify the security posture of its own systems, from those containing sensitive information all the way to assets with minimal value and little to no intellectual property. 

Even the latter comprise a risk: as a real-life example, a government contractor's marketing website was compromised and the credentials of one its executives scraped from the content management system. The executive reused the credentials on his email account, which was subsequently used in a social engineering ploy, ultimately giving the attacker access to the internal network. In fact, many compromises don't employ a direct attack on critical systems, but rather exploit a chain of vulnerabilities giving incremental access until the attacker gains access to a critical system or administrative credentials.

Interpretation + Context = Prioritized and efficient resource allocation

Bringing it all together, organizations that understand their own business and that of their vendors can interpret and prioritize the importance of Bitsight information in context, and assign resources appropriately to reduce their overall risk. Some of our customers find something like the following table useful in that regard:

In this table, the top header is the type of vendor and the cells indicate the impact of poor vendor security performance in the associated risk vector. The impact takes into account the type of service provided, sensitive data held by the vendor, and access to systems.

You'll note that certain risk vectors are critical across the board:

• Botnet Infections: If the vendor is infected with botnets, it's probable that a threat actor has remote access to their internal infrastructure.
• Malware Servers: If the vendor's web systems are compromised and infecting visitors—including you as a customer—the integrity of the vendor's infrastructure is in question, as is the integrity of the data it contains.
• Open Ports: Depending on the port and any exploitable vulnerabilities in the service, the vendor may have already been compromised. 

Consider this example of how to read the table: A manufacturing vendor with poor hygiene in “Insecure Systems” often indicates exposed industrial control or industrial internet of things (IIoT) systems; whereas, with most other vendor types an exposed IoT system is a security risk, but not to the extent that it is for a manufacturer.

Drink your tea; read your Bitsight leaves

As with most analyses, Bitsight's information is most relevant and useful when interpreted in context. The Rating and Risk Vectors are a simplified presentation of risk, and it's up to you to determine the context. This means:

• Understanding the risk tolerance of your own business
• Setting goals and monitoring performance against them (governance)
• Drafting and maintaining the resultant policies and associated processes and procedures
• Applying the above to your own business as well as your extended vendor network and supply chain
• Understanding how each business unit, vendor, asset, application, and data repository applies to your cybersecurity posture, and may include performing threat analyses on some or all of them

Bitsight provides evidence of how an organization applies cybersecurity, and each finding is an artifact. Just as any sentence in a book or magazine article may not be notable in itself, when composed as sentences, paragraphs, and chapters, the story unfolds. Interpreting the results is mostly science, but there's an artistic element as well. In the case of Bitsight, that means having a thorough understanding of cybersecurity and drawing inferences from the data.