Bitsight Algorithm Update: What You Should Know

Tags:

cybersecurity algorithm update
Written by Andrew Burton
Senior Manager, Thought Leadership

Bitsight is committed to creating trustworthy, data-driven, and actionable measurements of organizational cybersecurity performance. As part of this commitment, Bitsight periodically makes improvements to our ratings algorithm. These updates often include new observation capabilities, enhancements to reflect the rapidly changing threat landscape, and adjustments to further increase accuracy and correlation with outcomes. We also make some changes based on direct feedback from rated entities.

We follow a detailed process whenever we update our ratings algorithm, and all changes are rigorously governed by our Policy Review Board to ensure that they adhere to our principles and policies. Additionally, it is important to note that we always provide a preview of the changes to our users (and what the likely impact on their rating will be), well before they affect live ratings.

Our latest Rating Algorithm Update is live, so let's take a look at what changed and why.
 

Ratings Methodology Adjustments

Bitsight Risk Vector/Mapping

Updates

Results

Open Ports

Findings Related to RDP graded as ‘BAD’

Better reflects the risk of operating these services, especially in regards to Ransomware

Limited the rating impact of individual findings

Ensures that the rating impact of a single Open Port record is more appropriate for smaller organizations

Desktop and Mobile Software

Implemented Dynamic N/A Grades

Ensures fair grading in cases of low record visibility—important for work-from-home environments

Employee Counts

Improved algorithms for estimating organizational employee counts

Ensures accurate employee counts and updates over time

Breach Re-calibration

Recalibrated breach methodology

Improves grading scheme and approach for assessing breach impacts that is more consistent with other vectors

Headline Rating

Revised weighting of Compromised System Findings and SSL configurations

Improves ransomware risk assessment in today’s work-from-home environments

Increased Robustness of Security Rating Calculation

Improves transparency into changes in the Security Rating

User Feedback Updates

As noted above, Bitsight regularly seeks feedback from rated organizations and may make changes to the ratings algorithm based on that feedback. The following updates are based specifically on rated entity feedback.

Risk Vector/Mapping

Updates

Results

Compromised Systems

Reduced impact of repeat infections

Improves disproportionate rating impacts across malware families while retaining penalties for persistent infections.

SSL Configurations

Adjusted weighting and addressed obsolete protocols

Improves grading consistency and updates grading to reflect best practices.

Web Application Headers

Changed grading behavior around WAH fixes 

Improves consistency and intuitiveness of grading.

SSL Configurations + Web Application Headers

Removed penalty for Wildcard DNS Hostnames

Improves grading accuracy by deduplicating records for customers employing Wildcards in hostnames

For additional perspective on why we make changes to the Bitsight Ratings Algorithm, we spoke with Ethan Geil, Senior Director, Data and Research.

Ethan, how frequently do we make updates to the Bitsight Ratings Algorithm?

The last major update was in 2018, and we made a minor update in early 2020. 

Bitsight has always been very deliberate about both the updates and the release process. We are very mindful of the impact of methodology changes on all the companies we rate. Any significant update includes a preview and comment period, during which we gather feedback from rated companies. (Editor’s note: You can read more about that process here.)

How do we decide on what gets updated and what doesn’t?

Our first consideration is whether any update is consistent with our Principles for Fair and Accurate Security Ratings. Our goal is to provide ratings are empirical, objective, and as strongly correlated as possible with outcomes (including security incidents). 

As the cybersecurity and threat landscapes continually shift, we perform research on what risk factors appear to have the greatest impact on risk of negative outcomes. Updates to the algorithm reflect this research. As an example, there has been a significant uptick in the prevalence of ransomware attacks. Our research shows that poor performance in the SSL Configuration and Certificate Risk Vectors is predictive of ransomware attack risk. Thus, we increased the weight of those risk vectors.

Feedback from rated organizations is also a major consideration. We leverage the collective centuries of cybersecurity expertise among our customers to help inform the findings we consider for the rating, and how we evaluate those findings. 

We make updates to the algorithm periodically, why don’t we just update continuously?

Our goal for our ratings is to enable our customers to make informed business decisions about cyber risk. For that to happen, it’s essential to maintain a certain level of stability. This enables meaningful comparisons over time and analysis of trends, among other things. Additionally, the algorithm update process is extremely rigorous and requires extensive research, testing, and validation. Finally, as mentioned above, we commit to providing a sufficient preview and comment period before releasing significant algorithm changes. 

We aim to strike a balance between stability and rigor on one hand, and responsiveness to the changing threat landscape and methodological innovation on the other.

What is the most important thing for people to understand about ratings algorithm updates?

We are excited that the algorithm update incorporates a tremendous amount of research and development, in addition to invaluable input from our customers, and we are confident that this will make the Bitsight rating even more vital for making informed decisions about cybersecurity risk.