<img alt="" src="https://secure.hiss3lark.com/187069.png" style="display:none;">
law-justice-icon

BitSight’s Methodology & Governance Process

BitSight is committed to fairness and accuracy for all organizations that are rated through our platform. BitSight believes that security ratings help address one of the greatest risks facing our society over the next century. BitSight was founded with the goal of increasing transparency about cybersecurity, enabling dynamic, informed interactions between global market participants and incentivizing a more trustworthy and secure global ecosystem.

BitSight is committed to creating trustworthy, data-driven, and dynamic measurements of organizational cybersecurity performance derived from objective, verifiable information. As the pioneer of the security ratings industry, BitSight provides the only cybersecurity rating algorithm that is independently verified to correlate with data breaches. BitSight has also established the guidelines for responsible development of security ratings. In 2017, BitSight helped create the "Principles for Fair and Accurate Security Ratings,” a series of practices developed alongside some of the world’s largest and most risk-focused companies. These Principles affirm the critical role of security ratings in society and the important responsibility that BitSight holds in creating these measurements.

Building on the Principles of Fair and Accurate Security Ratings, BitSight uses the below as a framework for our methodology and governance.

Accuracy Accuracy-white

Accuracy

Ubiquity-icon Ubiquity-white

Ubiquity

Stability Stability-white

Stability

icon-2 Comparability-white

Comparability

Empiricism Empiricism-white

Empiricism

accuracy-validation transparency-white

Transparency

Security ratings are a valuable tool to help organizations understand their security performance and those of their vendors. Security ratings were created with the goal of increasing transparency about cybersecurity, enabling dynamic, informed interactions between global market participants and incentivizing a more trustworthy and secure global ecosystem. BitSight ratings specifically are correlated with financial performance and likelihood of data breaches to help organizations be as informed as possible when managing their cybersecurity. As the framework for creating the methodology behind our ratings, BitSight uses the US Chamber of Commerce’s Principles for Fair and Accurate Security Ratings, which we helped develop.

 

In this document we outline the principles and methodology behind BitSight Security Ratings, including:

  • Objectives
  • Comparison With Other Rating Systems
  • The Ratings Process
  • Network Mapping
  • Risk Vectors, Grading and Weighting
  • ...and more

Read the full document to learn more

DOWNLOAD

BitSight believes in the value of cybersecurity ratings because we know they represent more than just what’s happening within your attack surface. As the only cybersecurity rating independently verified by external organizations, BitSight takes program and vendor risk management a step further to ensure companies are equipped with a trusted view of their cybersecurity hygiene.

BitSight is the only Security Ratings provider with proven outside validation of its ratings, which have been proudly proven to correlate with data breach risk as well as business financial performance. Combined with a dedicated committee to govern its ratings algorithm and associated policies, BitSight’s customers can trust our data.

  • BitSight’s cybersecurity rating correlation to breach performance has been reported by AIR Worldwide and IHS Markit.
  • BitSight has proven correlation with financial performance according to external Solactive research.

To get started managing your organization’s cybersecurity with the only independently verified security rating on the market, request a security ratings snapshot report.

 

BitSight has published multiple pieces that highlight how our rating is correlated with data breaches, including:

  1. BitSight ratings correlation with breach - In this datasheet, learn what security rating scores are correlated with a greater risk of breach, and what can be done to decrease breach likelihood. Download Datasheet
  2.  
  3. Botnet infections correlation with breach - In this report, learn why botnet grades are a key metric in determining the likelihood of large-scale malicious activity. Download Report
  4.  
  5. File sharing correlation with breach - As early as 2015 we shared our findings on how organizations with file sharing occurring on their network saw correlation with network attacks. Read More

To uphold the fairness of our ratings, all rated entities are entitled to the below rights, as part of the review process:

  • Provide transparency about the security ratings process.
  • Standardize treatment for customers and noncustomers.
  • Practice responsible disclosure in how we share ratings.
  • Provide a process for appealing ratings content (for customers and noncustomers), including access to the policy review board.
  • Enable any rated organization (including noncustomers) to get access to their rating details.
  • Facilitate participation and engagement with standards bodies, regulators and governmental bodies.

BitSight firmly believes that integrity is the mark of a true security ratings authority.

We believe in providing transparency about our ratings and we provide information in our portal about how ratings are calculated (i.e. which risk vectors were considered and their relative weighting) in our Knowledge Base. When we change our algorithms, we provide advance notice and demonstrate how such change will impact ratings.

We treat customers and noncustomers the same—our algorithms do not take into account whether an entity is a customer or not. In addition, we provide free access to our ratings for a limited period of time to all rated entities and will always work with any rated entity to improve the accuracy of its rating, regardless of whether it is a paying customer.

We do not publicly discuss specific ratings of companies via public forums (e.g. news outlets, industry events, etc.). We believe that we provide valuable insight into security through aggregate and industry trends. We do not believe in discussing a company’s rating publicly without permission, as this can pose a security risk to an organization.

While we are confident in the quality of our data, we believe that any organization using BitSight Security Ratings should have a way to dispute its ratings formally if it is ultimately not satisfied with the response it receives from BitSight. BitSight has created the Policy Review Board (PRB), which reviews issues of accuracy, fairness, and balance regarding BitSight Security Ratings. The PRB will review the information presented and will recommend the appropriate approach for BitSight to take. For more information, see What Is The Policy Review Board.

We also believe that responsible disclosure includes collaboration and sharing of information with law enforcement and governmental organizations and we offer our Sovereign Ratings product to help support these goals. We are also a signatory to the Principles for Fair and Accurate Security Ratings.

BitSight is committed to creating the highest quality and most accurate security ratings in the industry. We are also committed to allowing all rated organizations -- not just customers -- the opportunity to challenge the assets, findings, and interpretation of those findings used to determine a BitSight Security Rating and provide corrected or clarifying data. BitSight seeks accurate and prompt remediation for any dispute.

 

This document highlights our dispute resolution process for any rated entity (organization that has been rated by BitSight), including:

  • Disputing Data and Findings
  • Ratings and Calculation Disputes
  • Appeals and Adjudication

Read the full document to learn more

DOWNLOAD

The BitSight Policy Review Board (PRB) is a committee created to govern the ratings algorithm and associated policies, and to ensure that they are aligned with our principles. As the highest level of ratings governance, the PRB also adjudicates appeals related to data accuracy and evaluation methodology. It is charged with providing a consistent, transparent, and systematic dispute resolution process that is available to all rated entities. BitSight seeks accurate and prompt remediation for any dispute.

 

This document covers the purpose, composition and processes used by the Policy Review Board including:

  • Scope
  • Governing Principles
  • Composition
  • Types of Decisions
  • Process

Read the full document to learn more

DOWNLOAD

Appeal on testing infrastructure (Mar. 2021)

 

A rated company challenged BitSight’s policy of including all events within a company’s infrastructure in the top-level rating of the company’s corporate tree (The BitSight ratings platform allows an organization to be divided hierarchically into subsidiaries, but the root of the tree always represents the entire organization.) In this case, the company used a portion of its infrastructure for deliberately running malware for research purposes; the company wished this infrastructure to be excluded. BitSight does allow each organization to designate a subset of its infrastructure to be included in its primary rating. Customers can choose to subscribe to either the top-level rating, or the primary rating, or both, but are encouraged to monitor the primary rating, as it often better contextualizes the security performance of the organization. However, the appealing company in this case felt that relatively few of its customers were monitoring the primary rating vs. the overall rating, and so wished for the infrastructure in question to be excluded from all parts of its ratings tree.

The Policy Review Board decided not to make a change in policy at this time, in part because of the importance of applying a consistent and objective methodology across the entire inventory of rated organizations, and also the importance of providing a comprehensive view to clients of the rated organization. However, the PRB directed BitSight’s research teams to investigate ways in which this might be done in the future. In the short term, the PRB recommended additional steps to encourage customers of the appealing organization to monitor the primary rating.

 

Appeal on a compromised systems event (Feb. 2021)

 

A rated company requested that the ratings impact of a compromised systems event be reduced, on the following grounds: the company felt that there was insufficient forensics information available in the BitSight portal for the company to locate and remediate the event; therefore, the incident continued for several days and resulted in further ratings impact. The PRB decided not to change the ratings impact of the event; as in similar cases in the past, the PRB reiterated that the responsibility for detecting and remediating security issues must ultimately lie with the company itself. However, the PRB did recommend improving the level of forensics detail available for such events. Additionally, the impact of events spanning multiple days is being revised as part of the ratings algorithm update later this year.

 

VPN SSL certificates (Jan. 2021)

 

Two rated companies requested a change in policy for evaluating certificates used for virtual private network (VPN) servers. Under current policy, certificates that are not rooted to a well-known public certificate authority are graded negatively. Certain VPN solutions are configured by default to use self-signed certificates; others use internal (non-public) certificate authorities, which prompted the request for the policy change. The Policy Review Board carefully considered this request. As part of the decision process, the PRB and Security Research team investigated several solutions from VPN vendors, including vendor support for certificates rooted to a public certificate authority.

Based on this investigation and the recommendations of the Security Research team, the PRB elected not to make a change in grading policy at this time. Using public certificate authorities is still considered best practice (and is supported by all major VPN vendors); internally rooted certificates pose additional risks. The PRB did recommend updates to the knowledgebase articles on the SSL Configurations risk vector to more clearly explain the rationale for this grading policy.

 

Retroactive adjustments to security incident severity (Feb. 2021)

 

A rated company made an appeal concerning an update in impact to a breach it had experienced. The ratings impact of a breach depends on the severity of the incident, and in particular the count of records exposed. In this case, new information about the breach emerged approximately six months after the incident was first recorded. Based on these new revelations, BitSight retroactively increased the ratings impact of the incident. This is in keeping with current policy, although the length of the delay was unusual. The rated organization requested a change in policy to limit the window of time in which security incident impacts can be changed. It also requested that it receive alerts for any changes to past incidents.

After extensive discussion, the Policy Review Board decided not to implement a cutoff on when new information might alter the impact of a security incident that has already been recorded. The PRB weighed the value of such additional information against the effect of these retroactive updates on the customer experience. Adjustments as delayed as this particular one (six months) are quite rare, and even then the change in impact is typically small (eight points in this case). On the other hand, BitSight has substantial analysis that indicates that one incident is correlated with increased risk of another in the future, and that this correlation holds for multiple years. Taking all of this into account, the PRB decided that it would be difficult to implement a policy that would preserve the accuracy of the rating in these cases.

Regarding the lack of alerts when the impact of an incident is retroactively updated, the PRB agrees that this should be addressed. This enhancement has been added to the official product roadmap.

 

Appeals on spam propagation findings (Jan. 2021)

 

Several rated companies escalated incidents involving spam propagation findings. The activity in question was detected via a network of sensors deployed across thousands of mail transfer agents (MTAs). The findings were flagged as suspicious because their HELO string did not match the sender’s IP address or domain. This is often indicative of malware used in spam campaigns. However, several rated companies had difficulty in locating the source of these events, in part because there is often a large volume of legitimate email traffic to search through to find isolated anomalies. The companies appealed on two grounds: (1) that their ratings should not be reduced for events which they could not locate, and (2) that the ratings impact was disproportionately large, especially since many events spanned multiple days, and therefore resulted in multiple rating drops. On point (1), the Policy Review Board reiterated that while BitSight will make every effort to provide information that can help organizations understand and remediate issues, the remediation responsibility ultimately lies with the organization itself, and failure to locate compromised machines or other security issues cannot be considered for ratings purposes. However, the PRB did recommend improvements to the forensics details provided for spam propagation events, and also improvements to the knowledgebase articles on the subject. On point (2) the impact of events spanning multiple days is being revised as part of the upcoming ratings algorithm update; details will be announced later this year.

 

Preview period when acquisitions introduce new infrastructure (Nov. 2020)

 

A rated company requested that BitSight grant a preview period for monitoring one of its acquisitions’ infrastructure before it was incorporated into their company tree as a subsidiary, and therefore affected the acquiring company’s rating. The PRB agreed that immediately after an acquisition, the security posture of the acquired company is unlikely to reflect the security posture of the acquirer; it takes time for the acquirer to make meaningful changes. To help establish an appropriate time period, the PRB solicited input from companies of various sizes on their typical acquisition timeline. Based on these discussions, the PRB decided to change the policy on acquisitions to the following: BitSight will incorporate an acquired company into the acquirer’s rating six months after the transaction was closed, or three months after BitSight finds out about the acquisition, whichever date is further in the future. Furthermore, acquirers who are BitSight customers will be permitted a Security Performance Management (SPM) view into the acquired company’s infrastructure during the preview period. A future product enhancement will annotate the acquisition on both the acquirer’s and acquired company’s reports.

 

Request to temporarily suppress findings on newly discovered assets (Oct. 2020)

 

A rated organization requested a “refresh” of all of the findings in the BitSight rating (which is not uncommon, and is supported by our policies). In the course of performing the refresh, BitSight discovered a number of new assets which were not previously included on the report. The rated organization had been working through a strategic plan to remediate all of the negative findings on its assets, and the new assets were not accounted for in this plan. The rated organization requested that the new assets not be included on its report until the conclusion of that plan. Upon review, while we were sympathetic to the difficulties with the plan, we concluded that omitting the new findings altogether would be at odds with the integrity of the rating system, and would also be a disservice to other companies monitoring this particular organization as a vendor. Instead, we offered to create a breakout entity to capture the new assets (under the same original parent organization) and another breakout entity containing just the original assets, so that the rated organization could continue to use that to track progress against its plan.

 

Grading of DKIM keys shorter than 2048 bits (Oct. 2020)

 

A challenge was made to request that we change our grading policy for short DKIM (Domain Keys Identified Mail) keys, on the grounds that shorter keys are commonly used and do not pose significant risks. BitSight’s current policy is to grade keys shorter than 2048 bits negatively. A brief technical background: DKIM is an authentication mechanism intended to verify that email was sent from the domain where it claims to originate. DKIM relies on a cryptographic signature algorithm, RSA, which in turn uses a cryptographic key of variable length (512, 1024, 2048, and 4096 bits are common) and is the product of two large secret prime numbers. The security of the algorithm depends on the difficulty of factoring the key; longer keys require more computational effort to “break.” As computing power has increased, key lengths which were once considered secure have become feasible to factor. As of 2015, NIST recommends RSA keys of at least 2048 bits. We also found that nearly all major email service vendors support 2048-bit keys. For these reasons, we elected to maintain the existing policy.

 

Stale DNS records (Sept. 2020)

 

BitSight’s policy is to include an IP address in an organization’s network map if there is a DNS (Domain Name System) record that associates one of the organization’s domain names with that IP address. Several organizations made disputes along the following lines: the organization divested itself or relinquished control of an IP address, but failed to update the corresponding DNS entry. Later, a negative event occurred on the IP address, was included on the organization’s report, and affected its security rating. In each case, the organization felt that because it did not, in fact, use the IP address at the time of the incident, the event should not be attributed to it. After review, we agreed with this position, but also concluded that the stale DNS record itself provided evidence of gaps within the organization’s security management practices. Our policy for these situations therefore changed as follows: upon receipt of sufficient evidence that the IP address changed hands, we will remove negative events, but we will also record a DNS hygiene security incident on the organization’s report.

 

Expired certificate grading (Sept. 2020)

 

An organization petitioned to change our grading policy for expired SSL/TLS certificates. Under current policy, expired certificates negatively affect an organization’s security rating. The organization argued that they should not, on the grounds that they don’t pose a security risk per se. We carefully considered the matter and reviewed recent recommendations for best practices from standards bodies such as NIST. This confirmed our understanding that continuing to use expired certificates reflects gaps in security hygiene, and may in fact pose some risks. We elected to maintain our current policy.

 

Security incident impact vs. company size (Aug. 2020)

 

An organization experienced a data breach at one of its subsidiaries. The organization challenged our breach rating model, which would have reduced the rating of the entire company by the same amount as the (relatively small) subsidiary. In response, we studied the frequency of breaches vs. organization size, and found that empirically, reported breach frequency increases logarithmically with organization size, and therefore, the same breach carries more information about security performance when it occurs at a smaller organization vs. a larger one. After careful review, we updated our breach rating model to incorporate organization size, and we released the update in October 2020.

 

Ratings impact of guest network events (Aug. 2020)

 

A rated organization challenged the inclusion of certain malware events in its corporate rating. These events were claimed to have originated from the company’s guest network. Our policy for guest networks on separate IP ranges is to allow the organization to create a breakout entity (under the same overall parent organization) to segregate these. In this case, however, the guest network shared the same egress IP address as the corporate network, so it was not technically feasible to determine, from an external perspective, where the events originated. Because it would be exceedingly difficult to create a uniform solution to handle this in a way applied fairly to all organizations we rate, we elected not to make a policy change at this time, but will continue to investigate technical solutions in the future.

 

Use of ratings for competitive marketing (Aug. 2020)

 

To protect confidentiality and to respect responsible disclosure considerations, BitSight’s terms of service prohibit rated entities from publicly sharing ratings, except that an organization can share its own rating (for marketing or any other purpose). However, one of our clients, which had the highest rating in its peer group, and one of the highest in its industry, wanted to use this fact to aid in its sales effort and share competitor ratings. After review, we decided that sharing comparisons to industry averages would not jeopardize confidentiality, so we decided to amend the terms of service to allow this. Furthermore, on a case-by-case basis, we may allow sharing of comparisons against a company’s peer group (if the group is sufficiently large to preserve anonymity of the companies within it).

 

Malware sandbox testing policies (July 2020)

 

Many companies conduct tests of their security controls (e.g. firewalls and intrusion detection systems) by running actual malware within controlled “sandbox” environments. Sandbox solutions are available from a number of well-established vendors, and running malware within a properly configured sandbox is expected to pose minimal security risk. The traffic generated by the malware running in a sandbox is very similar (or identical) to that generated by uncontrolled malware infections, and so is useful for ensuring that security controls are able to detect and/or block such traffic. However, this also makes the testing traffic externally indistinguishable from actual infections.

In the interest of consistency and scalability, BitSight’s previous policy was to uniformly treat all apparent malware events identically, regardless of attestations that the events were due to deliberate tests. In response to a number of disputes, and the increasing prevalence of the practice of sandbox testing, however, we reexamined and modified this policy. We will now remove events that can be convincingly demonstrated to have originated from controlled tests; we will accept evidence such as screenshots of or logs from the malware testing software.

Back to Top