<img alt="" src="https://secure.hiss3lark.com/187069.png" style="display:none;">
law-justice-icon

BitSight’s Methodology & Governance Process

BitSight is committed to fairness and accuracy for all organizations that are rated through our platform. BitSight believes that security ratings help address one of the greatest risks facing our society over the next century. BitSight was founded with the goal of increasing transparency about cybersecurity, enabling dynamic, informed interactions between global market participants and incentivizing a more trustworthy and secure global ecosystem.

BitSight is committed to creating trustworthy, data-driven, and dynamic measurements of organizational cybersecurity performance derived from objective, verifiable information. As the pioneer of the security ratings industry, BitSight established the guidelines for responsible development of security ratings. In 2017, BitSight helped create the "Principles for Fair and Accurate Security Ratings,” a series of practices developed alongside some of the world’s largest and most risk-focused companies. These Principles affirm the critical role of security ratings in society and the important responsibility that BitSight holds in creating these measurements.

Building on the Principles of Fair and Accurate Security Ratings, BitSight uses the below as a framework for our methodology and governance.

Accuracy Accuracy-white

Accuracy

Ubiquity-icon Ubiquity-white

Ubiquity

Stability Stability-white

Stability

icon-2 Comparability-white

Comparability

Empiricism Empiricism-white

Empiricism

accuracy-validation transparency-white

Transparency

Security ratings are a valuable tool to help organizations understand their security performance and those of their vendors. Security ratings were created with the goal of increasing transparency about cybersecurity, enabling dynamic, informed interactions between global market participants and incentivizing a more trustworthy and secure global ecosystem. As the framework for creating the methodology behind our ratings, BitSight uses the US Chamber of Commerce’s Principles for Fair and Accurate Security Ratings, which we helped develop.

 

In this document we outline the principles and methodology behind BitSight Security Ratings, including:

  • Objectives
  • Comparison With Other Rating Systems
  • The Ratings Process
  • Network Mapping
  • Risk Vectors, Grading and Weighting
  • ...and more

Read the full document to learn more

DOWNLOAD

To uphold the fairness of our ratings, all rated entities are entitled to the below rights, as part of the review process:

  • Provide transparency about the security ratings process.
  • Standardize treatment for customers and noncustomers.
  • Practice responsible disclosure in how we share ratings.
  • Provide a process for appealing ratings content (for customers and noncustomers), including access to the policy review board.
  • Enable any rated organization (including noncustomers) to get access to their rating details.
  • Facilitate participation and engagement with standards bodies, regulators and governmental bodies.

BitSight firmly believes that integrity is the mark of a true security ratings authority.

We believe in providing transparency about our ratings and we provide information in our portal about how ratings are calculated (i.e. which risk vectors were considered and their relative weighting) in our Knowledge Base. When we change our algorithms, we provide advance notice and demonstrate how such change will impact ratings.

We treat customers and noncustomers the same—our algorithms do not take into account whether an entity is a customer or not. In addition, we provide free access to our ratings for a limited period of time to all rated entities and will always work with any rated entity to improve the accuracy of its rating, regardless of whether it is a paying customer.

We do not publicly discuss specific ratings of companies via public forums (e.g. news outlets, industry events, etc.). We believe that we provide valuable insight into security through aggregate and industry trends. We do not believe in discussing a company’s rating publicly without permission, as this can pose a security risk to an organization.

While we are confident in the quality of our data, we believe that any organization using BitSight Security Ratings should have a way to dispute its ratings formally if it is ultimately not satisfied with the response it receives from BitSight. BitSight has created the Policy Review Board (PRB), which reviews issues of accuracy, fairness, and balance regarding BitSight Security Ratings. The PRB will review the information presented and will recommend the appropriate approach for BitSight to take. For more information, see What Is The Policy Review Board.

We also believe that responsible disclosure includes collaboration and sharing of information with law enforcement and governmental organizations and we offer our Sovereign Ratings product to help support these goals. We are also a signatory to the Principles for Fair and Accurate Security Ratings.

BitSight is committed to creating the highest quality and most accurate security ratings in the industry. We are also committed to allowing all rated organizations -- not just customers -- the opportunity to challenge the assets, findings, and interpretation of those findings used to determine a BitSight Security Rating and provide corrected or clarifying data. BitSight seeks accurate and prompt remediation for any dispute.

 

This document highlights our dispute resolution process for any rated entity (organization that has been rated by BitSight), including:

  • Disputing Data and Findings
  • Ratings and Calculation Disputes
  • Appeals and Adjudication

Read the full document to learn more

DOWNLOAD

The BitSight Policy Review Board (PRB) is a committee created to govern the ratings algorithm and associated policies, and to ensure that they are aligned with our principles. As the highest level of ratings governance, the PRB also adjudicates appeals related to data accuracy and evaluation methodology. It is charged with providing a consistent, transparent, and systematic dispute resolution process that is available to all rated entities. BitSight seeks accurate and prompt remediation for any dispute.

 

This document covers the purpose, composition and processes used by the Policy Review Board including:

  • Scope
  • Governing Principles
  • Composition
  • Types of Decisions
  • Process

Read the full document to learn more

DOWNLOAD

Request to temporarily suppress findings on newly discovered assets (Oct. 2020)

 

A rated organization requested a “refresh” of all of the findings in the BitSight rating (which is not uncommon, and is supported by our policies). In the course of performing the refresh, BitSight discovered a number of new assets which were not previously included on the report. The rated organization had been working through a strategic plan to remediate all of the negative findings on its assets, and the new assets were not accounted for in this plan. The rated organization requested that the new assets not be included on its report until the conclusion of that plan. Upon review, while we were sympathetic to the difficulties with the plan, we concluded that omitting the new findings altogether would be at odds with the integrity of the rating system, and would also be a disservice to other companies monitoring this particular organization as a vendor. Instead, we offered to create a breakout entity to capture the new assets (under the same original parent organization) and another breakout entity containing just the original assets, so that the rated organization could continue to use that to track progress against its plan.

 

Grading of DKIM keys shorter than 2048 bits (Oct. 2020)

 

A challenge was made to request that we change our grading policy for short DKIM (Domain Keys Identified Mail) keys, on the grounds that shorter keys are commonly used and do not pose significant risks. BitSight’s current policy is to grade keys shorter than 2048 bits negatively. A brief technical background: DKIM is an authentication mechanism intended to verify that email was sent from the domain where it claims to originate. DKIM relies on a cryptographic signature algorithm, RSA, which in turn uses a cryptographic key of variable length (512, 1024, 2048, and 4096 bits are common) and is the product of two large secret prime numbers. The security of the algorithm depends on the difficulty of factoring the key; longer keys require more computational effort to “break.” As computing power has increased, key lengths which were once considered secure have become feasible to factor. As of 2015, NIST recommends RSA keys of at least 2048 bits. We also found that nearly all major email service vendors support 2048-bit keys. For these reasons, we elected to maintain the existing policy.

 

Stale DNS records (Sept. 2020)

 

BitSight’s policy is to include an IP address in an organization’s network map if there is a DNS (Domain Name System) record that associates one of the organization’s domain names with that IP address. Several organizations made disputes along the following lines: the organization divested itself or relinquished control of an IP address, but failed to update the corresponding DNS entry. Later, a negative event occurred on the IP address, was included on the organization’s report, and affected its security rating. In each case, the organization felt that because it did not, in fact, use the IP address at the time of the incident, the event should not be attributed to it. After review, we agreed with this position, but also concluded that the stale DNS record itself provided evidence of gaps within the organization’s security management practices. Our policy for these situations therefore changed as follows: upon receipt of sufficient evidence that the IP address changed hands, we will remove negative events, but we will also record a DNS hygiene security incident on the organization’s report.

 

Expired certificate grading (Sept. 2020)

 

An organization petitioned to change our grading policy for expired SSL/TLS certificates. Under current policy, expired certificates negatively affect an organization’s security rating. The organization argued that they should not, on the grounds that they don’t pose a security risk per se. We carefully considered the matter and reviewed recent recommendations for best practices from standards bodies such as NIST. This confirmed our understanding that continuing to use expired certificates reflects gaps in security hygiene, and may in fact pose some risks. We elected to maintain our current policy.

 

Security incident impact vs. company size (Aug. 2020)

 

An organization experienced a data breach at one of its subsidiaries. The organization challenged our breach rating model, which would have reduced the rating of the entire company by the same amount as the (relatively small) subsidiary. In response, we studied the frequency of breaches vs. organization size, and found that empirically, reported breach frequency increases logarithmically with organization size, and therefore, the same breach carries more information about security performance when it occurs at a smaller organization vs. a larger one. After careful review, we updated our breach rating model to incorporate organization size, and we released the update in October 2020.

 

Ratings impact of guest network events (Aug. 2020)

 

A rated organization challenged the inclusion of certain malware events in its corporate rating. These events were claimed to have originated from the company’s guest network. Our policy for guest networks on separate IP ranges is to allow the organization to create a breakout entity (under the same overall parent organization) to segregate these. In this case, however, the guest network shared the same egress IP address as the corporate network, so it was not technically feasible to determine, from an external perspective, where the events originated. Because it would be exceedingly difficult to create a uniform solution to handle this in a way applied fairly to all organizations we rate, we elected not to make a policy change at this time, but will continue to investigate technical solutions in the future.

 

Use of ratings for competitive marketing (Aug. 2020)

 

To protect confidentiality and to respect responsible disclosure considerations, BitSight’s terms of service prohibit rated entities from publicly sharing ratings, except that an organization can share its own rating (for marketing or any other purpose). However, one of our clients, which had the highest rating in its peer group, and one of the highest in its industry, wanted to use this fact to aid in its sales effort and share competitor ratings. After review, we decided that sharing comparisons to industry averages would not jeopardize confidentiality, so we decided to amend the terms of service to allow this. Furthermore, on a case-by-case basis, we may allow sharing of comparisons against a company’s peer group (if the group is sufficiently large to preserve anonymity of the companies within it).

 

Malware sandbox testing policies (July 2020)

 

Many companies conduct tests of their security controls (e.g. firewalls and intrusion detection systems) by running actual malware within controlled “sandbox” environments. Sandbox solutions are available from a number of well-established vendors, and running malware within a properly configured sandbox is expected to pose minimal security risk. The traffic generated by the malware running in a sandbox is very similar (or identical) to that generated by uncontrolled malware infections, and so is useful for ensuring that security controls are able to detect and/or block such traffic. However, this also makes the testing traffic externally indistinguishable from actual infections.

In the interest of consistency and scalability, BitSight’s previous policy was to uniformly treat all apparent malware events identically, regardless of attestations that the events were due to deliberate tests. In response to a number of disputes, and the increasing prevalence of the practice of sandbox testing, however, we reexamined and modified this policy. We will now remove events that can be convincingly demonstrated to have originated from controlled tests; we will accept evidence such as screenshots of or logs from the malware testing software.

Back to Top