Policy Review Board Case Summaries

A rated company raised an issue around a domain that had been discarded in an acquisition. The domain in question had been part of a previously related brand that was discontinued, and the domain itself had been in use in a manner unrelated to their business, although the WHOIS record was not updated in the meantime. Historically, domain backdating requests have been treated differently from CIDR backdating requests; although ownership of an IP address can lead to situations of abuse, domains can provide more immediate brand and phishing risks if a bad actor decides to spoof emails and/or establish phishing/copycat sites. Bitsight has refused domain backdating requests in the past for the aforementioned reasons and also because best practice recommendations include updating WHOIS records and pursuing cases of domain squatting to reduce the risks of these types of outcomes.

The Policy Review Board decided to recommend a change in policy to allow for historical removal of data for the rated company upon correction of the underlying WHOIS record. This is in line with the current policy for backdating for IP address-based WHOIS records, although similarly Bitsight reserves the right to revert this backdating if new evidence is discovered.

Bitsight applies a methodical process for investigating and applying Breaches to ratings. This qualification process generally focuses on assuring proper company attribution, classification of the breach type, deduplicating media reports, understanding first and third party origin, and determining impact. However, the process does not validate the claims of journalists since a base level of trust is assumed via media outlets reporting on the incidents. In some cases additional evidence is required, and may even include soliciting feedback from the affected organization(s).

The Policy Review Board affirmed that solely relying on public media reports is sufficient justification for reporting a security incident or disclosure. However, the PRB has recommended an improvement to the existing policy to include more information about whether the reported events have been corroborated either by the affected parties or through independent investigation.

A rated company requested that Bitsight review the policies surrounding recorded Breaches originating from zero-day vulnerabilities. The company stated that these organizations may be managing an efficient program and applying patches as soon as they are available, but could be breached due to a flaw that is not publicly known. Security incidents are graded based on the type of incident and a few other factors.

The Policy Review Board decided not to change policy at this time. Breaches can be considered both a failure of external and internal controls and processes; there are many cases where an organization’s external perimeter is compromised due to a vulnerability, but no internal systems were affected due to fast incident response or intentional design considerations and other security controls to prevent further access. The PRB also maintains that Breach events provide unique and different insights about the strength of security posture controls for an organization, even in cases where an organization may have limited warning or knowledge of a potential exposure.
 

A rated company raised an issue with how their rating was impacted by a recorded Breach event. This particular company first experienced a Compromised Systems event that was detected by Bitsight. Later, they disclosed a ransomware incident publicly that they claimed was a direct consequence of the finding on their report. The rated company expressed that they felt the rating impact of the Compromised Systems finding or the recorded Security Incident should be reduced, as from their perspective the impact of these two related events might be double-counted if one led to the other.

The Policy Review Board decided to not recommend a change in policy at this time. Even in cases in which a detected misconfiguration or security controls issue is reported by Bitsight and those findings directly lead to Security Incidents, the PRB believes that information provided by public disclosures is unique and different in describing the state of security posture controls for that organization. These incidents can provide new information about the ability for an organization to react to security events and the strength of their compensating controls.

Bitsight has historically created ratings for entities named on annual reports, disclosed as acquisitions through reputable sources, or for notable products of rated companies. The presence of the latter products of rated companies were historically created by demand or in cases in which the product was well-known, leading to inconsistency in expectations when a given company tree was viewed.

The Policy Review Board determined that existing ratings without a legal entity as a basis should be collapsed into the most relevant parent rating when applicable and that future product-based ratings should not be created (unless by the express demand of the rated company) in order to help consumers identify which rating is the most relevant.

When a Breach event is associated with a rating, these events are typically assigned to the most relevant entity in the company tree for that organization. Although this policy is followed at the time of discovery for the breach, if a new Primary Rating is created by an organization it may not be assigned historical breaches that affected the rating it was derived from.

The Policy Review Board decided that Breach events should be associated to both applicable Primary Ratings and non-Primary Ratings, and that these breaches should be backfilled when Primary ratings are created.

A rated company raised an issue with how WHOIS attributions were determined in their company tree. This company used the same registration information for all WHOIS records across their entity and subsidiaries, which meant in practice that it could become difficult to determine which IPs and domains belonged to any part of the company tree. In these cases Bitsight has historically recommended that organizations update their WHOIS records to properly reflect which organization is the owner of the asset. In the interim, the rated company wanted to request a reassociation of those assets to their correct subsidiaries.

The Policy Review Board decided that, if a company has a single registrant for their WHOIS records, Bitsight could approve requests to reassign assets within the same tree. These requests would be subject to verification, and evidence would need to be provided if the verification process is inconclusive. We recommend in these cases that organizations correct these WHOIS records to reflect the rightful owner of the asset.

A rated company presented a situation in which, due to a contractual agreement they signed with a divested subsidiary, some assets were contractually required to be in their own company’s name, but the divested subsidiary controlled and operated the assets. This rated company requested that the assets in question be assigned to the divested company, and asked Bitsight what evidence they could supply in order to verify their situation.

The Policy Review Board agreed upon the need to verify the rightful owner, and that Bitsight would accept supporting evidence from the involved parties. If an organization believes that they may qualify and would like to apply for verification, Bitsight Support ([email protected]) can receive the request and coordinate the verification process.

In the absence of findings, some BitSight Risk Vector grades are assigned default grades. In some cases these default grades are selected based on the assumption that BitSight was innately unable to find enough information to assign a high level of confidence to the Risk Vector grade (which at the time of case review, Mobile Software, Desktop Software, Patching Cadence, are displayed as ‘N/A’ but graded as ‘A’, while other Risk Vectors default to the median grade of ‘C’, including DKIM, SSL Certificates, SSL Configurations, Web Application Headers. One Risk Vector, SPF, defaults to ‘F’ due to expectations that all rated companies would define an email security policy for their prominent primary domains.)

The Policy Review Board accepted a proposal to retain the last achieved letter grade when possible if a Risk Vector loses all previously applicable findings. This proposal was implemented in the 2023 Ratings Algorithm Update.

Rated organizations can request that Bitsight historically remove any Findings related to a certain IP address or CIDR range if that organization can show that the WHOIS record for those IP(s) were erroneously in their name. This proof typically comes in the form of an attestation from an ISP/carrier that indicates that the CIDR should have been reassigned at some point in the past and that a new tenant controls the IP(s).

The Policy Review Board decided to expand the policy to include more forms of attestation. If the WHOIS record has been updated through the Regional Internet Registry (“RIR”), Bitsight will support the request to backdate if an attestation is provided by a company officer. However, BitSight reserves the right to independently spot check the accuracy of the IP ownership, and should its investigation yield different results, BitSight can change the record if it discovers evidence of continued use of those assets beyond the stated date.

A rated company requested further data around Bitsight Mobile and Desktop Software findings, citing the need for more forensics in order to identify the devices reported in their Findings. The Mobile and Desktop Software Risk Vectors provide a sampling of user agents that have been observed by data partners to be egressing from company infrastructure owned by a rated organization. This data is intended to be a complement to endpoint detection and response (EDR) controls, and a high volume of outdated devices may be an indication that those controls need review.

The Policy Review Board recommended the addition of timestamps and the destination of the observed egress traffic when available. Although Bitsight does not recommend eschewing EDR-type solutions in exchange for Bitsight observations (which are a sample of possible observable machines for any organization), we believe that we should make as much data as possible available to consumers so that they can make informed decisions about their policies and programs.

The Policy Review Board examined the grading policy for detections of multiple Potentially Exploited events in a single day. Bitsight’s Compromised Systems section measures outbound detections related to malware (Botnets, Potentially Exploited), spam or other suspicious activity (Spam Propagation, Unsolicited Communications), and servers that attempt to distribute malware (Malware Servers).

Compromised System events are graded based on their frequency and duration. Single-day events are considered to be remediated if no future detections can be observed, and carry the least impact in comparison to events detected over a 3-day span (or longer), with the impact continuing to scale as long as the infection continues to beacon out. Likewise, multiple infections detected on the same day can also have an increased negative effect on the rating.

The Policy Review Board decided not to make a change in policy at this time. Compromised Systems findings are used to draw statistical inference about the strength of any controls that might mitigate infections; for instance, if an antiviral software is lacking up-to-date definitions it may not recognize a malicious file. Additionally, different infections on the same user device may demonstrate different controls issues depending on the malware family.

A rated company raised an issue with how Bitsight grades multiple certificate findings for the same hosts. Upon reviewing the data, it seems the default certificate, "Kubernetes Ingress Controller Fake Certificate", has a 1-day validity period, and it issues a new self-signed certificate each day. As each rescan will produce a new “Warn” finding, it has the potential for a single host to produce dozens of Warn results within the Event Lifetime period, disproportionately affecting the SSL Certificates grade.

The Policy Review Board has decided to not make a change in grading policy at this time. We believe that the self-signed certificates highlight a real risk: a system that was not intended to be publicly accessible, is in fact accessible from the Internet. The PRB agrees that it would be ideal to be able to further analyze these records in the platform to identify the root cause (exposed kubernetes ingress controller, in this case). Bitsight will investigate whether we can add additional context to the finding details. Unfortunately, it’s not technically feasible for us to deduplicate these findings in the short term: we use the certificate itself as a deduplication key, and we don’t have any way to match up “identical” certificates across different days in a consistent manner.

A rated organization requested that Bitsight grant it access to manage infrastructure found in another rated company’s report. This rated organization provided documentation to support a claim that it had permission to do so on that company’s behalf. Bitsight’s standard policy is that although customers have the ability to monitor third parties, they cannot by default exert influence on the infrastructure of those Bitsight ratings.

The PRB felt that there was considerable risk in granting this level of access without sufficient verification. The principles are similar to “know your customer” in the financial industry. For evidence of domain ownership, an industry-standard test to demonstrate ownership/control is to create a specified DNS record for the domain in question (e.g. https://letsencrypt.org/docs/challenge-types/). If an organization believes that this level of access is appropriate and would like to apply for verification, Bitsight Support ([email protected]) can receive the request and coordinate the verification process.
 

A rated company challenged Bitsight’s grading policy for Diffie-Hellman keys shorter than 2048 bits, as they believed that frequent key rotation could mitigate the risks of a shorter key size. Diffie-Hellman (DH) key exchange is used in SSL/TLS to establish a shared secret key between a client and server. Secrecy relies on the difficulty of inverting a cryptographic operation (discrete logarithm); in general, the longer the prime number p used in the DH key exchange, the more difficult it is to recover the secret key by brute force. A technique called Logjam (discovered in 2015) reduced the difficulty of this operation. It’s believed that recovering 1024-bit DH keys is currently feasible for a nation-state actor. As a result, NIST has recommended since 2015 that DH keys be at least 2048 bits.

The Policy Review Board decided not to make a change in grading policy at this time. The current grading policy that requires 2048-bit Diffie Hellman primes is well supported by recommendations from standards bodies, including NIST. There is no similar recommendation to support the practice of frequently rotating shorter, weaker primes.

A rated company requested that Bitsight make available a way for their organization to review newly discovered assets--and findings related to those assets--before these items could impact their corporate tree. Today, Bitsight rates all organizations based on any publicly discoverable IPs and domains and their configurations as soon as they are identified, and all of these assets must be visible at the top of the ratings tree (although they may originate from a subsidiary). This structure is designed to provide an unbiased view of a rated company’s organizational risk even if they have collaborated with Bitsight to provide additional self-published structure to their tree, which may otherwise alter how their risk is perceived by an observer.

The Policy Review Board decided not to implement a change in policy at this time. Any implementation of the requested review process would become a de facto change in Bitsight’s existing policy requiring all findings and assets to be visible upon discovery in an organization’s infrastructure.

A rated company challenged Bitsight’s grading policy for SSL certificate subject name mismatches. SSL certificates enable secure communication between a client and a server. Among other things, the certificate establishes the authenticity of the server (to prove to the client that the server is who it claims to be, and is not an attacker). To do this, the server hostname must be listed on the certificate. If it’s not, a certificate subject name mismatch error results when the client attempts to establish an SSL connection. This error can be ignored at the user’s discretion, but this opens the client to the risk of man-in-the-middle attacks.

The Policy Review Board decided to not make a change in grading policy at this time. Our policy is to grade cert name mismatches negatively because the TLS/SSL protocol requires a proof of authenticity to ensure the client is interacting with its intended destination -- a core factor in limiting the ability to conduct Man-in-the-Middle attacks. For example, if a client intends to visit a bank’s website, www.bank.com, but a certificate for www.website.com is returned by the server, it cannot trust whether it’s actually talking to www.bank.com. This is known as a certificate mismatch error and is established to be a TLS/SSL configuration error.

In general, we cannot change the rating impact of findings based on evidence of compensating controls; however, we do allow rated companies to annotate the risk or to exclude the asset in question from the company’s primary rating.

A rated company contacted Bitsight with a request to gain free access to their full rating, citing a need for visibility beyond that of Bitsight Core. Bitsight Core is a free offering provided to recipients of Bitsight’s Vendor Access program, which allows those Vendor Access users to continue their Bitsight access after the Vendor Access period expires, but provides only a limited view of an organization's rating.

The Policy Review Board decided to make no change in policy. Bitsight Core gives free limited access to a company’s rating, Risk Vectors, and rated Infrastructure. Although Bitsight Core is a more limited view than the full Bitsight Security Performance Management product, such access allows any company to review their rating through Bitsight and to validate and provide feedback on associated infrastructure and findings, and all rated companies can request amendments and provide feedback on their rating to Bitsight Support (at [email protected]). The Policy Review Board determined that Bitsight Core provides adequate access for a company to take action with respect to their Bitsight rating information, but a purchase would be required in order to gain full access to the Bitsight Security Performance Management product.

A rated company challenged Bitsight’s grading policy for SMTP ports that support SSLv3. Today, Bitsight negatively grades all services that offer SSLv3, even if a more modern protocol is also offered by the server. SSLv3 is a depreciated protocol that, under certain conditions, can be exploited using an attack called POODLE (Padding Oracle On Downgraded Legacy Encryption) which allows man-in-the-middle attacks to recover transmitted secrets. The appeal was introduced due to the need for SMTP servers to offer backwards compatibility for older clients and also due to claims that POODLE is an HTTPS-specific attack.

The Policy Review Board decided not to make a change in grading policy, either retroactively or in the future. The overwhelming majority of security practitioners and standards bodies consider SSLv3 to be obsolete and insecure. Although we acknowledge that the known SSLv3 vulnerabilities may be more difficult to exploit in practice for SMTP connections, the possibility remains; for example, the POODLE padding oracle exists for any connection, not just HTTPS. To this point, there is sometimes confusion due to the fact that the original paper that introduced POODLE used HTTPS as an example of an exploit mechanism, but POODLE is not restricted to HTTPS.

A rated company raised an issue with how Bitsight grades certain certificates. Established best practices dictate that internet-facing assets that serve leaf certificates must also provide intermediate and root certificates to prove their authenticity to the client. Bitsight performs trust anchor validation and negatively grades certificates that are either self-signed or fail to include the other certificates in their chain of trust, even if a client may be able to complete the chain with their own cached certificates. Some TLS/SSL services, however, require a response with a client certificate issued in advance or the server will terminate the connection with a TLS error and prohibit access to the application layer (e.g., a website); these services may be self-signed or otherwise fail the trust anchor validation process.

The Policy Review Board concluded that Bitsight should change policy when possible to exclude certificate chain validation for systems that require client certificate authentication, as the PRB considers it a reasonable mitigation to the risks posed by having self-administered trust anchors.

A rated company appealed Bitsight’s grading of the X-XSS-Protection header in the Web Application Headers Risk Vector. Currently, Bitsight assesses headers that are minimum expectations, referred to as required headers, and those that may be implemented optionally depending on the configuration of the web page. The X-XSS-Protection header is considered an optional header; these headers carry no weight if not included, and if included they can contribute positively or negatively to the impact of the record depending on its implementation.

After deliberation, the Policy Review Board decided to recommend a change in policy when possible to no longer grade this header. This decision was reached because (1) well-configured Content Security Policy implementations can render this header redundant, (2) OWASP now recommends setting the header to zero to disable the X-XSS-Protection auditor, and (3) almost all major browsers have depreciated the header.

A rated company requested to add a joint-venture subsidiary to their company tree. Today, Bitsight requires that subsidiaries are 100% wholly-owned or more than 50% owned if the company is compliant with the Accepted Accounting Standards.

The Policy Review Board decided to recommend a change in policy to allow joint-venture companies to be included under certain conditions. We will consider a company to be a subsidiary of a parent for rating purposes if 1) the subsidiary is wholly owned by the parent company, OR 2) the parent has above 50% ownership and is IFRS-10 compliant, OR 3) both the parent and the subsidiary agree in writing to the relationship.

A rated company raised an issue with an SSL Configuration that had been fluctuating between “Good” and “Bad” due to their Web Application Firewall (WAF). In some cases, Bitsight scans can be blocked by the firewall and served other content, such as a captcha page. If this is the case, successive scans could report different configurations depending on what content is offered.

Our research team thoroughly reviewed this case and determined that there was a clear way to remediate this issue. After research it turns out that the technology in use by the rated company was seen deployed by other organizations with TLS v1.2 and v1.3 enabled, which are currently considered secure. The company had an older version of the WAF that enabled TLS v1.1 which is vulnerable and no longer recommended.

A rated company requested that Bitsight begin to exclude expired certificates from grading that are hosted on assets that they no longer control, if they have ended a contract with that provider. Services that provide certificates that are expired are graded as “Bad”, and can indicate that the current controls an organization implements to manage these certificates needs to be reviewed. However, some hosting providers do not immediately delete client data upon termination of a contract, leaving certificates to expire with potentially little recourse for the affected organization.

The Policy Review Board concluded that Bitsight should implement a policy to remove these expired certificates from reports if it can be confirmed that there is no longer any active association between the provider and the originating company. This change in policy would only affect a rated company if there is no relevant WHOIS or DNS record that would indicate that the certificate is still in circulation.

A rated company requested that Bitsight create an exception for SSL Certificates signed by an internal Certificate Authority (CA). Best practice dictates that certificates that are publicly available must be signed by a public CA, allowing clients to verify that the site is authentic and in good standing. The rated company appealed on the grounds that some VPN gateway services have been observed with internal CA signed certificates and that these services are only intended to be used by a small number of clients.

The Policy Review Board decided not to implement a change in grading policy at this time. Many VPN services support the use of public CA certificates and recommend their use. A primary argument downplaying the risk of a subset of internal CA certificates is that they are only designed to communicate with a specific set of clients; in such circumstances, it is a best practice to prevent the exposure of such interfaces to the general Internet. Self-signed and internally issued certificates are additionally prone to broad categories of weak implementations that are either less likely to occur or outright prevented by public CA signing policies. A key risk uncovered by the SSL configuration risk vector are devices that are initialized and connected to the Internet, yet are never completely set up or are hardened properly (e.g. may have default credentials). Many such instances manifest as network appliances configured with self-signed or default certificates that are not rooted to a valid trust anchor and not designed to be used in production systems.

A rated company appealed Bitsight’s grading of the rdate protocol. Today, rdate is graded as a “Warn” issue and is a protocol designed to respond to requests for the current time. The rated company appealed on the grounds that (1) it felt that the time protocol posed no risk and (2) it has fewer known vulnerabilities compared to other services that are graded NEUTRAL, such as DNS or the alternative Network Time Protocol (NTP).

The Policy Review Board decided not to implement a change in grading policy at this time. This protocol has known security vulnerabilities; If the time is incorrect, it can be exploited by attackers to break secure connections and encryption certificates. This is in line with NIST recommendations, and NIST additionally points out that rdate “has poor error-handling capabilities in general, and many of the client programs that use this format are poorly written and may not handle network errors properly”. Implementations of obsolete protocols can also be a sign of poor overall configuration management.

A government agency requested that Bitsight prevent third parties from subscribing to ratings within its rating tree. At the time this was raised, Bitsight made available to customers any rating that it could generate for an organization. This agency raised concerns that data made available through Bitsight could pose a security risk if sensitive information about configurations were revealed through scans.

The Policy Review Board decided not to establish a policy of restricting subscriptions to ratings of any particular classes of entities, including national infrastructure. However, Bitsight does have a feature, Manage IP Visibility, which, when enabled, will prevent any subscribers from being able to view the infrastructure or IP addresses of findings for that entity. We strictly adhere to a responsible disclosure policy.

A rated company challenged Bitsight’s policy of including all events within a company’s infrastructure in the top-level rating of the company’s corporate tree (The Bitsight ratings platform allows an organization to be divided hierarchically into subsidiaries, but the root of the tree always represents the entire organization.) In this case, the company used a portion of its infrastructure for deliberately running malware for research purposes; the company wished this infrastructure to be excluded. Bitsight does allow each organization to designate a subset of its infrastructure to be included in its primary rating. Customers can choose to subscribe to either the top-level rating, or the primary rating, or both, but are encouraged to monitor the primary rating, as it often better contextualizes the security performance of the organization. However, the appealing company in this case felt that relatively few of its customers were monitoring the primary rating vs. the overall rating, and so wished for the infrastructure in question to be excluded from all parts of its ratings tree.

The Policy Review Board decided not to make a change in policy at this time, in part because of the importance of applying a consistent and objective methodology across the entire inventory of rated organizations, and also the importance of providing a comprehensive view to clients of the rated organization. However, the PRB directed Bitsight’s research teams to investigate ways in which this might be done in the future. In the short term, the PRB recommended additional steps to encourage customers of the appealing organization to monitor the primary rating.

A rated company raised concerns with how Bitsight rates cloud service providers (“CSP”). Many cloud service providers lease IP address space to their tenants, but choose to not reassign the underlying WHOIS records with the relevant Regional Internet Registry (“RIR”), as the tenancy for any given IP address could be short-lived and the RIR may not be able to react in a timely manner. Bitsight uses WHOIS information to identify IP address ownership in many cases and this lack of change in the records can mean that the configurations and actions of a tenant can be challenging to disambiguate from the actions of the cloud service provider.

The Policy Review Board recommended short-term improvements to the Bitsight reports in relation to cloud service providers in order to help Bitsight customers differentiate these reports from non-CSP ratings.

A rated company requested that the ratings impact of a compromised systems event be reduced, on the following grounds: the company felt that there was insufficient forensics information available in the Bitsight portal for the company to locate and remediate the event; therefore, the incident continued for several days and resulted in further ratings impact. The PRB decided not to change the ratings impact of the event; as in similar cases in the past, the PRB reiterated that the responsibility for detecting and remediating security issues must ultimately lie with the company itself. However, the PRB did recommend improving the level of forensics detail available for such events. Additionally, the impact of events spanning multiple days is being revised as part of the ratings algorithm update later this year.

A rated company made an appeal concerning an update in impact to a breach it had experienced. The ratings impact of a breach depends on the severity of the incident, and in particular the count of records exposed. In this case, new information about the breach emerged approximately six months after the incident was first recorded. Based on these new revelations, Bitsight retroactively increased the ratings impact of the incident. This is in keeping with current policy, although the length of the delay was unusual. The rated organization requested a change in policy to limit the window of time in which security incident impacts can be changed. It also requested that it receive alerts for any changes to past incidents.

After extensive discussion, the Policy Review Board decided not to implement a cutoff on when new information might alter the impact of a security incident that has already been recorded. The PRB weighed the value of such additional information against the effect of these retroactive updates on the customer experience. Adjustments as delayed as this particular one (six months) are quite rare, and even then the change in impact is typically small (eight points in this case). On the other hand, Bitsight has substantial analysis that indicates that one incident is correlated with increased risk of another in the future, and that this correlation holds for multiple years. Taking all of this into account, the PRB decided that it would be difficult to implement a policy that would preserve the accuracy of the rating in these cases.

Regarding the lack of alerts when the impact of an incident is retroactively updated, the PRB agrees that this should be addressed. This enhancement has been added to the official product roadmap.

Two rated companies requested a change in policy for evaluating certificates used for virtual private network (VPN) servers. Under current policy, certificates that are not rooted to a well-known public certificate authority are graded negatively. Certain VPN solutions are configured by default to use self-signed certificates; others use internal (non-public) certificate authorities, which prompted the request for the policy change. The Policy Review Board carefully considered this request. As part of the decision process, the PRB and Security Research team investigated several solutions from VPN vendors, including vendor support for certificates rooted to a public certificate authority.

Based on this investigation and the recommendations of the Security Research team, the PRB elected not to make a change in grading policy at this time. Using public certificate authorities is still considered best practice (and is supported by all major VPN vendors); internally rooted certificates pose additional risks. The PRB did recommend updates to the knowledgebase articles on the SSL Configurations risk vector to more clearly explain the rationale for this grading policy.

Several rated companies escalated incidents involving spam propagation findings. The activity in question was detected via a network of sensors deployed across thousands of mail transfer agents (MTAs). The findings were flagged as suspicious because their HELO string did not match the sender’s IP address or domain. This is often indicative of malware used in spam campaigns. However, several rated companies had difficulty in locating the source of these events, in part because there is often a large volume of legitimate email traffic to search through to find isolated anomalies. The companies appealed on two grounds: (1) that their ratings should not be reduced for events which they could not locate, and (2) that the ratings impact was disproportionately large, especially since many events spanned multiple days, and therefore resulted in multiple rating drops. On point (1), the Policy Review Board reiterated that while Bitsight will make every effort to provide information that can help organizations understand and remediate issues, the remediation responsibility ultimately lies with the organization itself, and failure to locate compromised machines or other security issues cannot be considered for ratings purposes. However, the PRB did recommend improvements to the forensics details provided for spam propagation events, and also improvements to the knowledgebase articles on the subject. On point (2) the impact of events spanning multiple days is being revised as part of the upcoming ratings algorithm update; details will be announced later this year.

A rated company requested that Bitsight remove their report from the Bitsight inventory. The Policy Review Board declined this request. Bitsight is committed to accuracy, fairness, transparency, and independence in ratings (https://www.bitsight.com/security-ratings-principles). We will work closely with any rated organization (whether or not it is a customer) to update a rating to better reflect an organization’s security posture. However, because of this same commitment to independence and objectivity, we cannot accept requests to discontinue rating any organization.

A rated company requested that Bitsight grant a preview period for monitoring one of its acquisitions’ infrastructure before it was incorporated into their company tree as a subsidiary, and therefore affected the acquiring company’s rating. The PRB agreed that immediately after an acquisition, the security posture of the acquired company is unlikely to reflect the security posture of the acquirer; it takes time for the acquirer to make meaningful changes. To help establish an appropriate time period, the PRB solicited input from companies of various sizes on their typical acquisition timeline. Based on these discussions, the PRB decided to change the policy on acquisitions to the following: Bitsight will incorporate an acquired company into the acquirer’s rating six months after the transaction was closed, or three months after Bitsight finds out about the acquisition, whichever date is further in the future. Furthermore, acquirers who are Bitsight customers will be permitted a Security Performance Management (SPM) view into the acquired company’s infrastructure during the preview period. A future product enhancement will annotate the acquisition on both the acquirer’s and acquired company’s reports.

A rated organization requested a “refresh” of all of the findings in the Bitsight rating (which is not uncommon, and is supported by our policies). In the course of performing the refresh, Bitsight discovered a number of new assets which were not previously included on the report. The rated organization had been working through a strategic plan to remediate all of the negative findings on its assets, and the new assets were not accounted for in this plan. The rated organization requested that the new assets not be included on its report until the conclusion of that plan. Upon review, while we were sympathetic to the difficulties with the plan, we concluded that omitting the new findings altogether would be at odds with the integrity of the rating system, and would also be a disservice to other companies monitoring this particular organization as a vendor. Instead, we offered to create a breakout entity to capture the new assets (under the same original parent organization) and another breakout entity containing just the original assets, so that the rated organization could continue to use that to track progress against its plan.

A challenge was made to request that we change our grading policy for short DKIM (Domain Keys Identified Mail) keys, on the grounds that shorter keys are commonly used and do not pose significant risks. Bitsight’s current policy is to grade keys shorter than 2048 bits negatively. A brief technical background: DKIM is an authentication mechanism intended to verify that email was sent from the domain where it claims to originate. DKIM relies on a cryptographic signature algorithm, RSA, which in turn uses a cryptographic key of variable length (512, 1024, 2048, and 4096 bits are common) and is the product of two large secret prime numbers. The security of the algorithm depends on the difficulty of factoring the key; longer keys require more computational effort to “break.” As computing power has increased, key lengths which were once considered secure have become feasible to factor. As of 2015, NIST recommends RSA keys of at least 2048 bits. We also found that nearly all major email service vendors support 2048-bit keys. For these reasons, we elected to maintain the existing policy.

Bitsight’s policy is to include an IP address in an organization’s network map if there is a DNS (Domain Name System) record that associates one of the organization’s domain names with that IP address. Several organizations made disputes along the following lines: the organization divested itself or relinquished control of an IP address, but failed to update the corresponding DNS entry. Later, a negative event occurred on the IP address, was included on the organization’s report, and affected its security rating. In each case, the organization felt that because it did not, in fact, use the IP address at the time of the incident, the event should not be attributed to it. After review, we agreed with this position, but also concluded that the stale DNS record itself provided evidence of gaps within the organization’s security management practices. Our policy for these situations therefore changed as follows: upon receipt of sufficient evidence that the IP address changed hands, we will remove negative events, but we will also record a DNS hygiene security incident on the organization’s report.

An organization petitioned to change our grading policy for expired SSL/TLS certificates. Under current policy, expired certificates negatively affect an organization’s security rating. The organization argued that they should not, on the grounds that they don’t pose a security risk per se. We carefully considered the matter and reviewed recent recommendations for best practices from standards bodies such as NIST. This confirmed our understanding that continuing to use expired certificates reflects gaps in security hygiene, and may in fact pose some risks. We elected to maintain our current policy.

An organization experienced a data breach at one of its subsidiaries. The organization challenged our breach rating model, which would have reduced the rating of the entire company by the same amount as the (relatively small) subsidiary. In response, we studied the frequency of breaches vs. organization size, and found that empirically, reported breach frequency increases logarithmically with organization size, and therefore, the same breach carries more information about security performance when it occurs at a smaller organization vs. a larger one. After careful review, we updated our breach rating model to incorporate organization size, and we released the update in October 2020.

A rated organization challenged the inclusion of certain malware events in its corporate rating. These events were claimed to have originated from the company’s guest network. Our policy for guest networks on separate IP ranges is to allow the organization to create a breakout entity (under the same overall parent organization) to segregate these. In this case, however, the guest network shared the same egress IP address as the corporate network, so it was not technically feasible to determine, from an external perspective, where the events originated. Because it would be exceedingly difficult to create a uniform solution to handle this in a way applied fairly to all organizations we rate, we elected not to make a policy change at this time, but will continue to investigate technical solutions in the future.

To protect confidentiality and to respect responsible disclosure considerations, Bitsight’s terms of service prohibit rated entities from publicly sharing ratings, except that an organization can share its own rating (for marketing or any other purpose). However, one of our clients, which had the highest rating in its peer group, and one of the highest in its industry, wanted to use this fact to aid in its sales effort and share competitor ratings. After review, we decided that sharing comparisons to industry averages would not jeopardize confidentiality, so we decided to amend the terms of service to allow this. Furthermore, on a case-by-case basis, we may allow sharing of comparisons against a company’s peer group (if the group is sufficiently large to preserve anonymity of the companies within it).

Many companies conduct tests of their security controls (e.g. firewalls and intrusion detection systems) by running actual malware within controlled “sandbox” environments. Sandbox solutions are available from a number of well-established vendors, and running malware within a properly configured sandbox is expected to pose minimal security risk. The traffic generated by the malware running in a sandbox is very similar (or identical) to that generated by uncontrolled malware infections, and so is useful for ensuring that security controls are able to detect and/or block such traffic. However, this also makes the testing traffic externally indistinguishable from actual infections.

In the interest of consistency and scalability, Bitsight’s previous policy was to uniformly treat all apparent malware events identically, regardless of attestations that the events were due to deliberate tests. In response to a number of disputes, and the increasing prevalence of the practice of sandbox testing, however, we reexamined and modified this policy. We will now remove events that can be convincingly demonstrated to have originated from controlled tests; we will accept evidence such as screenshots of or logs from the malware testing software.