Thoughts on the Future of Security Risk Measurement from SIRACon

Having just returned from my first SIRACon, I'd like to take a few moments to record my thoughts. Overall, the conference was fantastic. The talks were superb and the small size allowed me to rub shoulders with most everyone. Thinking back on the talks I attended, there were three major points that were repeated by most speakers.

First, for information risk managers to tackle the challenges currently facing the industry, they must have more scientific tools. Like any other scientific discipline, the information risk industry will benefit greatly from running experiments targeted at collecting much needed data and validating models of risk. Adam Shostack from Microsoft summarized this sentiment in his presentation “Building a Science of Security” and it was echoed consistently in other talks.

The second theme I heard at the conference was the need for better models. Up to this point, the status quo has been to create models based on measuring adherence to policies and procedures which, despite being well thought out and well intentioned, are not the best measure of security effectiveness. Innovators like Allison Miller demonstrated how EA is using data driven models to combat fraud and argued that such behaviors can benefit the industry as a whole.

Other speakers, such as Alex Hutton (whose talk I unfortunately missed), also advocated for developing data driven models to assess and measure risk. Patrick Florer suggested that models developed by other fields, such as immunology, could be adopted and used by the risk industry.

Lastly, the biggest takeaway was that without the proper data the industry will not be able to develop predictive models. Adam Shostack opened his talk with the quote that there are two types of companies: "those that have been hacked and those that don’t know they’ve been hacked". Yet, despite the assertion that the most if not all companies have been breached it is surprisingly difficult to develop models of breaches. This is because there is very little data on breach events available to security researchers. Shostack ended his talk with an exhortation that information risk professionals work towards encouraging organizations to more freely share information about data breaches.

In closing, SIRACon was a great experience. I learned more than I expected and met a lot of interesting, smart people. I encourage anyone interested in learning more about the challenges facing the information risk industry to attend the next event.