Defeating Quantifornication

Quantifornication is Bryan’s term for decision makers plucking likelihood, consequence and control effectiveness ratings out of thin air. Risk professionals have watched with increasing discomfort the prevalence of quantifornication as they have learned about the effect of bias on decision making.

Decision makers believe their judgement based on years of experience must be right; they don't see the need to put any more effort in when it is going to take time to get accurateestimates. Too many decisions makers prefer to fire buckshot and hit the target somewhere rather than use a scope and single bullet and hit bullseyes.

What is not understood is that improving many of your risk ratings, hitting bullseyes, is not nearly as difficult as most perceive it to be. Solving quantification of risk is seen as this big challenge. Either you need to be an actuary or statistician and/or you need loads and loads of data. To defeat quantifornication we first need to defeat these perceptions. Enter Dr Andrew Pratley.

For most people that have had to sit through an entire class of statistics, you’d reasonably assume the staff are sadistic and enjoy seeing people fall asleep, and slowly stop turning up. If you met the staff outside of this context, you’d never pick them to teach this subject. Like most educators, they’re passionate and spend considerable time and effort trying to explain ideas.

Why do so many educators in statistics consistently fail to translate these ideas into something that people can both remember and use? The problem is the language, not the numbers. Part of the problem with statistics, as is the case with most technical subjects, is the unique terminology. The COVID-19 pandemic has shown us many things, one unexpected outcome was the mainstream discussion of distributions, as in “we must flatten the curve!”. Distributions are the basis of statistics. We all have an intuitive understanding of statistics, we get lost in the detail without a framework.

Whilst there is a learning curve to the terminology, most of us are left with a dizzying array of formulas, methods and tables to work out how to use. None of which make sense outside of a specific context. So let us help you.

Statistics answers three types of questions:

1. Questions about probability – what is the probability of something happening within a certain range?

2. Questions about differences – how do these two things compare? i.e. are these two things statistically different?

3. Questions about relationships – how do these things relate? i.e. if I do more of one thing, how does that change something else

Everything that will transform our lives in the next 20 years from the application of machine learning and AI uses a combination of these three approaches.

What’s hard to see is how the problem you might want to solve fits into one of these three categories. To help you, we have developed the iQ3 Framework. A framework for applying these three core questions of statistics to the risk matrix. 

As you know, the risk matrix has two axes – likelihood and consequence and a third aspect, controls that shift likelihood and/or consequence up and down the ratings scales.

Using the iQ3 Framework we could link:

1. Questions about probability link to the likelihood axis. 

2. Questions about differences link to control measures.

3. Questions about relationships link to any of (i) likelihood & consequence (ii) likelihood & control measures or (iii) consequence & control measures.

The easiest example and the best place for a risk professional to start is question number two. Is one control more effective than another? Answering this will allow targeted investments.

Take phising emails training. Let’s say you have two types of training options for staff to combat phising attacks. You could easily collect data on the amount of phising emails opened by staff trained one way and the other and run a statistical test to see if one is method produces a different outcome, e.g. fewer phising  emails opened..

If you want to see a full worked example prepared by Andrew and I for the Risk and Cyber Week Conference, go to www.bryanwhitefield.com.au/riskandcyber.

Bryan Whitefield and Andrew Pratley

Bryan is a management consultant specialising in risk-based decision making, strategic leadership and planning born from his more than twenty years of facilitating executive and board workshops, and designing and implementing risk management programs for hundreds of organisations.

Andrew is an Adjunct at the University of Sydney Business School and teaches courses in the discipline of business analytics to both undergraduate and postgraduate students. Andrew is an independent consultant, speaker and expert witness in applied statistics.