Today, I stumbled across one of the most badly designed risk matrices I have ever encountered. It has been published by the Government of Western Australia, Small Business Development Corporation, and is offered as a guiding example to small businesses (Ref. 1, 2):
The risk matrix contains several major flaws.
Scales with Inappropriate Ranges
The Consequence scale ranges as high as “Multiple Fatalities” but the Likelihood scale only goes as low as “Less than once in 2 years.” The user needs to be able to determine from the risk matrix when the likelihood of a multiple-fatality event has been reduced to a broadly acceptable level. One criterion for determining the limit for multiple-fatality events is F = 10-3/N2 per year where N is the number of fatalities (this criterion has been applied to a number of activities in the Netherlands such as the national airport, Shiphol). At this point, you may see another problem with the scales – vagueness – just how many fatalities are envisaged in the multiple-fatality consequence category? Suppose this category corresponds to 2 to 10 fatalities. Then, on the Dutch criterion, the frequency limit would be 2.5 x 10-4 (once in 4,000 years) to 1.0 x 10-4 per year (once in 10,000 years). Thus, the likelihood scale should go down very much lower than once in 2 years in order to accommodate multiple-fatality scenarios.
Scales with Strange Gaps
Suppose an event is predicted to occur once in two years on average. This is a higher frequency than for category 1 (defined as less than once in two years) but lower frequency than for category 2 (defined as at least once per year). Thus, there is no category in which to place the event that is expected to occur, on average, once in two years.
Likelihood Categories that are Almost Certain
From the Likelihood categories definitions, events in likelihood categories 2 to 5 occur quite frequently – from once per year (category 2) to every week (category 5). These events are almost certain to occur. Events that are almost certain to occur – or have already occurred – are better treated as issues rather than risks. The term "risk" should be reserved for unwanted events that are possible but not those that may be confidently anticipated to arise. This avoids cluttering the risk register with events that occur routinely.
Zero Width Categories
Consequence category 4 applies to the event of exactly 1 fatality so has zero width. It would be better to combine it with category 5 and call it, say, “1 to 10 fatalities.” This avoids giving the risk assessment team the challenge of distinguishing scenarios that cause a single fatality from those that cause two or more fatalities.
Inappropriate Risk Criteria
Consider a multiple-fatality event that is predicted to occur once in 3 years, on average. The risk matrix classifies this as a “moderate” risk that “may require corrective action.” Of course, such a severe event with such a high frequency should be placed in the highest category of risk and described as completely unacceptable.
Category Labels Treated as Quantitative Measures
The category labels 0 to 5 do not express the magnitude of the likelihood or the consequence. They could equally have been labeled A to F. Just because the labels look like numbers does not mean that they can be treated as numbers. The author of this risk matrix has multiplied these numerals, which are really just labels, in a pretense at producing a measure of risk. Then, for example, “Extreme” risk is defined as a product from 15 to 25. This oversimplified approach to designing a risk matrix will nearly always result in a risk matrix with an inappropriate coloring pattern, i.e. it will misclassify risks. This is a grievous type of error in risk matrix construction to which I have previously devoted an entire article.
I find it shocking that such a poorly designed risk matrix should have been presented as official guidance to small businesses by the Government of Western Australia.