Hello, my name is Ian Nicholson, VP of Solutions at Emerald Associates.
Over the past 20 years, I’ve gained a deep understanding of both the capabilities and the limits of Oracle Primavera Risk Analysis (OPRA).
Back in 2006, when it was still known as Pertmaster, it was more than capable of meeting the requirements of the average project manager. However, during the last decade, Oracle has been gradually sunsetting the product, downgrading it to “Controlled Availability” status last year. A move that essentially rendered it no longer fit for purpose.That being the case, I’ll be publishing a series of articles in the coming weeks that explain why I believe it’s high time OPRA users upgraded to Safran Risk (SR).
Part four investigates the concept of ‘Risk Analysis by Exclusion’, comparing OPRA’s approach to the process with Safran Risk.
What is Analysis by Exclusion?
I first heard about analysis by exclusion a few years ago while I was helping a large mining company refine their risk processes. Curious, I asked for a demonstration of how it worked in OPRA.
In step one, the existing risk model was run through OPRA with all extant risks attached. Once completed, the team would then rerun the model multiple times, removing risks one at a time and comparing the outputs from each updated model with the initial analysis. The result was that the team were able to more accurately gauge the impact of individual risks on the wider project.
However, it quickly became apparent there were a few drawbacks.
The Problem with Risk Analysis by Exclusion in OPRA
The main issue was time.
To effectively answer the question “What is the impact of this specific risk on the schedule?” and generate usable data, OPRA users would have to:
- Run the model with all risks turned on and record the results
- Manually turn off a risk, rerun the model, and record the results
- Turn the risk back on
- Repeat steps two and three to sample all risks in a project
Additionally, there were issues with quantifying the impact of each risk. Tornado charts ranked activities and risks automatically in OPRA, but they were also based on Pearson’s Product Moment. This meant they were categorized by percentage rankings: something project managers struggle to interpret.
All-in-all, I was informed it would take about a week to perform comprehensive risk analysis by exclusion on a large construction schedule using OPRA; more if the model was subjected to changes during the process.
One of the key tenets of workplace efficiency is that if a process must be repeated time and again, it should be automated.
For starters, this saves time that would otherwise be wasted performing mundane tasks. But it also improves accuracy — which is crucial for risk analysis by exclusion and the process more generally.
Imagine you forgot to turn Risk Factor #9 back on before switching off Risk Factor #10, the results of that iteration of your analysis would be skewed. The team might then act on the wrong Risk Factor, simply because its impact was incorrectly overstated.
This got me thinking about automating the process in OPRA using visual basic. I brought the question to my friend and colleague Russell Johnson — who also happened to be the former CTO of Pertmaster.
His response was interesting. While technically possible, he warned me several big obstacles stood in the way.
- Because there was no way of creating a native risk driver event with OPRA, simply creating and identifying a suitable model would be difficult
- Natively storing and processing results was similarly impossible within the software’s standard parameters
- OPRA was notoriously slow; especially in comparison with something like Safran Risk
Safran Risk is Both Faster and More Accurate
These days, the question of manual vs automated analysis by exclusion in OPRA is moot, seeing as Oracle dropped visual basic support years ago. Nevertheless, when developing Safran Risk, Russell Johnson remembered the difficulties experienced by OPRA users and so decided to automate the process.
The results speak for themselves.
Analysis that would have taken 40 hours to complete in OPRA takes mere minutes when conducted via Safran Risk. Moreover, it offers far greater accuracy, reducing the likelihood of mistakes to zero.
To understand how it does this, we need to take a closer look at how analysis by exclusion works in SR.
Analysis by Exclusion in Safran Risk
To begin, Safran Risk allows users to perform analysis in single or multiple passes.
During a single pass, SR runs through each risk once. It then displays an output with each risk individually excluded — essentially, it’s the same exercise my client was performing, only automated.
Below is a diagram displaying the results of a single pass analysis with all risks on a demo ‘Drone’ project included.
Compare that with the diagram below which shows the results when only one risk and opportunity have been removed.
In the examples given here, the black line represents all risks turned on (the “None” line) and the green line represents the same model with only the Testing Overrun risk turned off. This indicates that, at P70, we would save 33 days if we could eliminate the Testing Overrun risk.
However, the cumulative saving of removing multiple risks is not entirely clear in this example. We also have to bear in mind the fact that turning off one risk may mitigate another; particularly if they’re closely correlated. We could get around this by performing the task manually: turning off the top risk, running the analysis, running the next risk, etc. But this only serves to increase the workload.
To truly understand the interaction of the top risks, we must run the same analysis but using multiple passes.
In multiple pass mode, the system runs for a pre-determined number of iterations as specified by the user. With each new iteration, it removes the top risk; with the previous iteration’s risks turned off. This prevents large impact risks from overshadowing lower impact risks, while also displaying the cumulative impact mitigating each additional risk will have on the schedule.
So, in the example below, we turn off the top five risks cumulatively. Pass one turns off Testing Overrun; pass two both Testing Overrun and Design, and so on.
Again, look at the P70 values:
- When the Testing Overrun risk is excluded, the schedule improves by 33 days (the same as the single pass).
- When Testing Overrun and Design are excluded, the schedule improves by another 33 days (a change from the 29 days of the single pass)
- When Testing Overrun, Design, and Ash Cloud are excluded, the schedule improves by another 21 days (a reduction from the single pass result of 23 days)
- The total savings when we remove the top 5 risks is 103 days, the same as when we ran them individually, but the individual savings are different.
This information is invaluable, as it allows the project team to gauge the impact removing these risks would have on the project. But what about the costs of mitigation vs the cost of the risk itself?
Mitigation vs Risk
In a previous blog, I wrote about the advantages of integrated cost and schedule risk analysis. Here, thanks to Safran Risk’s integrated model, I can review the same information as before, but on a cost basis.
The final diagram shows what our cost Analysis by Exclusion looks like for every risk: that is cost and schedule. Notice there are a few more risks shown in this display, since there are now cost risks that do not have a schedule impact included.
As you can see, it tells us that by removing the Design Specification risk and ensuring that the Outsource opportunity occurs, we can save 49 days and $295,000. At the same time, any costs associated with the mitigation strategies are also included in the model, ensuring you have direct access to all the information you need from a single dashboard.
Come back in a few days for part five in my series, where I’ll be investigating the features that make modelling risk events and uncertainties so much easier in Safran Risk.