After publishing my previous blog post Through the Looking Glass, I was encouraged by the thoughtful questions and reflections it sparked. It’s a clear sign that we, as a community of project professionals, are not just consuming analysis - we’re challenging it, building on it, and applying it in the real world.
A couple of particularly sharp questions stood out, touching on how we engage teams in scenario thinking and how we communicate uncertainty to decision-makers in a way that actually drives change. These are big topics, but crucial ones. Below, I’ve shared my perspective in a response, drawing from both experience and a few hard-earned lessons.
Q: Given that your analysis points to a range of ‘most likely’ outcomes (fins), would you recommend further deep-dive studies or team role-play to see how the characters involved might react to being dropped into any of those scenarios (and if so, what might that look like)?
I would absolutely encourage any deep-dive study or team role-play. It’s so important not to fall into the trap of just spitting out fancy S-curves and tornado charts in the hope people will relate to these. I would recommend seizing any opportunity to actively engage project teams in such an exercise. Our models and reports should be covered in the team’s fingerprints if we expect the very same people to feel ownership in the results and recommendations. If you’re lucky you might have a project manager who is fluent in QRA jargon, but I have only worked for two organisations where this depth of understanding flowed all the way to the highest levels, and they were both companies where the majority of leadership had climbed the ranks working on international mega projects. It is almost impossible to know our audience or to control the onward distribution of our analysis. Just as the truth on the ground bears almost zero resemblance to the glossy pages of shareholders’ Annual Reports, so too the messaging from our reports will be massaged with spin-on-spin at each hierarchical layer.
I have found most middle management want something they can easily copy & paste up to their leadership and so my personal preference is to give them exactly that. This will not prevent someone in comms from writing a Trumpesque bumper sticker proclaiming, ‘Make Project A great again’, but experience learned the hard way has taught me this can increase your chances of the key messages making it through to decision makers. Your odds will be greatly improved by building a brand for reliable predictions but don’t expect this to happen overnight. I don’t blame our leadership for their scepticism in SRAs, CRAs & CSRAs because there are so many terrible examples out there. A small part of me dies every time I am asked to review reports showing narrow distributions with near perfect symmetry, or sensitivity plots expecting readers to lose sleep over some meaningless abstract detail. I invest a lot of time in developing CSRA models that are as simple as possible with a careful balance of honouring the 80-20 Rule/Pareto Principle for the schedule (a strong planning background really helps here) whilst ensuring 100% of the Capex and all identified threats and opportunities have activities to be mapped onto. Creating fit-for-purpose models overcomes technical issues associated with CLT (Central Limit Theorem) and provides the perfect tool for facilitating iCSRA workshops where conversations can be held at the right level in an integrated team environment.
Q: How else do you best find communicating the results of these desktop studies, e.g. how do you make recommendations to senior managers/decisions makers and ensure they listen and ideally pivot?
Hmm, great question, but incredibly tough to answer. I’ll give it my best shot… so ask yourself what is the ultimate objective of your analysis? Are you effectively making a recommendation on contingency levels – good luck depoliticizing that one! – or are you providing input to an economist that will indirectly lead to metrics that help land a decision(s), or are you writing a report to internal/external stakeholders? I will skip over the topic of contingency setting as it would be glib to attempt crossing such a minefield (I could write volumes on the treatment of uncertainty vs discrete risks) in this forum. Decision makers will often be presented with a standardised suite of KPIs covering technical, commercial and economic areas among others. They are likely to assume the economic metrics are based on a risked P50 (or perhaps even higher percentile) expected outcome but unfortunately this is rarely the case. This is not due to deliberate manipulation of data, or any deceitful practice. It is merely a failure of project managers and economists to recognize their Business Cases often combine a P50 schedule with P50 costs in the belief that this represents a true P50 picture of the Capex phasing which of course it does not! Depending on the strength of the relationship between the cost & schedule the Joint Confidence Level (JCL) of achieving the P50 CL of both the cost AND the schedule is likely to fall within the range of 25% (zero correlation) to 40% (very highly correlated) but will typically be somewhere between 30%-to-35%. As a result, our Capex profiles (combine time and money) are unwittingly spreading fake news by selling a 50% Confidence Level (1/2 chance of achieving) when the JCL is closer to a 33% likelihood (1/3 chance) of occurrence. I have heard some organisations recognise this inherent problem and so deliberately prescribe higher percentile Confidence Levels (CLs) for cost and for schedule (e.g. P60, or P70) to help ensure their Capex profiles are closer to a P50 combined outcome. I understand NASA prescribes a P70 JCL but very few projects in my sector would clear economic hurdle rates with such conservative standalone confidence levels (circa P85 CL each) in the schedule and costs. I have only just started on this campaign journey of JCL awareness myself, but I hope that by persuading project managers to embrace JCLs when phasing Capex in their economics we can help set them up for success and in so doing also lead to better decision quality. To cite the example in my recent blog, I would certainly recommend the economist runs a case for each of the ‘fins’ – the aspirational dolphin target, the threat of the mother shark and the outlier risk of being bitten by her baby!
There’s an old joke that the second most profitable thing in the world is a badly run oil company. I was always impressed by the top tier EPC contractors I worked for as they really knew how to manage risks. It was the only way to thrive and survive within such tight profit margins and I often think we would all prosper by adopting their mindset as we navigate cyclical markets that reach far beyond our limited sphere of influence.
Catch up on the original post
Are you curious what sparked this discussion? Read the original blog post, Through the Looking Glass, here.
