What can the latest NAO Equipment Plan report tell us about the Integrated Review and beyond?
The largest review of recent years is about to release its strategic objectives and corresponding funding pathways for the Defence industry. If you are operating within the Defence sector, this has the potential to be the single most important review in the past half-decade. We have therefore taken this opportunity to review the latest audit report on the MoD Equipment plan to understand what findings were of importance to the Planning & Controls community, with a view to extracting from its findings what the key improvement initiatives and trends may be across the industry as a whole over the following few years.
What is the MoD Equipment Plan Report and what is the Integrated Review?
The MoD Equipment plan report was released in early January 2021 by the National Audit Office, whose role is to provide recommendations for improvement alongside a review of the robustness of the assumptions underpinning the MoD's Equipment Plan. This audit and its findings are onward used by parliament to evaluate the affordability of the 2020 to 2030 Equipment plan.
This report has been released at a critical time in the larger Defence industry timeline, as the long-awaited Integrated Review is set to be finalised imminently following a slight delay from the original timeline. It will outline the strategic (re)priorities and rebaseline the spending accordingly for billions of pounds of funding. These significant shifts in focus and the rebalancing of the portfolio tend to happen cyclically, with the last significant shakeup being the Strategic Defence and Security Review (SDSR) in 2015. More recent reviews have been conducted, but none to the scale of the SDSR 2015 and the forthcoming Integrated Review.
We can already predict big changes to come from the Integrated Review, as the November 2020 announcement by the government, injecting a further £16.5 billion of defence funding over the next four years was a definitive precursor to partially address the funding challenges that the Defence Industry has been wrestling with for the past few years. However, that additional funding comes at a price - and that price is to fulfil the expectation that efficiencies will be maximised and focus will be on ensuring a realistic package of savings measures along with a keen focus on technological advancement across the industry as a whole.
What does this mean for the Planning & Controls community?
The NAO report has shone some light on the areas that it feels are in requirement of improvement to ensure cost efficiency and robustness of assumptions and plans in its report, and we have summarised our interpretation of how these may shape the Planning & Controls domain over the next few years.
The good news is that there seems to be a real focus on quality Planning and Controls at the epicentre of strong robust delivery. It is clear from the report that Planning, Risk and Financial forecasting are of paramount importance to get right and improve upon, with Governance and Assurance playing a pivotal role in ensuring the correct balance is struck in a consistent and transparent manner.
The bad news is that as an industry we have a long way to go to improve our systems and processes, and the report findings are critical in their assessment of what they deem as a largely over-optimistic plan, assumptions and cost forecasts with significant challenges both in terms of funding and deliverability.
So what can we take from this and what are the major improvement points? How will this shape our industry going forward? We will discuss in our view, the five key takeaways below from the report, and what we believe will be the investment and progression areas within our industry over the forthcoming years;
1. In Year Financial Planning
One of the trickiest elements of large scale planning of this nature is the different financial and political windows that require adept navigation.
At the micro-level, budgets are often locked down by departments on an annualised in year process, forcing short term financial management decisions to be made to "balance the books" rather than making the right choice for the long term benefit realisation of the portfolio.
The NAO Report finds that
[this short term focus has] "led to higher overall costs and larger funding shortfalls in later years" [combined with the mandate of] "stopping projects or deferring expenditure into later years".
They conclude by stating that this short-term approach to "manage costs is increasingly restricting their ability to develop the capability they need"
On the macro level, we also need to take into account the political climate. Most of these programmes span multiple different government cycles, and with changes in priority accompanying changing political parties, it is an endless and complex balancing act between replanning and delivery against a set baseline criteria of long term benefits and capabilities.
Both of these factors are unlikely to change anytime soon, and they provide the uniquely complex landscape of large complex engineering programmes.
As Planning & Controls professionals it is, therefore, our duty and service to continually challenge
these short-term deferral decisions when there is a clear cost-benefit analysis to indicate that the longer-term capability solution will be significantly impacted by this short-term knee-jerk reaction to balance the here and the now.
We are realistic that not every decision will be the right one, and sometimes deferral is inevitable even though it is not the correct long-term decision for reasons outside of the programme teams control, however regular conversations around long term capability delivery and benefits realisation held regularly at all levels of the supply chain may help to start to improve and influence decision making within all stakeholder groups. As servant leaders within our organisations, we should see ourselves as a leading voice in facilitating these discussions.
2. Optimism and Forecasting
The NAO report concludes that the overall equipment plan lacked realism with over-optimistic and inconsistent judgements with forecasting costs. In fact, it stated that the Cost Assurance and Analysis Service (CAAS) estimates that costs were understated by £3.9 billion.
When the report concludes its recommendations on improving the consistency of judgements which underpin budgets and assessments of forecast costs, it summarises three core areas, which we believe are firmly within the Planning & Controls domain. We will cover the first point of its conclusion here, and the remaining two points under Schedule and Cost Risk Analysis below.
The first recommendation the NAO states is that the plan should
"ensure all adjustments to cost forecasts are fully evidenced and transparent and supported by a clear rationale"
For there to be clear rationale and evidence for budgeting there must be an iron-clad linkage between planning, finance and risk.
This means in essence that you have fully burdened schedules, accurately scheduled with clear risk profiling linking into that central planning toolset. Without full data and process integration between your cost system, scheduling system and risk management true transparency and evidence links cannot be ascertained. Relying on a plethora of spreadsheets and siloed systems needs to become a long and distant memory within the Planning & Controls industry as a whole and not just in its giants. Unfortunately, however too often we have witnessed highly immature Planning & Controls frameworks with standalone Financial systems and a standalone Risk Management process - this is a recipe for false hope and unrealistic forecasting, and when compounded and rolled up to the highest levels such as this equipment plan, the impacts of these fundamentals not being in place throughout the supply chain shine through in a painful way.
There are patches of absolute excellence in this type of integration and holistic Planning & Controls maturity, however, the Equipment plan will only ever be as robust as its weaker links therefore as a community we should challenge cross-industry improvements that flow both up and down the supply chain - We need to challenge decision-makers in our role as Planning & Controls professionals to ensure that integrated planning & controls are the driving force behind change and progress, rather than the last to do on a teams list as delivery is king. The dichotomy of an organisation that does not invest in improving their Integration both in terms of data and process is that delivery won't be fully optimised without it, or if it is, it is often by luck rather than design, and that luck will always run out at some point.
3. Schedule and Cost Risk Analysis
Undertaking robust schedule and cost risk analysis is a challenge across the industry, and it has been a long-standing methodology within the sector to estimate most costs and schedule at the 50th percentile, which means in basic terms that the project is just as likely to cost more and/or overrun than it is likely to under-spend and/or be delivered early. The NAO has highlighted this practice as an area for significant improvement for many years and draws the conclusion that forecasting at the 50th percentile irrespective of complexity and maturity levels of the project is not always an appropriate course of action and this year's report repeats its findings and highlights this as a key area for improvement.
Progression towards a more effective and realistic cost and schedule forecasting process is slow within the industry, with pockets of higher risk projects such as the Dreadnought submarine programme and the Type 26 shipbuilding programme taking on a more prudent 70th percentile estimating process to reflect the likelihood of cost increases due to their complexities, but most still following older processes sticking to P50 estimates as the norm.
Unfortunately, with practice as embedded as this, it is going to be difficult and slow to attain a full root to tip transformation in the planning and controls domain with estimating and confidence levels, as the level of analysis and validation of a P70 plan and cost forecast is, we argue, much higher than that which is required for a P50, therefore for the industry to decrease its level of uncertainty within forecasting it has to significantly increase its focus on planning and controls, which will increase the level of spending required in this area for organisations. Although on paper this seems like a good and positive direction for the industry to aim toward, it can only be done through effective collaboration cross supply chain, and there are often many commercial reasons why organisations choose not be fully transparent with their cost and schedule forecasting.
The challenge also brings with it a significant uplift requirement for competence and skillsets - not all companies within the industry have access to the specialisms needed to uplift their current teams. Ensuring all teams within a major programme, irrespective of organisation, are SQEP to conduct robust cost and schedule risk analysis is crucial for the department if it is to attain true P70 results in this domain. We predict this will be a slow burn, with painfully slow progress toward higher confidence plans.
4. Data and Analytics
The huge advancements in options to increase capacity for data integration and exploitation within project and programme deliveries is a trend that will continue to grow over the following years. It is therefore no surprise that the NAO report concludes that the department and wider industry need to focus on strengthening their analytical capabilities. Perplexingly, the report focused on the need for the development of staff with financial qualifications alongside improvements in its use of data, however, we take a view that this shift should be much more fundamental at its core.
Most organisations that exist within this complex defence space have vast swathes of data within their footprints but gaining true integration and exploitation remains a challenge. The main issue we find that time and time again is that this rich landscape of data is unsorted, unprocessed and lacks a common integration strategy - there is then an overarching culture of departmental silo working which further compounds the issue.
Establishing core common principles and an overarching data strategy to govern what good data looks like together with a clear map of how data flows in and out of systems and teams will help to establish a single source of truth and transparency through data and systems. The old silo-ed working of the past must be torn down to make use of much more integrated and collaborative efforts from cross-functional teams. We agree that there is always room for specialisms, with financial forecasting only one of a plethora of specialisms needed in the planning and controls space, but it is only when there is a recognition that all data is and requires to be integrated in a transparent way that we believe that progress can be truly made in this domain.
Creating true data analytics and exploitation that can be relied upon takes time and investment from the organisations. The initial foundation of good cleansed and sorted data takes time and is often invisible in terms of tangible outputs and benefits until the integration is much further developed. However, we believe the more time invested upfront in data cleanse and mapping activities, the greater the level of integrity and trust the eventual data analytics can command.
5. Quality and Assurance
Finally, and the most important element which binds all of the aforementioned elements together is quality and assurance within Planning and Controls.
The NAO report concluded that "neither could demonstrate that their quality assurance work had ensured that the Plan was based on consistent and fully documented data and assumptions".
This in part will be due to the lack of transparency of the data and the lack of full integration across the stakeholder communities, but it is also a strong indicator that further improvement needs to be sought in the type and frequency of surveillance and governance style quality and assurance reviews.
Elements such as Integrated Baseline Reviews and other such assurance reviews are in-place where there is often a contractual requirement inbuilt for the team to undertake said assurance, however, we believe that all projects and programmes benefit from such reviews (tailored where required) and as Planning & Controls professionals we should strive to provide that facilitation role in conducting these type of surveillance and benchmarking assurance activities within our teams and out through the wider business. Without robust quality and assurance, each step of the recommendations acts in isolation from one another. True quality and assurance action with iterative improvement loops help to drive improvement and gain more consistency in outcomes.
Comments