Getting the KPIs right: The critical first step in Asset Acceptance
Best practices in systems engineering and test consider the definition and alignment of an asset’s functional, operational and HSE performance requirements — and the associated key performance indicators (KPIs) — to be the critical first step in any acceptance procedure.
The reason is the direct link between requirements and KPIs:
- The requirements define how the asset should operate, and meeting the requirements is the basis for any acceptance decision.
- The KPIs define what to measure to determine if the requirements are met.
Because of this direct link, KPIs must be carefully derived from requirements. The most common mistake in executing an asset acceptance program is to ignore this link by assuming that simply stating the requirements is enough to ensure a valid acceptance program. If the KPIs are not aligned with the requirements, there is no effective way to ensure the requirements are in fact met.
Getting the KPIs right is a 4-step process:
(1) State the requirements and align the KPIs
For each requirement there needs to be one or more specific KPIs identified that can tell us if that requirement can be met. Most operating assets will already have a set of KPIs in place to measure and track functional, operational and HSE performance. Each requirement should be aligned with one or more existing KPIs. Requirements that do not align with an existing KPI represent a gap where a new KPI must be defined to satisfy the requirement.
If a KPI cannot be defined, there is a problem with the requirement itself. By definition, a requirement must be measurable. If no measure (KPI) can be defined, then the requirement is invalid and must be either redefined or discarded.
(2) Validate what is measured to satisfy the KPI
KPIs can be either backward looking (reactive) or forward looking (predictive). The choice of using a reactive or predictive KPI is highly dependent on the desired outcome. For an asset acceptance, the predictive KPI is more useful. A predictive KPI would indicate the probability that the asset is going to continue to be compliant with the requirement. A reactive KPI would simply indicate the asset was compliant in the past. Reactive KPIs are interesting and informative, but for acceptance, you are much more interested in how the asset will perform for you than how it has performed for someone else.
For example, consider the US GOM well control regulation that the BOP be capable of shearing all tubulars that will be used in the drilling plan. A strictly reactive KPI would rely on reviewing the the design and testing that prove the BOP is capable of shearing the tubulars. This would indicate that the shear rams were capable of shearing the tubular by design, but provides no indicator that it actually will shear when required.
A proactive KPI would also include a review of the maintenance systems and ongoing inspection and test protocol that keep the shear ram functional. The proactive KPI indicates the shear rams will continue to be able to meet the regulation in the future.
(3) Validate HOW the KPI is measured
In order to be useful, a KPI must be stable and well characterized. For a typical instrumented KPI such as a pressure gauge reading or a cycle count, this is relatively simple to validate. However, process oriented KPIs such as maintenance, inspection and test history are highly dependent on the people and process support tools that were involved.
In some cases, the KPI process stops here as it is often not possible to identify a stable, characterized KPI. There are several possible reasons for this including missing sensors, sensors that do not have the accuracy or sample rate necessary for the measurement, and poor manual record keeping.
For example, if no maintenance records are kept on the asset, then it would be impossible to determine if the shear ram is maintained in a way that keeps it in compliance with regulations. In this case, the original KPI is simply not valid and therefore the original requirement may in fact be called into question. If you can’t trust the measurement, it can’t be used as a KPI.
(4) Determine the acceptable boundaries for each KPI
The last step goes beyond defining static limits for what is acceptable and what is unacceptable. Processes have normal variations so the acceptable levels must include a definition of acceptable variances as well as how much data is required to establish when a positive or negative trend is developing.
In summary, Athens Group’s Proven PracticesSM Methodology for Asset Acceptance implements a five-stage process that starts with the alignment of requirements and KPIs (as described above) and then follows a natural integrated system-oriented flow. Each stage builds upon the prior stage to ensure that the asset’s full integrated systems capability is validated against functional, operational and HSE requirements.
Copyright 2016 Athens Group. All Rights Reserved.
SUBSCRIBE to our TECHNICAL NEWSLETTERS! Click here