Operations & Maintenance Toolkit: Performance Monitoring

Performance Monitoring 

In a public-private partnership (P3), ensuring long-term success requires thorough planning and oversight during the operations and maintenance (O&M) period. Performance monitoring is critical to maintaining the standards set forth in the P3 agreement and achieving desired outcomes throughout the project lifecycle. This process starts with a well-executed procurement, where the selection of a qualified O&M provider is based not solely on cost but on a “Best Value” approach, considering experience, resources, and the provider’s ability to manage the asset effectively. The following content delves into key elements for establishing performance expectations and ensuring ongoing accountability through detailed operations and maintenance documentation, financial assessments, and technical approaches tailored to the specific needs of each P3 project.

O & M Documentation

A successful procurement process and the selection of a qualified proposer are key to ensuring adequate performance throughout the life of the P3 agreement. Regardless of the procurement structure, whether Progressive P3 or more traditional two-step RFQ/RFP process, solicitation documents must include a comprehensive set of submission requirements and corresponding evaluation criteria that cover all phases contemplated in the contract: predevelopment (if applicable), design, construction, operations, maintenance, and/or rehabilitation. 

One of the many advantages of a P3 is that selection is based on the concept of “Best Value” rather than selection of the lowest bid. In addition to price, the Best Value concept considers additional factors in the evaluation and selection of a qualified proposer such as the O&M Service Provider’s capacity, resources, experience with projects of similar scope and complexity, key personnel to be assigned to the project, and technical approach to operations, maintenance, and rehabilitation. 

Prior to developing submission requirements and evaluation criteria, it is recommended that the following items are completed: 

    • Project Scope: The extent of the project scope should be fully defined and understood. 
    • Data Collection: Full understanding of all available existing information such as prior studies and plans to assess the need for additional studies, which may be included in the project scope. 
    • Procurement and Contractual Structure: Whether a traditional design-bid-build, design-build, or a design-build-finance-operate-maintain, the project delivery structure will dictate the contents of the submission requirements and evaluation criteria. 

Qualifications

Determining whether a proposing O&M service provider is qualified to do the job is critical to the success of the O&M period. To that end, evaluating their experience and past success in projects of similar scope and complexity is essential. Generally, the solicitation documents should request the items listed here from proposing O&M service providers. Note that this list is not exhaustive, and additional submission requirements should be developed and evaluated based on the unique requirements of each project. 

    • O&M Service Provider’s Information: Includes the provider’s legal name, management structure, and organizational chart showing internal relationships, assigned point of contact, and contact information. This should also include an organizational chart showing the context and relationship of the O&M service provider to other members of the responding team during the O&M period and other phases of the project (as applicable), including subcontractors. 
    • O&M Service Provider’s Reference Projects: List of projects detailing the project’s name, location, scope of work description, owner’s name and contact information, and project references with names, titles, and contact information. 
    • O&M Service Provider’s Key Individuals: List of provider’s nominated staff who will hold key positions during the provision of O&M services, with names, company titles, project titles, and roles, and resumes demonstrating the professional experience of each nominated individual on the nominated role and on projects of similar scope and complexity. This section should also include an organizational chart demonstrating reporting relationships among the O&M Service Provider’s key individuals. 
    • O&M Service Provider’s Financial Capacity: A critical issue in P3s is the selection of an appropriate private sector partner. Given the long-term uncertainties and extensive risk portfolios associated with these projects, the O&M service provider faces significant challenges in maintaining contractual standards over periods ranging from 30 to 99 years. To enhance the risk management process and assess the financial capacity of the concession, several key factors are addressed
        • Long-Term Financial Stability and Compliance with Covenants:  The financial stability of the O&M service provider is essential for sustaining operations throughout the contract period without encountering financial distress. P3 contracts often impose stringent performance and maintenance standards. A financially capable O&M provider is more likely to adhere to these standards, thereby avoiding penalties and ensuring that the project achieves its intended objectives. 
        • Risk Management: Financially robust O&M providers are better equipped to manage uncertainties and economic troughs. This includes addressing unexpected costs, economic downturns, and other financial challenges that may arise during the project lifecycle. 
        • Quality of Service: A financially capable O&M provider is more likely to invest in high-quality resources, including skilled personnel, advanced technology, and efficient processes. This investment leads to superior service delivery, continuity of operations, lower turnover, and training costs. 
        • Investment in Upgrades and Innovations: A strong financial position enables the O&M provider to invest in upgrades and innovations over the contract period. Such investments can result in improved efficiency, reduced operational costs, and enhanced service quality. 
        • Stakeholder Confidence: The financial capacity of the O&M provider instills confidence among all stakeholders, including public authorities, investors, and the public. This confidence is vital for sustainability of the P3 project. 
        • Financing and Insurance: Financially strong O&M providers are more likely to secure favorable financing terms and insurance coverage. This can reduce the overall cost of the project and provide a safety net in case of unforeseen events. 

Technical Approach

Along with understanding the qualifications of proposers, it is also important to evaluate the O&M service provider’s project understanding and technical approach to each aspect of the O&M period—operations, maintenance, and rehabilitation—to understand whether their philosophy and approach aligns with that of the owner. It is important to note that the amount and depth of technical submittals requested should depend on the type of procurement (one-step versus two-step process) and the procurement timeframe. 

The amount and depth of technical submittals will depend on the type of procurement (whether a one-step or a two-step process) and the procurement timeframe and may generally consist of the items listed here. Note that this list is not exhaustive and additional submission requirements should be developed and evaluated based on the unique requirements of each project. 

    • Overall Approach to O&M Services: A narrative describing the provider’s overall approach to the provision of O&M services, the management of subcontractors, quality and performance monitoring, reporting, the management of overlapping services and corresponding interfaces with the public agency, and the management of scheduled and unscheduled O&M work and their efforts to minimize or mitigate any impact.  
    • Approach to Handback: A narrative describing the provider’s approach to ensuring compliance with handback requirements set forth in the P3 agreement and Performance Specifications. 
    • Helpdesk Management and/or Customer Service Approach: A narrative describing the O&M Service Provider’s approach to the management of helpdesk services or customer service, how the helpdesk service will be integrated with their CMMS, and how the O&M Service Provider will monitor response times and rectification times.  
    • Environmental Management Services: Environmental Management Services and the topic of sustainability and resiliency are becoming more prevalent, and many public agencies are requesting the integration of these services into the management and operations of infrastructure assets during the O&M period with the goal of leveraging long-term costs. To that end, the submission requirements for the technical approach may include a request for a narrative describing the O&M service provider’s approach to the management of environmental and sustainability services and how such services will be integrated into their O&M services. Could probable be précised 
    • Utility Management Services: A narrative describing the O&M service provider’s approach to the management of utility services, plan to meet any energy efficiency requirements, and projected energy models. 

O & M Performance Specifications

One of the advantages of the P3 structure is its performance-based approach to defining the project specifications. In a traditional design-bid-build, the Architect of Record will prepare, along with the design of the asset, a set of project specifications that is prescriptive in nature and specifies materials, colors, finishes, brands and models, and means and methods of installation, along with specific standards and codes. 

Project specifications for P3s, on the other hand, are performance-based. Rather than prescribing specific materials, finishes, colors, brands, models, or installation means and methods, performance specifications describe the outcome. As a result, performance-based specifications encourage key value drivers during the design and construction process through the introduction of innovations that result in design, construction, and operational efficiencies. 

Performance specifications for P3 projects generally consist of specifications for (a) design and construction, (b) operations, maintenance, and rehabilitation, and (c) Handback or P3 agreement expiry and are typically a separate schedule to the P3 agreement. 

Operations, maintenance, and rehabilitation: the drive for accountability and monitoring. OM&R is a separate and largest part of the P3 agreement as it addresses two relevant concepts: (a) maintenance responsibilities and requirements, and (b) performance payments (linkage to the payment mechanism).  

OM&R has three components: (a) operations and maintenance performance requirements, (b) asset preservation (rehabilitation / renewal), and (c) expiry or handback requirements.  

Must be developed so that the concessionaire chooses its own approach to durability in order to get best value from life cycle optimization.** This needs worked into better style. 

Key Performance Indicators

Performance-based payment contracts often utilize Key Performance Indicators (KPIs) to measure and evaluate the performance of a service provider. KPIs help ensure that important service delivery factors are continually met and that any issues are tracked for improvement and/or service deduction penalties. The client sets KPIs and is a determining factor in payment mechanisms within the Project Agreement. Performance Monitoring Reports are provided each month, detailing the applicable KPIs and providing statistics relevant to their performance. This report is reviewed in monthly meetings with the client, during which continuous improvement steps will be taken, and standard operating procedures will be refined to improve any service inefficiencies.  Although client needs may differ, creating a wide range of KPI interests, the following are the most common KPIs: 

    • Comfort conditions: Ambient temperature and relative humidity (70–73F and 30-55% RH). 
    • Client Call Availability: Help desk response within 3-5 rings. 
    • Response Time Verification: Meeting agreed on response time and rectification times. 
    • Uptime of Equipment: Reported equipment failures within tolerance levels. 
    • CMMS planned maintenance: 90% achievement in monthly maintenance plans. 
    • Energy Performance: Meeting agreed energy targets with annual pain share/gain share recognition. 

Should KPIs not be met, the client can impose service penalties, which are deducted from the Project Company’s monthly payment until they are rectified. These imposed penalties are non-negotiable, and the Project Company rolls down the deduction to the Service Company’s payment. 

Usually, these service penalties have a sliding scale severity, including default and, ultimately, contract termination should service performance remain unacceptable. Financial surety packages (Line of Credit) held by the client are cashed to procure services with another party following default. .