The Scottish Government

Listen

ISBN 0 7559 1097 4 (Web only publication)

This document is also available in pdf format (164k)

Research Aims

This report summarises the results of a research project commissioned by the Scottish Executive to:

• Provide an initial assessment of the relative costs and benefits of Public Private Partnership ( PPP) procurement in comparison to conventional procurement, and of the underlying causes of good and bad performance;
• Assess the potential merits of centrally collecting additional performance monitoring information and other data in order to inform future PPP performance evaluations, and provide recommendations on what data should be collected.

Methodology

We sent questionnaires to the authority and private sector contractor responsible for each operational PPP in Scotland. Figure 1 shows how capital spend on the 64 live PPP schemes breaks down by sector. We also interviewed a sample of public and private sector contract managers. It is worth noting the inherent limitations of an interview-based methodology, where those interviewed have a vested interest in the projects.

Figure 1: Capital value of operational PPP projects by sector

As the Scottish Executive recognised at the outset, our study covered a wide range of topics, across many sectors and can therefore only provide an initial assessment of PPP performance. Furthermore, our findings are based on Scotland's earliest PPPs and many of the lessons learnt from these projects have already been applied to more recent PPPs. Nevertheless, we believe that our research has identified some interesting areas for further research.

Main Findings

Below we summarise responses to the key questions asked by the Scottish Executive.

Cost

A Public Sector Comparator ( PSC) was prepared for 84 percent of projects and in all these cases the proposed PPP showed a saving versus the PSC. But to make a full assessment of the extent to which PPP procurement delivers overall cost savings in comparison to conventional procurement we recommend that a number of PPP and conventional procurements in the same sector are compared over a significant length of time.

Competition

We found a generally healthy level of competition for the PPP projects surveyed. The level of competition appears to have reduced in recent years. This is an area that the Scottish Executive should continue to monitor to ensure that the most economically advantageous price is achieved.

Procurement and Construction Performance

The PPP procurement process is expensive and represents a greater burden for smaller projects. The mean time taken to procure the PPP projects surveyed was 28 months, which was generally perceived to be slower than non- PPP procurement. We note, though, that the increased costs and timescales associated with procurement should be weighed against their impact on overall timing and value for money. It is also worth noting that 'bundling' of smaller projects can help to spread procurement costs across several discrete projects.

The proportion of projects delivered on time showed a significant improvement over historical experience. There was also greater price certainty during construction in the PPP projects surveyed than has been the case historically using conventional procurement. Nevertheless, on more than half of projects, the price changed at the preferred bidder stage because of a change to the design or service specification.

Design

Our survey found that authorities were happy with design quality. PPP projects were perceived to perform the same as, or slightly better than, non- PPP projects with respect to aesthetics, functionality and environment.

These perceptions reflect, in part, the fact that many authorities played an active role in the design process.

Innovation

There was evidence from the survey and interviews that PPP stimulated innovation, although authorities were less positive about the role played by the PPP contractors than the contractors themselves. As with design, we believe that the authorities' views reflected their level of involvement in design decisions.

Operational Performance

We found no evidence that PPP operators delivered a better or worse standard of service than the public sector. Authorities commented that service standards were determined by the standards specified in the contract and the budget available.

Most authorities thought that the standard of service met or exceeded expectations. Availability performance was rated highly, which might reflect the strong financial incentive on contractors to meet availability targets. Cleaning and minor repairs were cited as the soft facilities management ( FM) services where performance was least satisfactory. Hard FM services rated relatively poorly. However, we understand that this reflects some frustration on the part of authorities over the resolution of snagging issues and minor repairs, rather than concerns about lifecycle maintenance. But real issues were raised about the interaction between hard and soft FM providers in PPP structures.

Flexibility

Our survey results showed that authorities find PPP contracts less flexible than non- PPP contracts. Authorities reported that their experience of making changes has been time consuming and slow. Where changes have been implemented across a sector they have been delivered later at PPP facilities.

In interviews, authorities commented that contractual change procedures were complex and that as a result they sought to wrap up multiple changes into a single negotiation and to make one-off payments rather than changing the financial model. Some authorities expressed concerns about the costs of new works, in the absence of the ability to tender them.

Contractual Relationships

Survey evidence showed that the majority of relationships between authorities and contractors are good. In interviews it was clear that both parties recognised the benefits of developing a long-term partnership. Payment mechanism deductions are being implemented on the majority of projects.

Budgetary Issues

Many authorities commented that, at the time their project was procured, PPP was the only option given their capital constraints. This does not necessarily suggest that the PPP projects undertaken did not provide value for money, and our survey found that the majority of authorities considered PPP to represent good or excellent value for money (Figure 2).

Figure 2: Authority perceptions of value for money.

It is too early to compare maintenance strategies between PPP projects and contemporary non- PPP projects. However, our survey confirmed the observation that PPP does ensure that funds for maintenance of particular assets are effectively ring-fenced over the life of the contract.

Risk Transfer

Survey respondents believed that risks were, for the most part, allocated appropriately. More issues (for example concerns about ambiguities in the contract drafting) were raised in sectors with significant interfaces between the PPP contractor, public sector staff, users and other contractors.

Recommendations for Further Research

The evidence supports the strong view that PPP transfers construction risk to the private sector more effectively than historical procurement methods and is likely to deliver value for money where there is strong competition and the projects are large. This is clearer in certain sectors, for example water, than in others, such as health. Our survey confirmed that the well known problem of transaction costs for smaller projects is also an issue in Scotland.

There is not yet enough evidence to conclude whether PPP transfers post-asset completion risks any more effectively than conventional procurement. This should be a focus for future research. At issue is whether the potential benefits of PPP, such as whole life costing, are sufficient to outweigh its disadvantages, such as lack of flexibility, and whether there might be variations in the contracting approach that would increase net public sector benefits without losing the benefits of transfer of construction cost risks.

We recommend that the Scottish Executive's central database is extended to collect additional data on the procurement and construction phases of both PPP and non- PPP projects. This will enable real comparisons to be made about the relative benefits of PPP procurement in the areas of competition, timescales, delivery within budget, construction flexibility, risk transfer and perceptions of design quality and innovation.

The Scottish Executive should consider holding some cost benchmarking data centrally to support contract managers involved in benchmarking or market testing exercises (e.g. on the input costs for soft facilities management).

Our judgement is that the most valuable areas for further work relate to the operational phase of PPPs. In general though, we do not recommend that the procurement and construction database is extended to collect operational data at this stage. Rather, we recommend that the Scottish Executive commissions sector specific studies to assess the relative operational performance of PPP projects. This further work might include some or all of the following three elements:

• A periodic audit of all PFI schemes (say every five years) along the lines of the current work - but also looking in detail at a smaller number of projects within a particular sector.
• Detailed studies into some of the specific components of PPP contracts identified in this Study.
• Sector specific studies that seek to compare performance across PPP and non- PPP contracts. We recommend that the Scottish Executive might consider carrying out a further, ongoing evaluation of PPP and non- PPP schools, building on the work carried out by Audit Scotland. If possible, such studies should include the identification of a small number of standardised cost and performance indicators that could be collected over time, and be used in a full comparative efficiency analysis of the sample projects.

Caroline Low, Daniel Hulls, Alan Rennison
Cambridge Economic Policy Associates