The Scottish Government

« Previous | Contents | Next »

Listen

Appendix 1: Methodology

Scope

We examined the performance of 69 operational PPP projects in Scotland, based on surveys of public and private sector contract managers.

The study was designed to cover the following topics:

• Cost: To what extent does PPP procurement deliver overall cost savings in comparison to conventional procurement? How far can this be assessed with reference to factual data, rather than through comparisons with the assumptions used in Public Sector Comparators?
• Competition: Is there generally a healthy level of competition for PPP contracts, and hence is the lowest market price for the PPP service being obtained?
• Procurement and construction performance: How far does the use of PPP impact on procurement costs and timescales? To what extent does the use of PPP deliver projects more quickly and on budget in comparison to conventional procurement?
• Design: To what extent are PPP designs perceived to be better (or more inferior), e.g. in terms of aesthetics, functionality, environmental performance, etc? Is the relevant good practice guidance being followed?
• Innovation: To what extent do PPP contractors deliver genuinely innovative solutions?
• Operational performance: To what extent do PPP operators deliver a better (or worse) standard of service than the public sector, e.g. in terms of availability, cleaning, catering, etc? To what extent do service levels fall short of or exceed the original expectations of the contract?
• Flexibility: Are PPPs seen as being more or less flexible than standard contracts, e.g. in terms of making alterations to the asset, levels of service, etc? What has been the experience of attempting to make changes to contracts?
• Contractual Relationships: Are relationships between customers and contractors perceived to be based on a "partnership" approach or an "adversarial" approach? What has been the use and impact of penalty payments/bonuses and/or the threat of termination of the contract?
• Budgetary Issues: How valid is the argument that the use of PPP delivers benefits due to budget restrictions in the public sector (particularly capital budgets)? For example, is it valid to argue that PPP procurement brings forward investment and/or ensures that optimal maintenance strategies are followed?
• Risk transfer: To what extent is risk transferred in practice? Is it always clear where risk lies? Is there any evidence of contractors or customers seeking to shift risk onto the other party after signing the contract?

In December 2004 we sent questionnaires to the authorities and contractors responsible for each operational project in Scotland. We received responses from 56 percent of authorities and 59 percent of contractors, although note that not all respondents were able to answer all questions, particularly if they had not been involved during the project's procurement. During January and February 2005 we carried out interviews with 20 authorities and 10 contractors to explore issues raised in their responses.

We held discussions with the National Audit Office, Audit Scotland, 4Ps, HM Treasury, and Partnerships UK to understand previous and planned research in this area and, as far as possible, to ensure that our survey was comparable to and complementary to other research.

Figure 21 below shows survey responses and interviews by sector.

Figure 21: Surveys and Interviews

Figure 22 below shows survey responses by capital value and sector.

Figure 22:Survey response by capital value and sector

Other sources of data

This report has been written based on evidence from the survey and interviews together with a review of other literature. We have drawn on the following reports:

• PFI: Construction Performance, NAO, 2003;
• Managing the relationship to secure a successful partnership in PFI projects, NAO, 2001;
• Modernising Construction, NAO, 2001;
• The Private Finance Initiative: The Contract to Complete and Operate the A74(M)/M74 Motorway in Scotland, NAO, 1999;
• The Skye Bridge, NAO, 1997;
• Review of Large Public Procurement in the UK, Mott MacDonald, 2002;
• Taking the initiative: using PFI contracts to renew council schools, Audit Scotland, 2002; and
• PFI: meeting the investment challenge, HM Treasury, 2003.

« Previous | Contents | Next »