The Scottish Government

« Previous | Contents | Next »

Listen

6 VALUE FOR MONEY ASSESSMENT

6.1 Background

6.1.1 The following chapter presents a value for money (vfm) assessment of the principal programmes targeted at the business, domestic and public sectors. The development of a vfm analysis provides a useful means of comparing different government sponsored programmes aimed at achieving the same outcomes.

6.1.2 The chapter reviews the inputs, activities and outcomes produced by the programmes to provide an indication of their scale and resultant impacts. The key cost effectiveness ratio, which is presented in this chapter, relates to a comparison of inputs in the form of financial expenditure by the Scottish Government to outcomes relating to the estimated lifetime carbon savings in tonnes generated from this expenditure.

6.2 Limitations of cost effectiveness ratios

6.2.1 A number of issues are raised when attempting to assess the cost effectiveness of energy efficiency and microgeneration programmes. There is a need to be clear about what is actually being measured and the methodology used to determine how much carbon emissions are avoided as a result of increases in energy efficiency and the implementation of microgeneration schemes.

6.2.2 The cost effectiveness ratio used in this vfm analysis is a straightforward means of ranking policies by comparing the financial cost to government (input) per tonne of carbon saved (outcome). This provides an indication of the relative cost effectiveness of different government programmes in meeting an overarching strategic government objective of reducing carbon emissions.

6.2.3 Given the complex nature of assessing the cost effectiveness of public support through a standardised £/tC ratio, is advisable that the reader exercises caution when interpreting the cost effectiveness figures outlined in this chapter. There are a number of caveats which are raised below which should be considered when seeking to interpret the standardised cost effectiveness measures.

6.2.4 Firstly, the ratio does not take account of any other private financial costs arising as a result of the policy (such as to households and firms), and does not assess any benefits which may arise. This means that the cost effectiveness focuses purely on public costs rather than the total net costs involved in the implementation of carbon savings. For example, the provision of advice and auditing services generally rely on the beneficiary implementing the recommended measures using their own resources, and the resultant reductions in carbon are then attributed to the advice and auditing services provided. This means that programmes which focus on advice and auditing appear more cost effective (from the point of view of government) as the private costs involved in implementation are not included in the ratio. However, as the focus of this analysis is on the targeting of public expenditure, this approach is perfectly reasonable.

6.2.5 Another issue to consider is the extent to which the carbon savings accruing to different programmes can be fully attributed to the financial support from the Scottish Government. There are a number of issues here in relation to the type of support that is provided as well as the level of leverage from other sources. In the case of the provision of advice and auditing services, some of these services also leverage in funding from other sources such as Defra. Where the funding is from elsewhere in the public sector, it may be appropriate to apportion outcomes on a pro-rata basis between different funding sources. These modified ratios are specified where appropriate.

6.2.6 Also, the ratios presented compare financial costs in one year only with the lifetime carbon savings arising from this annual expenditure. The estimate of lifetime carbon savings is based on the annual savings generated multiplied by the assumed lifespan of the technologies. This will vary depending on the type of technology being assessed. For example, the assumed lifespan for loft insulation is much longer than for heating systems. In line with Defra guidance ⁴, the estimated lifetime carbon savings have not been discounted in this cost effectiveness analysis and all calculations which were presented as CO2 rather than carbon have been converted into units of carbon equivalent using the conventional conversion factor of 12/44 ( e.g., Y tonnes of CO2 are equivalent to 12/44 x Y tonnes of carbon).

6.2.7 The methodology used to estimate carbon savings arising from EST consumer activity is based on a sample survey, undertaken by its in-house Evaluation Unit. Respondents are asked to state the extent to which the implemented measures were attributable to the information and advice provided by the EST. Carbon savings generated in Scotland from consumer marketing activity are based on a pro-rata share of UK survey results, while carbon savings arising from EEAC activity are based on a specific survey of Scottish beneficiaries. A standard ratio is used to estimate the carbon savings arising from each implemented measure. This does not account for any local variation in the household fuel mix.

6.2.8 Additionality of support is assessed in terms of the carbon savings accruing in the household sector by asking respondents the importance of EST advice in relation to the implementation of measures. However, there may be instances of double counting as the existence of a public grant may also be required in order for some measures to be implemented. This is perhaps most relevant when attributing carbon savings arsing from the information and advice provided for the implementation of microgeneration systems. However, this currently accounts for a very small proportion of the total estimated carbon savings attributable to EST support.

6.2.9 The methodology used by the CT relates to actual implemented savings across all companies supported. As the CT focuses on a much smaller number of larger beneficiaries, it is possible to provide more accurate figures relating to the actual carbon savings arising from their activities rather than being based on a sample. The CT is also the only organisation in the UK where its methodology for calculating carbon savings has been externally audited. The BEP and ESTBAN also base their estimates for carbon displacement on guidance from the Carbon Trust.

6.2.10 In terms of the support to businesses, it is assumed (by the delivery agents) that all of the savings implemented as a result of the auditing services provided are attributable to the programmes. However, our survey does indicate that some of the measures would still have been implemented without the intervention in the form of the energy audit.

6.2.11 All of the carbon emissions reported against specific programmes relate to implemented rather than identified carbon savings. This requires there to be follow up between the delivery agent (such as the CT, EST and BEP) and the beneficiaries in order that the actual carbon savings accrued through support can be assessed. In the case of the CT and BEP, the level of implementation is assessed among all beneficiaries, while ESTBAN has used a large statistically representative sample of the businesses, which covered 70% of all beneficiaries in the programme.

6.3 Cost effectiveness of support to businesses

6.3.1 Performance data in relation to the three key organisations delivering energy efficiency support to businesses is summarised in table 6.1. This has been based on data produced for the resource efficiency review undertaken by the Scottish Government, supplemented by data form the EST in relation to the BAN. The figures relate to the direct support delivered by these organisations to businesses during 2005/2006, with the exception of the BAN which relates to 2006/2007. Where ranges were provided (as in the case of the Carbon Trust) the midpoint of this range has been presented in the table.

6.3.2 Over 70% of the Scottish Government funding for these three organisations was allocated to the CT, being provided with £2.5m to deliver support to businesses in Scotland. This was followed by around £612k allocated to the EST for the delivery of the Business Advisor Network and £400k to the BEP.

6.3.3 It should be noted that in addition to funding from the Scottish Government, a number of these programmes also leverage funding from other sources. For example, the Carbon Trust in Scotland benefits from the economies of scale of being part of a UK wide organisation. Defra provides funding for UK wide core activities such as publications, website and site survey administration. Scotland's share of this funding amounts to around £1m. The BEP also leverages funding from other sources (£0.5m), both from the public and private sector, to deliver its range of services. This is not only focused on direct support to businesses in the form of resource audits, but also includes a range of other initiatives and environmental training and awareness raising events.

Table 6.1: Annual performance of principal energy efficiency programmes in Scotland

6.3.4 The overall carbon displacement reported by the three organisations amounted to nearly 280,000 tC, over the lifetime of the implemented projects during the year, of which 95% was attributed to programmes operated by the CT. This was followed by 3% from the BAN and 2% relating to the BEP. On average, the CT led to lifetime savings of around 520 tC per business, compared to around 40tC for the BEP and 17tC for the BAN.

6.3.5 In terms of comparing financial inputs with the outputs and outcomes generated by the support, the cost effectiveness ratios vary significantly between each of the organisations. Support delivered by the CT is the most cost effective, costing £10/tC over the lifetime of projects (Table 6.2). The focus on higher energy consuming businesses, where the potential for carbon savings is significantly higher, means that cost effectiveness of support can be maximised. Taking account of Scotland's pro-rata share of Defra funding, the ratio rises to £13/tC.

6.3.6 The BAN and BEP exhibit much less cost effective ratios ranging between £58/tC and £63/tC for BEP and BAN respectively. The focus of the BAN is on the provision of energy audits to companies with a fuel bill of less than £50K. This market is not served by the CT, precisely because it is not felt to be cost effective.

6.3.7 In addition to the carbon impact of support, the BEP also generates an output in terms of waste minimisation, which does not directly contribute to the outcome relating to carbon savings. Therefore, the funding to the BEP contributes to waste reduction targets relating to the EU Landfill Waste Directive in addition to its contribution to lowering carbon emissions.

Table 6.2: Cost effectiveness of energy efficiency programmes from annual expenditure

1.1.3 The results from the business survey provide an indication of the additionality of the carbon savings generated by each of these three organisations. The findings suggest that additionality is lowest for the energy audits provided by the EST Business Advisor Network, as 35% of firms that received a BAN energy audit indicated they would have implemented all of the measures without the support. This compares to 20% for the CT and just 10% for the BEP.

6.3.8 The lower additionality of the BAN energy audits suggests that for a larger number of businesses, the energy efficiency measures recommended as a result of the audit are more self-evident, and are more likely to be implemented anyway by the business. This would suggest that in a number of these cases face-to-face audits are not necessary, and more could be done with these businesses remotely, such as through provision of advice over the phone, or through the promotion of more self-help guidance.

6.3.9 Feedback on the BEP suggests that it has an effective business model, which could be used more widely as a model for the support structure for business support. An ongoing personal relationship, which facilitates implementation through ongoing advice and guidance over the phone, was felt to be an important aspect to the model, reflecting its higher level of additionality compared to the BAN and CT.

6.3.10 In terms of the loan funding available to the business sector, 44 loans were issued in 2006/2007 through Loan Action Scotland, compared to one CT loan during the same period (Table 6.3). The CT loan was provided to a charitable organisation which at the time of application, did not qualify for LAS funding. Subsequently, the LAS loan criteria have been extended to include charitable organisations.

Table 6.3: Annual performance of Loan Action Scotland and Carbon Trust Loans

6.3.11 The cost effectiveness of LAS loans has improved considerably over the period of operation from a peak of £60/tC in 2004/5 (Figure 6.1). Across the UK as a whole, the CT interest free loan scheme reported a cost effectiveness ratio of £30/lifetime tC during 2006/2007.

Figure 6.1: Loan Action Scotland carbon cost effectiveness, 2000 - 2007

6.3.12 The Wise Group reported that a combination of factors have led to this drop. Firstly, there has been an increase in applications for technologies which offer higher carbon savings such as biomass systems. Secondly, while there has been an increase in the volume of loans issued compared to the previous year, the processing and administration costs have not risen to the same extent. The scheme benefited from a more simplified financial appraisal for applicants using a credit profiling service, which was recommended in the LAS evaluation and brings it in line with the system used by the CT loan scheme.

1.1.4 Our business survey included five firms that had received a loan from LAS. The firms indicated that two out of the five loans provided to these firms were non-additional as these firms would have implemented the measures anyway without the loan. The results from the Evaluation of the Loan Action Scotland programme suggest that 20% of loans were fully additional, 64% were partially additional (timing or scale additionality) while the remaining 16% would have happened anyway. Non-additional loans would tend to reduce the carbon impact of public funding allocated to this programme, as the carbon savings would have accrued anyway without public intervention.

6.4 Cost effectiveness of support to public sector

6.4.1 The following table brings together some information on the two revolving loan schemes available to the public and higher education sector in Scotland, CEEF and Salix Finance. Performance data on CEEF has been obtained from the evaluation of the scheme undertaken in 2006, while data on Salix was provided by the CT for the two loans issued through this scheme in Scotland in 2006/2007.

6.4.2 The cost effectiveness of CEEF is reported as being on average £66/tC across the portfolio of 537 loans (which were issued over the period from the launch of the scheme to September 2006) (Table 6.4). The cost effectiveness ratio effectively treat the revolving loan as a grant, given that the funding will not be reclaimed by the Scottish Government, but will be recycled by the local authorities and re-invested in new projects. The difference between this scheme and a direct grant, is in the fact that the savings accrued through the investments (equivalent to the initial loan amount) are ring fenced to be re-invested in further energy saving opportunities. With a grant programme, the local authority would have greater discretion to invest these savings in other areas.

6.4.3 The ratio for Salix is not directly comparable with CEEF as the method used for calculating cost-effectiveness for Salix is based on the opportunity cost of capital. This takes account of the fact that the scheme is a loan rather than a grant programme. The costs relate to the interest forgone as well as a share of the administration costs of the scheme. Salix operates as a source of match funding, and so all projects must obtain funding of 50% from other sources. This is not a condition of CEEF, and so it is the discretion of each local authority, how much additional funding is leveraged into the project from other sources.

Table 6.4: Overall performance of CEEF and Salix Loans

6.4.4 Feedback from the survey of public sector organisations highlights the highly additional nature of CEEF funding. It was felt that none of the projects discussed would have happened without the existence of CEEF funding, given the capital borrowing constraints apparent within these public sector organisations.

6.4.5 One of the means through which the cost effectiveness of the CEEF scheme could be increased would be for a greater proportion of projects to be focused on more cost effective measures such as insulation. As is shown in figure 6.2, cavity wall insulation is the most effective measure, generating lifetime carbon savings of around £20/tC, although it only accounted for a small proportion (4%) of all projects. However, increasing the number of projects focusing on improving the insulation envelope of buildings will be dependent on identifying suitable properties where these measures can be implemented.

Figure 6.2: Lifetime cost effectiveness and CEEF project implementation by technology

6.4.6 Over a quarter of projects were focused on building controls, the largest share of any one measure. This measure would generate lifetime carbon savings of around £80/tC, four times less cost effective compared to cavity wall insulation. While the building control measure may perform relatively well in terms of a payback criteria, given its shorter lifespan compared to insulation, investment in this measure is less cost-effective over a longer term period.

6.5 Cost effectiveness of support to household sector

6.5.1 The EST has received between £1.7m and £2.0m annually over the past three years to deliver energy saving information and advice to the domestic sector. This has included items such as awareness raising campaigns and online advice delivered centrally by EST as well as telephone advice and event-based promotions delivered through the network of EEACs. Overall, this generated around 400,000 contacts within the Scottish population during the three year period 2004/2005 to 2006/2007. (Table 6.5)

Table 6.5: Performance of EST household programmes, 2004/5 - 2006/7

6.5.2 In terms of the outcomes generated from these activities, the Evaluation Unit in EST reports that its activities have resulted in lifetime carbon savings of over 600,000 tonnes, from activities undertaken during this three year period. This is based on sample evidence of beneficiaries, where respondents are asked to state the influence of EST on subsequent behaviour, resulting in energy savings. Carbon savings generated from consumer marketing activity are based on a pro-rata share of UK survey results, while carbon savings arising from EEAC activity are based on a specific survey of Scottish beneficiaries.

6.5.3 Results from our household survey indicate that information and advice provided by the EST had the greatest impact with relation to microgeneration, where around 40% of all those installing these systems would not have done so without support from the EST. Additionality was much less in relation to the implementation of behavioural changes (11%) and implementation of energy efficiency measures (13%). Nevertheless, these results would still indicate that over 53,000 households in Scotland would not have installed energy efficiency measures over this three year period without the advice and guidance provided through the EST. While additionality is higher in relation to information and advice relating to microgeneration, the survey findings also underlined that EST support was more likely to be a necessary rather than a sufficient condition for implementation as grant support was also required in most cases in order for implementation of these technologies to occur.

6.5.4 The cost effectiveness ratios have been derived from the figures obtained from EST. These suggest an overall cost effectiveness of EST activity in the household sector of around £9/tC for the lifetime carbon savings generated. (Table 6.6)

Table 6.6: Cost effectiveness of EST household programmes

6.5.5 The EST Evaluation Unit has undertaken work on the impact of the SCHRI programme in terms of generating carbon savings. These figures suggest that the overall cost effectiveness of the programme over the past three years of funding has been £208/tC (Table 6.7). These figures demonstrate that given the current cost of microgeneration technologies, investment in this sector is currently less cost effective in generating carbon savings compared to investment in the market for energy efficiency.

Table 6.7: Performance of SCHRI Household and Lowland Community Stream, 2004 - 2007

6.5.6 A summary of the lifetime carbon savings generated from the annual expenditure on the key energy efficiency and microgeneration programmes outlined above is presented in table 6.8. The activities of the CT in Scotland are likely to generate the most significant share of savings (45%), followed by the consumer marketing activities of the EST (25%).

Table 6.8: Summary of estimated lifetime carbon savings from programme annual expenditure, tC

6.5.7 A summary of the cost effectiveness ratios presented in the value for money assessment is presented in a matrix format in table 6.9. The data presented focuses on performance during the year 2006/2007, where available. The table categorises the programmes against the type of intervention undertaken. Firstly, the provision of market infrastructure through online and telephone advice and auditing services, and then resource acquisition through loans and grants. The cost effectiveness of the main fuel poverty programmes in Scotland, Warm Deal and the Central Heating Programme (main programme) are also included in the table.

6.5.8 The table provides a useful indication of how cost-effectiveness in terms of carbon displacement can vary across different programmes. There are a number of points which need to be accounted for when reviewing these results.

• the analysis relates to the displacement of carbon over the lifetime of the energy efficiency and microgeneration measures
• cost effectiveness focuses specifically on the amount of Scottish Government funding allocated, rather than the total resource cost

Table 6.9: Summary of cost effectiveness by programme annual funding, £/Lifetime tC,

6.5.9 The provision of infrastructure services through public sources will facilitate implementation, although this will require subsequent behavioural change and/or resource acquisition to occur at a later date in order that the carbon savings can be realised. If the provision of infrastructure addresses the source of market failures and barriers (such as informational barriers) then this approach is cost effective as it will stimulate private agents to implement the measures using their own resources, rather than relying on public subsidy. The table above demonstrates that the provision of infrastructure is more cost effective than the provision of direct subsidies in the form of grants.

6.5.10 The most cost effective programmes are the two key programmes which operate at a UK level, the CT auditing services and the EST consumer marketing, at £10 and £6 respectively per lifetime tC. These two programmes also constitute the most significant sources of carbon reduction at 45% and 25% respectively, of the total lifetime carbon displacement by all the programmes outlined above. Both programmes benefit from extensive economies of scale by being part of UK wide initiatives, whereby the fixed costs relating to these services are spread more extensively across the UK.

6.5.11 The auditing services provided by the CT perform favourably in terms of cost effectiveness compared to the smaller programmes targeted at smaller companies through the BAN and BEP. These programmes do not benefit from the economies of scale from being a UK wide programme, and focus on smaller companies where the potential carbon savings per company will be lower. As noted earlier, the CT results in the lifetime carbon displacement of around 520 tC, compared to around 40tC for the BEP and 17tC for the BAN.

6.5.12 In addition, the review of additionality of the energy audits presented in figure 6 suggests that BAN performs less well in terms of net carbon savings generated compared to both the CT and BEP. The fact that BAN combines a lower level of project additionality as well as lower carbon savings per firm assisted means that the funding allocated to the BAN is the least cost effective out of the three programmes providing auditing services for businesses.

6.5.13 The least cost effective programmes, in terms of carbon displacement, relate to the provision of direct subsidies for resource acquisition for non-insulation measures. This underlines that support provided through a financial grant is the most expensive type of support that can be provided. Stimulating the market through a reduction in informational barriers (through the provision of information and advice) or through the provision of interest free loans, are more cost effective as the costs of implementation of measures are borne by the end users.

6.5.14 The provision of direct subsidies for resource acquisition for non-insulation measures includes the Central Heating Programme, which provides central heating systems for the over 60s, and SCHRI which provides grants towards the purchase of a number of microgeneration technologies. It should be noted that carbon displacement is not a primary objective for either of these programmes. The Central Heating Programme ( CHP) is designed as a programme to tackle fuel poverty, while SCHRI focuses on encouraging the wider adoption of small scale renewables in Scotland.

6.5.15 The CHP provides the full cost of the installation of the system, and so does not generate any additional leverage from private sources. While leverage is higher for the SCHRI. (households are required to pay for 70% of the cost of the equipment), the overall costs of these systems are considerably more expensive when compared to a traditional gas central heating system. The sector for small scale renewables is still under-developed and the costs of production for a number of these technologies, (particularly solar PV) are likely to continue to fall over the next decade.

6.5.16 In terms of direct resource acquisition of energy efficiency measures through the provision of grants, programmes which focus on improving the insulation envelope of buildings, such as the Warm Deal programme, are more cost effective when compared to programmes focused on heating and microgeneration systems. In addition, if the implementation of measures can be achieved without the provision of direct subsidy, but rather through the provision of infrastructure services, then this approach is generally more cost effective.

« Previous | Contents | Next »