Scottish Government

2 INTRODUCTION

2.1 Scope of the ccf review

2.1.1 The Scottish Government's Climate Challenge Fund ( CCF) was set up to help communities combat climate change by reducing their carbon emissions. The CCF made 331 awards to 261 communities located throughout Scotland in seven funding rounds between 2008 and 2011, with further funding announced in March 2011 for 130 projects. The Fund is administered by Keep Scotland Beautiful ( KSB) on behalf of the Scottish Government.

2.1.2 In March 2010, the Scottish Government commissioned Brook Lyndhurst and Ecometrica to carry out a review of the Climate Challenge Fund to explore the kinds of impacts projects have had, and to identify key factors contributing to projects' successes. The Scottish Government's aims and objectives for the review are summarised in the box below.

2.1.3 The CCF review was not a traditional fund evaluation which counts and tallies all the inputs, outputs and outcomes from a programme. This kind of approach would have been highly challenging, bearing in mind the set up of the fund and the individual projects. The review took a more pragmatic approach, focusing in depth on 21 selected projects, working at a range of scales and with different focuses, in order to illustrate the many ways community initiatives can encourage low carbon behaviours and to provide lessons about the barriers and opportunities for engaging people in low carbon living.

2.1.4 By 'success factors' this report refers to particular aspects that facilitated changes in participants' behaviour, whether that was an element of the project set up and overall approach, or the use of a particular hook, message or incentive.

2.1.5 The review took a largely qualitative approach, using interviews to get an in-depth understanding of the range of motivations, barriers and success factors. In addition, Ecometrica calculated reductions in CO ₂ emissions for a sub-set of eight of the projects to illustrate the carbon reduction potential of different approaches. Quantitative data has also been drawn from the projects' own reports to KSB, but wider statistical data has not been collected during the review.

2.2 METHODOLOGY

Selection of the 21 sample projects

2.2.1 The 21 projects included in the review were selected from an original long-list suggested by the Scottish Government and KSB officers, according to a range of criteria. The selection criteria ensured that a range of characteristics was covered - including the amount of funding awarded, urban and rural projects from different areas in Scotland, projects using different types of interventions and a spread across different behaviours in the topic areas of:

• Energy efficiency
• Energy generation
• Food
• Transport
• Waste

The selection included 'multi-strand' projects that are trying to influence behaviours and lifestyles on a broad front, as well as those with a more singular focus - such as transport or food growing.

2.2.2 Projects in the review also varied significantly in size: the amount of CCF funding ranged from around £7,000 to upwards of £650,000 (amounts are shown for each project in Appendix A). The proportion of each project's work funded by the CCF varied from just under one-third to the whole amount.

2.2.3 Part-way through the review, due to difficulties in data collection on some of the projects, a further project was added to complement what was emerging from the review, bringing the total number of projects to 21.

2.2.4 Table 2.1 gives an overview of the projects reviewed to provide context for the research findings. The categorisation in the table has been based on the primary activity of each of the projects. Similarly, the 'Areas Covered' and 'Brief Project Outline' columns only include key strands of project activity, not the entirety of their work ¹. For a fuller picture of the projects covered in the review, please refer to the Project Descriptions in Appendix A.

2.2.5 It is worth noting here that even though the 21 projects all originate with community activists, the nature and level of wider community involvement differs greatly between them. Many provide what is effectively a service to individuals within their communities; some deliver short term behaviour change initiatives to specific target audiences; others bring people together to build long term relationships and continuing joint action ( e.g. growing projects); and a few are seeking to get local people fully involved in transformative programmes of collective action on climate change.

Research method

2.2.6 Most of the evidence for the review was collected through qualitative interviewing, supported by quantitative analysis of carbon emissions by Ecometrica, and some documentary evidence from projects' own reports to KSB. (The detailed methodology for calculating carbon emissions is outlined in Appendix B, and more detail on the method for the qualitative primary research is given in Appendix C.)

2.2.7 The first phase of research involved a 2-3 hour visit to each of the shortlisted projects to meet the project managers and interview them in depth about what the project expects to achieve, how it has evolved, how it is set up and delivered, and how its impacts are being evaluated. Project managers were also given diaries to record their ongoing thoughts on successes during the course of the review.

2.2.8 The next stage of research involved a mix of face-to-face and telephone interviews with project participants, to explore their experiences of the projects, and in particular any changes in behaviour as well as what had led them to change. Project managers were asked to provide contact details of 30 participants each (though this target was revised down on smaller projects) who would be willing to be interviewed for the purposes of the review. Project managers were asked to aim for a mix of interviewees across the project's reach, including a small number of non-participants or drop-outs if possible, to investigate barriers to participation. Interviews were also carried out with project volunteers and staff members. Where other interview approaches were more suited to the project activities and focus, some participants were interviewed at project events or (for example, on school-based projects) in focus groups. We had varying degrees of success at gaining access to project participants, and the final total of participants, volunteers and staff members interviewed stands at 477.

2.2.9 The final stage of primary research involved follow-up interviews with project managers, to reflect on project delivery and successes, and to play back and test with them some of the research team's emerging thoughts from interim analysis.

2.2.10 The projects were due to complete their final evaluation reports for KSB just as this report was being prepared. Where these evaluation reports are available, relevant findings from them have been incorporated into this report.

Carbon assessment

2.2.11 In order to explore the carbon impacts of the CCF projects, a detailed quantitative assessment of a sub-group of eight of the projects was undertaken by Ecometrica, with the aim of producing estimates of the emission reductions achieved by each project over the course of the lifetime of the projects' outputs.

Table 2.1: Projects reviewed

Energy projects

CCF #	Name of organisation	Name of project	Areas covered					Brief project outline
			Food	Energy efficiency	Energy generation	Transport	Waste
CCF 055	Deaf Connections	DEAFinitely Greener						Translate climate change information into British Sign Language and distribute it via DVDs and web. Train deaf mentors/advisors for deaf people. Use community centre to demonstrate energy efficiency.
CCF 182	East Neuk Communities Group	East Neuk and Landward Energy Network						Energy champions engage local residents in efficiency improvements through home energy checks and 'hand-holding' them through implementation
CCF 198	Sustaining Dunbar	Dunbar 2025						Follow the transition model, 'visioning' a sustainable future for Dunbar. Energy audits and advice offered.
CCF 209	Barrhill Community Interest Company	Climate Champions Network						Use intermediaries to promote subsidised insulation to local residents. Engage people in energy efficiency.
CCF 247	Transition Edinburgh South	Switched on to switching off in South Edinburgh						Intermediaries engage local residents in energy efficiency through motivational interviewing.
CCF 304	Fintry Development Trust	Fintry Renewable Energy Supply Company						Visits to households to advise and offer support on energy efficiency or generation.
CCF 415	Raploch Community Partnership	Low Carbon Raploch Project						Survey households in conjunction with the Home Insulation Scheme, with follow up visits, and a community access point.
CCF 466	Millburn Academy	Energy+Action = Change						Carbon education, monitoring and calculations done in science and maths lessons. Includes use of energy monitors at home.

Food projects

CCF #	Name of organisation	Name of project	Areas covered					Brief project outline/description
			Food	Energy efficiency	Energy generation	Transport	Waste
CCF 016	Transition Town Forres Ltd	Transition Town Forres						Awareness raising through events, information and film showings. Activities centred on community garden and farmers' markets. Carbon footprinting surveys conducted.
CCF 065	Urban Roots Initiative	Toryglen Transitions						Development of community market gardens, woodland conservation, community carbon footprint survey and workshops, schools work.
CCF 126	Falkland Centre for Stewardship	Fife Diet						A membership network providing advice and contacts for sourcing food locally.
CCF 232	Perth and District YMCA	Community Carbon Champions (The Three Cs)						Trainees develop vegetable plots in people's gardens. Also receive training and education around climate change and energy efficiency.
CCF 312	Heal the Earth	Assloss Walled Garden						Develop allotment site for local community. Aiming to improve well-being and strengthen the community.
CCF 367	Care and Repair	Edinburgh Garden Share Scheme						Pair up people who need help looking after their gardens with people who want to grow food.

Transport projects

Multi-strand projects ²

2.3 DATA QUALITY AND LIMITATIONS

2.3.1 The qualitative interviews provide a wealth of evidence to analyse and draw upon. In particular, the qualitative nature of the data lends itself well to the exploration of success factors with respect to different behaviour change techniques and target behaviours, including some interesting case studies.

2.3.2 While qualitative data is particularly useful for revealing the how and the why behind observed impacts, it is important to recognise its limitations. In terms of impacts, the data exemplifies the kinds of impacts that projects have had and their apparent prevalence, but is not suited to quantifying or generalising the overall impact of the CCF or the 21 projects considered.

2.3.3 It needs to be noted that the participant interviews were not evenly spread across the projects. The number of interviews per project was a function of project size, participants' willingness to be interviewed, and the efforts of the project manager to recruit participants for interviews. To some extent, the research team was able to compensate for the latter two issues by attempting to gain access to participants through different means, for example by attending project events. When it became clear that interview targets were unlikely to be met in certain cases, the project team agreed with the Scottish Government to expend no further effort in pursuing interviews with participants on three of the projects, and an additional project was added to the review to enhance the range of projects explored.

2.3.4 A further point worth noting is that the participants who have been willing to take part in interviews are likely to be the most interested and active participants in each project. Only in very rare cases were we able to interview non-participants or drop-outs. This means that we have been more likely to hear the success stories and less likely to hear about failures and problems - though we have occasionally come across those, too. The findings of the review should therefore be read in the light of these limitations.

2.3.5 During the follow-up interviews, project managers were asked to return the 'success diaries' given to them in the initial interviews, but although we know that at least some had been making use of them, none arrived in time for the contents to be considered in the context of this report. This reflects the projects' own monitoring and evaluation experiences, which suggest that diaries are largely an ineffective approach to evaluation due to the amount of effort required to complete them or to manage participants to do so (see paragraph 6.3.6).

2.4 REPORT STRUCTURE

2.4.1 Each section of the report draws together the main themes that were evident in the qualitative data, supported by examples from specific projects and quotes from participants or project managers. In keeping with conventions for reporting qualitative data, indications of the weight of response are given in terms such as "many" or "a few", rather than as numbers or percentages. The exception is the section on carbon emissions where quantitative estimates are provided.

2.4.2 Examples are provided throughout the report from individual projects to illustrate the points made. In the spirit of the review - the purpose of which was to identify success factors and to learn from projects' experiences - we have opted to name projects where these examples demonstrate success, but to anonymise them as far as possible where the examples could reflect negatively on them (bearing in mind that this is not always possible, as some projects' characteristics make them highly recognisable).

2.4.3Chapters 3-7 present the research findings from the review as follows:

• Impacts on attitudes, behaviours, carbon emissions and other aspects of sustainability
• Motivations, barriers and success factors - for changing specific behaviours
• Cross-cutting success factors
• Monitoring and evaluation
• Thoughts on the future of CCF projects

2.4.4 In Chapter 8 we give a slightly more interpretative view of the role of CCF communities in responding to climate change. Chapter 9 provides a concluding discussion on what the review has revealed about the CCF, the general lessons for community-led climate change projects, and the implications of the review for the CCF or other policy programmes.

2.4.5 Each chapter of the report (with the exception of the conclusions) begins with a summary of key points.