The Scottish Government

« Previous | Contents | Next »

Listen

8 CONCLUSIONS AND KEY LESSONS

Introduction

8.1 This section presents the main conclusions of the evaluation together with a discussion of their implications and identification of key lessons for future activity.

Conclusions

8.2 As stated in Chapter 2, the overall aims of the Cultural Pathfinder Programme were to

• encourage participation in cultural activities amongst under-represented groups;
• develop effective means of community consultation to inform the development of plans for cultural provision; and
• explore planning methods for cultural provision and develop partnerships and links to Community Planning processes.

8.3 The Programme has supported 13 very different projects across Scotland, all of which have contributed, to varying degrees, to these aims. However, our overall conclusion is that the Pathfinders have performed well against the first two aims, with achievement against the third more mixed, as discussed below.

Participation

8.4 In relation to the first aim, the Pathfinders have made a strong and effective contribution. As reported in Chapter 4, we estimate that the projects have reached up to 1% of Scotland's population, and the evidence is that they have engaged across a range of groups currently known to be under-represented in terms of their cultural participation.

8.5 In so doing, the projects have tested a very wide-range of approaches and techniques for engaging people and encouraging interest and participation in cultural activities. They have also generated useful learning regarding how to engage different sections of the community and these have informed the lessons detailed below.

8.6 However, performance has been variable across the projects, and it is clear that some have been more successful than others in terms of their reach, and the breadth of the engagement achieved, although this also reflects differing levels of ambition amongst the projects. It is also clear that many of the people involved in the projects were engaged through more traditional means such as surveys rather than through participation in cultural activities. As a result, estimated figures for participation do not actually describe cultural participation, but rather participation in the Pathfinder projects.

8.7 This then raises the question of the level of participation that would allow the Programme to be considered a success, to which there is no simple answer. As stated in Chapter 4, the volume of participation is, in any case, an impoverished measure that says little about issues of the quality of the participative experience or its impacts.

8.8 However, on both of these issues, the Pathfinder projects have performed well. Reported levels of participant satisfaction were strong throughout the Programme, and there is clear evidence from a range of projects of the impacts that cultural participation can have on individuals and communities.

8.9 The difficulty here is that none of this is particularly new. Numerous community arts initiatives and projects have been subject to evaluation previously, with findings that consistently demonstrate the social impacts of cultural participation across a range of measures from health improvement and self-esteem to capacity building and community cohesion. This is not necessarily a criticism of the Programme, or of the supported projects, but more a reflection of the fact that these kinds of effects are already well- established. Indeed, while participation was a core aim of the Programme, it is its focus on community consultation, planning and strategic engagement in which there was always greater scope for innovation.

8.10 However, the variable levels of priority attached to culture across local authorities and CPPs suggest that messages about the value of culture are not being communicated as effectively as they might. Given the wealth of information collected through the Programme, it is important that this is now collated and communicated more widely in a concise and accessible way.

Community consultation

8.11 One of the key aspirations for the Pathfinder Programme was that it would explore and pilot different means of consulting with communities, thereby gathering valuable information and feedback that could input to wider planning work around culture.

8.12 There are two main elements to this aim:

• the development of effective consultation processes; and
• effective capture and use of the consultation outputs in planning activities.

8.13 In relation to the first of these two elements, the Pathfinders have performed well. The projects have developed and tested a wide-range of consultation processes and methods, some of which have been more successful than others. Certainly, there is evidence from a number of projects of the considerable potential of well-designed and facilitated creative processes to allow people from often marginalised groups to express themselves (see Chapter 4).

8.14 Importantly, many of the creative processes used in the projects demonstrated their capacity to build confidence within individuals in a way that enables and empowers them to make their voices heard and to contribute to the community. As discussed in Chapter 4, some of the projects provided very striking evidence of this, and have generated useful learning around how to deliver meaningful engagement and consultation. More traditional consultation processes do not appear to have such impacts and many of the more marginalised groups targeted by Pathfinder activities are in any case often excluded from such processes. The value of cultural participation, as evidenced by many of the projects, is that it can reach beyond many traditional community engagement processes and can bring the most excluded in from the margins.

8.15 As such, the Pathfinder Programme has provided ample demonstration of the potential of cultural participation to facilitate community consultation and engagement.

8.16 Interestingly, amongst those projects that identified themselves as taking a 'cultural planning' approach, the evidence suggests that consultation has been effective (in most cases). However, it is not always easy to distinguish the specifics of this approach from one that makes effective use of creative means of engagement and consultation but does not describe it as 'cultural planning'.

8.17 Having noted that, we conclude that the Pathfinder Programme has delivered effective community consultation, and has done so in ways that have often been highly creative and imaginative. Again, this has provided useful learning and has input to the key lessons described below.

8.18 However, delivering effective consultation is useful only if the outputs of the consultation process are captured and used to inform wider planning and action. In this respect, the performance of the Pathfinders has been more mixed.

8.19 This relates both to the issues with evaluation processes identified in Chapter 6 and the success of the wider strategic engagement around the projects, as discussed below.

Planning for culture and Community Planning

8.20 The evaluation demonstrates the very strong performance of some of the projects in developing creative means of consulting with communities and feeding the outputs through a range of key partners and into wider planning processes.

8.21 The projects that have been most successful in achieving wider strategic engagement and in developing stronger links to Community Planning processes and structures share a number of important characteristics:

• they were strategic in their intent from the outset, and sought early engagement with key partners and maintained this engagement throughout the projects;
• commitment and leadership were evident at different levels, from delivery and project management to senior level support and influence;
• they were able to build on existing partnerships in which there was either a willingness to engage with culture, or a sufficiently experienced and influential team making the case to the wider partners;
• culture was defined in broad terms, and not just as arts development;
• there was a clear demonstration and articulation to partners of the wider role of culture and the broader outcomes it could help to deliver; and
• there was a recognition of the need to work with or develop appropriate structures to facilitate partnership working and engage with wider Community Planning processes.

8.22 These projects also demonstrated the strongest strategic legacies in terms of the development of new approaches to cultural provision and mechanisms for engaging meaningfully across service departments within the local authorities and with CPPs.

8.23 However, as discussed in Chapter 4, some of the projects struggled to engage with the CPPs and, as a result, the strategic impacts of these projects were less apparent. However, the evidence also points to a range of contextual factors for why this may have been the case:

• the status and profile of culture across the CPPs is variable, and some of the Pathfinders faced a very difficult task to engage the CPPs in discussions around culture;
• there are structural barriers in some cases, particularly where there are no clear structures for engaging wider partners in culture and communicating with CPPs;
• not all local authorities attach high priority to culture and where this was the case, projects experienced greater difficulties in securing ongoing commitment to take forward the outputs of the Pathfinder work at a strategic level; and
• for some it is simply too early to have a clear sense of how the Pathfinder will influence future planning, although the evidence suggests that plans to build on the Pathfinder work are being considered in most projects.

8.24 However, despite the mixed results in relation to CPP engagement, the Pathfinder Programme has provided very valuable learning about how culture might be integrated into Community Planning processes.

Overall assessment

8.25 The Pathfinder Programme has in large part achieved what it set out to do in so far as it has explored different methods and approaches and has generated valuable learning that can inform future policy and practice. It has also created important strategic legacies that offer strong potential to enhance the delivery of cultural provision in many areas.

8.26 The Programme demonstrated strong levels of additionality, and has helped to raise the profile and understanding of culture across a wide range of partners, as well as providing opportunities for cultural participation for citizens across Scotland. It has also demonstrated the value of cultural participation as a means of facilitating meaningful community consultation, suggesting strong potential for its wider application in Community Planning contexts.

8.27 In an exploratory initiative such as this, it would be expected that some projects would achieve more than others, and this should not be seen as a criticism of the Programme. Indeed, all of the projects delivered work of value, even if some will have greater long-term strategic impact than others.

8.28 Assessing the value for money of the Programme is more difficult. As noted in Chapter 5, projects generally struggled to identify meaningful indicators, defaulting to detailing the costs of individual elements of the projects. Without meaningful benchmarks, it is not possible to determine whether or not these costs represent value for money, and measures such as cost per participant (even if they were available) would be insufficient. On the only available measure of value for money (leverage - financial and in-kind) the Programme has performed well (63% of the total Programme costs were levered from local authorities and other partners).

8.29 However, to capitalise on the investment in the Programme it is also important to consider the wider implications of the issues raised by the Programme in the context of cultural policy in Scotland.

Wider issues and implications

8.30 The policy environment in which the Pathfinder Programme has operated changed during the life of the Programme, and it is important briefly to reflect on this in order to highlight implications for current policy.

8.31 The Scottish Government is clear that its role is not to dictate to local authorities how their budgets should be spent, but rather it has agreed outcomes to be delivered, as articulated in the local SOAs ¹⁶. From 2009/2010, these will be agreed with CPPs across Scotland, underlining the critical importance of culture being able to articulate clearly its role in contributing to national performance targets.

8.32 Throughout the evaluation, there were mixed responses to this change in policy direction. In particular, feedback in a couple of the projects was that the policy change had led to a decline in interest in the Pathfinder process for some partners, as it was felt that resources would be difficult to obtain in the future. There were also signs of a misunderstanding by some who believed that the resource local authorities devoted to culture, from Scottish Government funds, had been ring-fenced. In contrast, others saw the change in policy as an opportunity for culture to demonstrate its contribution across a range of outcome areas.

8.33 More specifically, a few of the projects reported that the shift away from the notion of cultural entitlements during the life of the Programme had created difficulties and required a change in approach. However, views on the entitlements agenda were always mixed, and while some felt it was a positive step, others perceived problems with definition and implementation.

8.34 As stated in Chapter 2, our view is that the Pathfinder Programme remains relevant to the current policy environment, even if it was developed in another context. The core drivers of understanding community needs and aspirations for culture and engaging wider partners in the value and role of culture are arguably even more important in the context of current policy.

8.35 However, with no new statutory responsibility on local authorities to deliver culture, its position remains vulnerable (although it is not alone in this). The evaluation has identified the critical role played by individuals in facilitating wider engagement in culture and promoting its benefits to CPPs and other delivery partners. The challenge is that these individuals are not evenly distributed across the system, and there is a real need to consider how to develop stronger and more consistent cultural leadership in Scotland to champion and drive forward cultural policy and delivery at national and local levels.

8.36 The Programme has also highlighted an issue with language and definition. As discussed, many of the projects adopted a broad definition of culture as way of life, and the feedback from some of the consultation work was that many communities also understood culture in similarly broad terms. It was also reported that pulling the definition of culture away from a narrow focus on the arts helped to engage wider partners eg those who shared the aspiration to promote such aspects as positive lifestyles and vibrant, engaged communities, to which culture (as generally defined) can contribute positively.

8.37 However, this poses a challenge for policy making. If culture is defined as way of life, then its reach in policy terms is considerable, and there is a need for some focus. It was also reported at the workshop with project managers that CPPs need clearer messages about culture, about its impacts and about how it can contribute to wider priorities.

8.38 While debates about the definition of culture are important and interesting, the language can be confusing (and indeed some of the Pathfinder consultation work found negative views of the term 'culture' amongst some communities). In particular, 'cultural planning' is not well-understood as a term or concept, and even though a number of the projects identified themselves as adopting this approach, they did not all adhere in full to the cultural planning methodology (a few did not complete an audit of local cultural assets).

8.39 This is not to discredit in any way the value of the cultural planning approach. Indeed, the 'cultural planning' projects in the Pathfinder Programme generally delivered strong outcomes. Rather, some of the feedback, particularly from wider partners, was that the language of cultural planning is confusing and the distinctions between cultural planning, planning for culture and good community arts practice are not widely understood, or necessarily important. The key point about activity in each of these three areas, however defined, is to achieve meaningful engagement by communities, service users and partners, and to focus on delivering project objectives using culture.

8.40 Therefore, there is a need for greater clarity about what is meant by culture and cultural provision, and, crucially, about the scope and boundaries of cultural policy. This is important for the sector as a whole, but particularly so when seeking to engage partners across other service areas.

8.41 Finally, the findings demonstrate that the means by which productive strategic engagement around culture can be developed at a local level are highly dependent on local circumstances. General principles can be identified, but their application will vary according to the structures, processes and individuals in place in different areas. One size will not fit all.

8.42 There are also clear barriers to the wider acceptance of culture as important, both in its own right and in relation to its wider contribution. These are also highly dependent on local circumstances, and even on the interests, skills, commitment and vision of individuals. This again creates vulnerability and inconsistency - key challenges for national cultural policy to address.

Key lessons

8.43 In line with the requirements of the brief, we have not sought to make firm recommendations regarding policy or practice, but have identified key lessons and aspects of good practice across the Programme. This final section collates these to provide a concise set of lessons that could usefully inform future activity. They are split into two broad areas:

• lessons regarding strategic planning and engagement; and
• lessons regarding community engagement and participation.

Strategic planning and engagement

8.44 In seeking to develop wider strategic partnerships to support the development of effective planning for culture, a number of broad principles seem important:

• define clear strategic objectives at the outset, communicate them effectively to partners and maintain focus on them throughout;
• it is crucial that culture can demonstrate its value and impact across different service areas and policy priorities. Without this, it will struggle to gain recognition within Community Planning and SOA delivery processes. It is therefore important to collect and articulate clearly the evidence for these impacts and to communicate this effectively;
• seek early engagement with key partners and communicate clearly the value and role of culture across the wider planning context and in specific service areas;
• leadership is key, and this applies at different levels of the process. The support of senior managers and elected members is crucial, but so is experienced and committed project management. Culture needs strong champions at all levels;
• structures are critically important, not only for facilitating initial engagement, but also for sustaining ongoing partnership. They are also key to supporting routes into wider partnership structures, in particular CPPs;
• culture's potential for facilitating community input into wider planning debates is clearly demonstrated by the Pathfinders. Therefore, a consultative approach that combines creative processes with strategic engagement and the development of appropriate structures is a useful outline model;
• while broad definitions of culture can help with initial strategic engagement, focus is needed for planning, and it is important to be clear about what is meant by culture and about the role of cultural policy and provision with that definition; and
• it is very important that processes remain flexible and responsive to local circumstances. While there are clear examples of good practice in the Programme, these may not be directly transferable to other areas due to the diversity of local authority and Community Planning structures and processes, and the different character of the cultural sectors and wider communities.

Community engagement and participation

8.45 As stated, participation in creative consultation processes can empower individuals and communities to express their needs and aspirations for local services, creating a powerful vehicle for community input to wider planning. The Pathfinder experience identified a number of useful lessons for how specific groups can be reached and engaged, as follows:

• informality and a non-judgemental approach can be useful, particularly in engaging the most marginalised communities, and particularly where there is likely to be a high degree of suspicion or negative perceptions of culture. Treating people fairly and with respect may seem an obvious lesson, but it is nonetheless important;
• the quality of the creative process is paramount, and the role of creative practitioners is crucial. Sensitivity and patience are key attributes, as is flexibility and the need to establish boundaries and frameworks within which communities can engage;
• many of the barriers to effective participation in culture are well-known: eg transport, accessibility, lack of appropriate provision, perceptions of culture and cultural venues - and addressing these directly can encourage participation, even if sustaining it is more challenging;
• taking culture out of cultural venues and into communities can help to widen access and encourage engagement by removing the barriers relating to the perception of cultural venues as unwelcoming or 'elitist';
• the choice of creative medium is also important, and should be tailored to the needs and interests of different communities. What engages older people may not work with teenagers or BME communities;
• embedding cultural activity (and even facilities) in communities helps build familiarity and trust and can overcome some of the known barriers to participation;
• working with existing groups, partners and representatives within communities facilitates community access and helps to build trust and encourage participation;
• cultural leadership within communities is also crucial, and the role of the creative animateur is an important one. The Pathfinder Programme identified a number of experienced and skilled practitioners, and the experience of this group could be a useful input to the development of future policy and practice; and
• ongoing communication and consultation is also important, and gives communities the sense that their contributions are being heard. This not only builds engagement in specific processes, but can also promote wider confidence in consultation and planning.

Evaluation

8.46 As discussed, the quality and depth of evaluation across the Programme has been variable, and a number of useful lessons can be drawn from the programme's experience in this respect:

• evaluation is crucial if culture is to demonstrate its relevance and importance. It is therefore essential that cultural projects and programmes adopt a suitably robust approach;
• in future programmes, it may be worth considering a set of consistent measures and indicators to be collected for all supported projects, thereby providing a basis for assessing the relative performance of projects as well as the programme as a whole;
• evaluation plans are useful in providing a guiding framework for evaluation processes, but there is room for improvement in the specification of appropriate indicators. In particular, indicators and measures should be realistic and should follow a logical chain from inputs (financial and otherwise), through the activities delivered (the immediate outputs), and on to medium and longer-term impacts and outcomes;
• too many indicators wrongly specified can cause confusion and create an unnecessary reporting burden. It is better to focus on a smaller number of clear and appropriate indicators than attempt to measure/describe every aspect of the project;
• basic data should be collected as a matter of course, including numbers and characteristics of participants, numbers and types of activities and financial and other inputs;
• data collection methods for assessing the impacts of cultural activities should be robust, transparent and defensible, and should not rely on anecdotal feedback alone; and
• creative evaluation processes can be useful in engaging participants in sharing learning and in supporting advocacy efforts, but they should be seen as additional to more formal evaluation, that can evidence and articulate effectively the impacts of the projects against appropriate objectives and measures.

Final comments

8.47 The Cultural Pathfinder Programme in Scotland was a useful and productive exercise that explored a range of approaches to community engagement and planning for culture. It has largely achieved its aims and, in addition to the impacts and legacies of the projects, it has identified important lessons for the future about how to engage communities and planning partners in culture.

8.48 In wider policy terms, the Pathfinder experience should prove valuable. The Scottish Government is clear that its role is not to dictate to local authorities about how budgets should be spent, but rather it has agreed outcomes to be delivered, as articulated through the SOAs.

8.49 As the SOAs are agreed with CPPs from 2009/10, it is important that the cultural sector in its broadest sense is able to engage productively with this process, demonstrating clearly its potential contribution to identified outcomes. The Pathfinder Programme has provided much in the way of useful learning about how this engagement might proceed.

« Previous | Contents | Next »