SCOTTISH ADULT LEARNING PARTNERSHIP (SALP) PATHFINDER PROJECT - EVALUATION PLAN
Evaluation Plan
This Pathfinder's evaluation plan appears below. It details the main elements of the project including the inputs, outputs, their projected outcomes of the project and the way that it will be measured. For further explanation of the Evaluation Plan, please see the Evaluation Toolkit.
NAME OF PATHFINDER PROJECT - Talking 'bout my generation Scottish Adult Learning Partnership |
Brief description A locally focused, pan-authority partnership and exchange project to establish bonded groups of older people in order to engage them through shared stores, to strengthen their voice in articulating their aspirations to take part in culture and to make an impact on their local Cultural and Community Planning processes. |
Overall aim To bring together and empower older people and improve the extent and quality of their community and cultural engagement, platformed through a national exchange network that discovers a new form of theatre over a 3 month engagement with their peer group, artists, and each targeted local councils' community planning partnerships (CPPs). |
Specific aims - strengthen the voice of older people as valued citizens
- impact on the local Community Planning agenda in each area through cultural planning
- generate social capital
- explore the theatre arts (voice and word, image, dance, music and presentation of ideas) by exploring stories from the 'Big Lives' of the participants using drama techniques
- explore with participants what cultural engagement means to them and the ways they would/might like to expand that
- develop a community that exchanges through dialogue between geographically disparate groups and discovers its members' aspirations for cultural engagement
- examine the political voice in art by looking at the national collections and contemporary repertoire as stimuli for creation of new theatre
- measure the impact of our process on the well-being and connectivity of the participants
- create digital media to transform participants' own imagery and photographs into a projection and light-scape environment for performance
- connect older learners in the arts from other projects and initiatives from across the country by inviting them into our National Theatre of Scotland (NToS) Exchange and Café events
- build professional capacity so that we widen the group of artists and educators who have confidence to work with this age group in a non-reminiscence process
- capture and share the learning using film and the evaluation toolkit provided
- capture reactions to participants' 'voices' and aspirations from participating local authorities (8) and Community Planning partners
|
Resources (inputs) Funding: £38,600 with £30,000 secured as a Cultural Pathfinder investment In-kind support: £8,600 from local authority partners and SLP |
Activities/services (outputs) Stages in making and evaluating the project Stage 1. Visualise the idea and potential shape for a project Stage 2. Invite in the creative partners to the shaping of the whole Stage 3. Design model to show others idea and share learning from pilot Stage 4. Work out what is required to achieve idea from central and local perspectives Stage 5. Form relationships with the four participating authorities Stage 6. Agree delivery framework and responsibilities of local delivery and central creative teams Stage 7. Work with each delivery team to customize and deliver evaluation model Stage 8. Re-assess idea in light of sign-up, demographic of each group and local Community Planning priorities Stage 9. Re-design if necessary Stage 10. Assess risks in proceeding Stage 11. Map levels of cultural engagement with the participants Stage 12. Deliver workshops and create work Stage 13. Share progress and exchange inputs and learning across whole Stage 14. Build each installation and create central Exchange Stage 15. Undertake creative self-assessment involving all delivery partners and participants Stage 16. Capture reaction of all 8 local authorities - in light of participants' expressed aspirations - as regards their future strategic planning & intentions for cultural provision/access for older people Stage 17. Reflection and external assessment See Annex 1 for 'Questions to ask ourselves' at different stages Please see Project timeline doc and evaluation DVD from Pilot Phase See Annex 1 for details of Evaluation Team |
| |
Indicators set for key performance aspects (list 2-3 for each key aspect - more are provided below, only as examples ): | Achievement against each indicator set: |
Activity (output) - - One Café Culture to launch the project in each area
- During September each participant will create a project 'passport' to include the baseline date that we need to collect.
- We will hold a minimum of 12 cultural democracy workshops with 4 core tutors working over 3 months
- We will hold a minimum of 3 workshops involving our digital artist who will introduce the use of digital media to record and generate visual imagery
- There will be up to 5 specialist tutors in each group in dance, music, design, and playwriting
- There will be one NToS Exchange event two thirds of the way into the 20 week process
- There will be a local performance platform for an invited audience of CPP members, friends and interested others from across the development of Cultural Policy and Arts Education in Scotland
- A series of workshops which will provide a variety of creative activities developed as a list of options with each participating group
- We will contact members of the relevant CPP and invite them to participate in the project as audience and with a view to them providing sustainability to the groups created during the Pathfinder process
| Activity (output) - |
Participation (output) - - A minimum of three Council officers from across services will attend each Café Culture event
- Will seek to achieve active participation by 20 people over 50 years of age in 20 week process in each of 4 local authorities
- There will be one additional input to the workshops from the 4 local Community Planning Groups
- There will be a further input into the workshops from either the National Galleries, Dance Base, the National Theatre of Scotland-Learn, Scottish Opera or Scottish Playwright's Studio
| Participation (output) - |
| |
Satisfaction (outcome) - - Participants in the workshops will be satisfied with the content and tutors involved
- Contributors to the workshops (eg CP groups and arts organisations) will be satisfied with the organisation, delivery and outcome of the workshops and other activities
- Participants in the project (ie older people) will be satisfied with the level of consultation and involvement
- Participants will feel that the Café Culture has added a quality dimension to their lives and that their breadth of involvement with culture has developed
- Local authority partners will have bought into the project and will actively support a continuance of cultural planning work with this age group
| Satisfaction (outcome) - |
Please see stages and activities sections for more detail about how we will agree upon the quantitative & qualitative data, output and outcome indicators that each of our Delivery partners in each authority will gather/use from August through until a projected December completion of this Pathfinder phase of the project. Impact (outputs and outcomes) - - A Café Culture will be created to bring together all local authority services with a connection to older people to grow the reach of the project and the network of potential participants
- Exchange opportunities between participants will have taken place to link each of the groups into the wider project and active peer group
- Older people will have participated in the project and National Theatre of Scotland Exchange activities
- The project will have secured local community planning partnership attention at performance events
- Production of footage for the project's docu-film that evidences the process and how CPP and audience inputs were added to SLP's collective manifesto
- There will be a record of participants who have signed up to ongoing cultural planning or cultural activity and of the activities they plan to engage with as a measurement of the impact of the project
- The well-being of participants will be shown to have improved because of their involvement in the project
- Partners involved will become members of a national working group with peers from the pilot phase, the National Companies and funders, who will work together to develop the next phase of the project
- Culture's impact will be evident in local strategic (CP) planning for older residents
- Longer term outcome - that groups of older people become connected to each other and to a network across the country that values their voice and experience
| Impact - |
Value for money (output and outcome) - - Liz Gardiner (Fablevision) will focus on the question of value for money and the added value of theatre as a cultural planning tool, by interviewing the local delivery and creative teams - and the project's local authority partners
| Value for money (output and outcome) - |
Our final evaluation will be in three parts: 1. the docu-film - an edited DVD record that will include a summation of the whole process, performance footage, participants' sound-bites and audience manifesto commitments with excerpts from the tutors' journals 2. the kist and its contents - artefacts, images, new writing, exchange items and ideas that have been built up and passed between the groups 3. the written document based on the toolkit, capturing all activity, measured evidence and impacts - compiled from the activities of the whole delivery, creative and evaluation team; to include the passports, postcards, digital portraits and excerpts from the tutors' journals with learning points and recommendations to be pulled together by Liz Gardiner. It will also include context statements by the CPPs regarding strategic planning for older members of the community. |
Data collected: |
Baseline data will include: · Participants' personal and demographic details · A mapping of the cultural and democratic involvement opportunities that already exist · Participants' experience and uptake of culture before starting the project (range and frequency) · A record of the physical and virtual means, the personal, council and community networks and partnerships used to reach potential participants · A plotting of how many people attended the initial event, the Café Culture and signed up for the project and how many stayed with the project as active participants · An analysis of the effectiveness of our local advertising and word of mouth marketing campaigns · Personal statements about why people have chosen to be involve · The extent to which Community Planning in each of the LAs already includes and promotes Cultural Planning and any developments in approach resulting from the project See Annex 1 for details of methods to collect baseline data | Interim report stage | Final report stage |
Outcomes |
Progress information (how is the project developing/did it develop, compared with the original plan? |
Main learning points |
Annex 1
Evaluation Team
Our evaluation team is made up of:
- Jaimini Jethwa - docu-film maker
- The local authority delivery teams - SLP contacts within local community learning and development teams
- The Creative Team - four SLP and NToS Director tutors with digi artist Kenny Bean
- Liz Gardiner (LG) - Convenor of the National Cultural Planning Steering Group and our External Evaluator
- Jacqueline Whymark (JW) and SAC funded Older Learners and the Arts project administrator who are responsible for driving, monitoring and supporting the evaluation process going on 'in the field'. The administrator will also plot the expenditure and income generated by local authority partners as the project evolves and draws in both cash and in kind investment from the partners and their CPPs.
Of course, the project evaluation will be undertaken too by all of the participants and agencies etc involved in delivery including the local authorities and CPPs: what is their journey? Having captured their feelings and individual expectations or starting point we will go on to capture their experiences weekly and at key Exchange points in the collective journey of the project (as detailed under our activities section of the plan).
The Creative Team will meet as a group with the External Evaluator, digi-artist Kenny Bean, film-maker, Robin Baillie (National Portrait Gallery), Simon Sharkey (NToS) and JW in August, before the project delivery stage, for an artistic briefing and in order to agree each team member's responsibilities within the overall evaluation process. The artistic briefing will be an opportunity for the whole team to explore and contribute to the artistic and political intentions for the project.
We are working to make portraits with the Elders in our communities, by making 2D photographic and live 3D performance portraits of our participants by uncovering and spotlighting their stories, needs and asks for our audience of decision-makers and local community planning partnerships. The dramaturgical line is to make living portraits using all of the theatre arts and digital media.
At this full-day meeting we will also create or distribute the tools (including the passport, journal, kist and digital hardware) that the Creative tutors will need to use throughout the project.
JW and/or LG will meet individually with each of the local delivery teams after their Café Culture to customise the project model and agree the details of baseline and quantitative data that needs to be collected and talk through the evaluation tools to be used by the local teams during the 20 week process. At these meetings we will also set the dates for a future 'collaborative meetings' to which we will invite all of the delivery teams and evaluation team and through which we can collate and share our learning as we go along. Proposed dates for these 'collaboratives' are last week October and first week December. At these meetings we will share the indicators that each authority is using and thereby share our learning between our four authority partners.
Invited specialists who visit into the process (including Robin Baillie and Simon Sharkey) and the project's audiences at NToS Exchange and local performance events will also be asked to record 'sound-bite' reactions to camera after each workshop or performance event.
We will ask CPP members and other involved council staff and officers to contribute at least one learning point or commitment to a collective manifesto for the Elders' continued voice in our national CPP processes and/or the development of cultural planning as part of the CPP process in their area.
Questions to ask ourselves, our partners and the Creative Team at Stages 2, 3 and 4
- What impact do we want to make?
- What activities are we going to use?
- How will we involve the participants as evaluators of their own journey?
- What outputs and which specific indicators will we use to measure their effectiveness?
- How shall we capture the results?
Additional questions to ask ourselves with the local delivery teams at Stages 5, 6 and 7
Stages 6 and 7 are all about customising the model to suit the individual authority partners. Each local authority delivery team will be interested in evaluating specific impact across a range of qualitative criteria depending on their particular operational context. It is envisaged, for example, that for Inverclyde, the mapping of existing cultural activity and linking of people into existing learning and well-being initiatives for this age group with the Council services already on offer will be of most interest; whilst in East Ayrshire, the CPP was represented at the Café Culture and they are keen to recruit new voices into their local CPPs to feed the regional partnership.
- What baseline data do we need to collect?
- What qualitative evaluation data are partners interested in collecting and at which points in the process?
- How will we share the outputs and learning across SLP's national partnership of community learning and development providers?
- How can the learning from the Pathfinder provide and share good practice illustrations for use in the evolving Scottish Executive Local Authority Guidance documents for effective cultural planning within a community planning framework?
Once we have agreed whole and local project timeline(s), outcome indicators, delivery expectations and evaluation responsibilities, we will agree our interim reporting stage(s) and collective evaluation meeting points for Creative team and reps from each delivery team.
Methods to collect baseline data:
Each participant will take part in the I am… Maori greeting exercise at the four Cafés Culture to give us a more personal record of who our participants are and where they come from. These individual greetings will be developed into the creation of a group Haka for use by the groups at national and local exchange and performance events as an expression of identity and confidence.
Also as part of the Café Culture we will map participants' current cultural - literally - on a visual map of the region with room to record connections beyond the local geography.
Each Creative tutor will introduce their group of participants to our process and their part in our overall evaluation. We will start by explaining how each participant can feed into this shared learning by regularly answering the single question 'And how are we feeling today?' during each workshop and at key points throughout the project. This simple measuring device will provide us with a cohesive plotline for qualitative feedback during the three-month process.
Exchange opportunities for participants from the pilot authorities (Argyll and Bute, North Lanarkshire, Angus and Fife) will be offered and will include an invitation to perform their work at SLP's showcase event on Saturday 18 August in the Imagination Lab on site at the Edinburgh International Book Festival. We will also support individuals from the pilot groups to bring their experience into the 20 week Pathfinder workshop process and will invite the pilot groups to perform some of their work as part of the NToS Exchange in November.
At a meeting of all of the local authority partners we will capture the level of their engagement with the project and comment on how ir is likely to influence future cultural (and wider0 provision for older people.
A film will be made of the NToS Exchange event (fast growing into a national Festival and platform for not just our eight authority groups, but also for other Pathfinders and local authority initiatives that focus on older learners in the arts) that brings the outputs and outcomes to a wider audience as part of our dissemination plans.
Each core tutor will keep a journal and record key events, successes and failures, subjective observations, quotes and ideas in it each week on behalf of themselves and their group. The journals will be used as part of the exchange between groups and will be submitted as evaluation documents.
A virtual and/or physical kist (travel chest) will be an important record of progress on our participants' journey. Any participant or group can put work in progress into the kist; images, music, text, artefacts that have been explored and could be used by another group as a stimulus for creating new work.
Kenny Bean will teach the groups how to develop a digital image bank of self-portraits and group work that will feed the creation of a performance space into which the local CPPs and participatory audience will be invited.
Jaimini Jethwa will be the film journalist travelling into and between each of the groups to create a composite docu-film that captures the confidence of the group at the beginning and end of the creative processes as detailed earlier.
The performance events at the end of the 20 week process will be proof of the impact of the project in themselves and they will all be recorded evaluation.
SLP is particularly interested in the ' valorisation' process [1] that will thread through, evolve and lead out of this project. We convene regular international practice-sharing events and policy briefings for our partnership of learning providers working across adult education (including our Political Literacy training courses, the Women as Social Entrepreneurs events, our 1000 Learners' Voices events and annual SAC funded Seed Gathering events to stimulate creativity across the formal and informal education sector divide). After we have delivered this evaluation plan and produced the three final 'documents', we will develop at least one national showcase event to disseminate the findings and optimise the value of the Pathfinder investment.
This event - or series of interventions - will enable us to share the learning from 'Talking 'bout my generation' as part of our ongoing mission to integrate creativity and cultural planning across Scotland's Learning Partnership's delivery and development plans. The interventions will impact too on our partners' effective delivery of the country's evolving cultural, educational and enterprise policies.
[1] "Originally a French term "valorisation" is a concept which is now widely used and accepted in the European educational and training community. It can be described as the process of disseminating and exploiting the results of projects with a view to optimising their value, strengthening their impact, transferring them, integrating them in a sustainable way and using them actively in systems and practices at local, regional, national and European levels."