« Previous | Contents | Next »
Listen
The full research report of which this document provides a summary (Identifying and Quantifying the Outcomes of Community Learning and Development - Development and Testing of a Research Instrument by Avante Consulting) can be found atwww.scotland.gov.uk/learningconnections.
Why was this study undertaken?
There is increasing emphasis on all public services focusing clearly on outcomes - on the difference they make.
Existing evidence shows that CLD has important outcomes in personal development and building community capacity, which relate closely to the outcomes set out in the national performance framework.
It is important to be able to identify and evidence these outcomes more precisely, and as far as possible to quantify them.
Key findings
This pilot study:
- Confirmed the contribution of CLD to key positive outcomes -
- People becoming more confident individuals, effective contributors, responsible citizens and successful learners
- Improvements in how community groups function
- Positive impact on communities.
- Through doing this, demonstrated a viable means of identifying outcomes of CLD.
- Indicated that identifying outcomes of CLD retrospectively provides one useful source of evidence, which should be complemented by evidence gathered in other ways.
Aim
The aim of the research was to develop and assess methods that identify and quantify, as far as possible, the impact of community learning and development ( CLD). The impact of CLD was defined as the distance travelled in relation to the outcome framework set out in Delivering Change: Understanding the Outcomes of Community Learning and Development1 (2006) by individuals and communities over a specified period.
Context
The research was set in the context of a range of work aimed at increasing understanding of outcomes in CLD, improving the evidence base of the outcomes delivered, and strengthening the focus of CLD practice on achieving outcomes.
This includes: the work of the former Learning Connections Performance Information Project Working Group, which identified the need for further research into CLD outcomes; Delivering Change, which set out for the first time the range of outcomes expected; How Good is Our Community Learning and Development?22( HGIOCLD?2), the HMIE self-evaluation framework; and a programme of support for outcome based practice in CLD, commissioned by Learning Connections.
This programme of work was brought into sharp focus by the November 2007 Concordat between the Scottish Government and the Convention of Scottish Local Authorities ( COSLA). Many of the outcomes and indicators set out in the National Performance Framework relate to the work of CLD.
The study built on a review of existing evidence on the links between the activity of ( CLD) and the difference that it makes ( Literature Review of the Outcomes of CLD3, Communities Scotland 2006). The task was to develop and pilot an instrument that could test whether it is possible to identify and quantify retrospectively the outcomes of CLD for individuals and communities in Scotland. The retrospective approach was adopted in order to assess the validity of an alternative to a longitudinal study of the outcomes of CLD.
Approach
The study took place in three distinct phases during 2007:
1. Construction of an instrument to measure outcomes. Based on a review of relevant literature, a range of outcome indicator statements were drafted and tested with CLD stakeholders across the country in a series of workshops. A statistical analysis was used to help finalise the instrument to be field tested by clarifying whether the indicator demonstrated the desired outcome.
2. Testing the instrument for validity and to identify ways in which it offered reliable results; as well as to understand the impact of CLD activity in the pilot population sites of Aberdeen and Aberdeenshire. Briefing meetings were held with managers and staff, resulting in 43 practitioners supporting the test. The fieldwork then took place over six weeks during the autumn of 2007.
3. Analysis of the findings to establish the contribution that the instrument can make and recommendations for the future.
There were a number of constraints that had a bearing on the study, these include issues of attribution, retrospection, links to inputs and outputs, worker influence, numbers of participants in each phase, study timeline, participant feedback and measurement scale.
Findings
Sample
- 231 participants took part in the test of the instrument
- 74% of participants were female
- 24% were young people (up to the age of 25) and
- 33% were over 60 years of age.
- The majority (66%) of participants were engaged in adult learning activity
- 19% of participants were engaged in youth work
- 15% were engaged in community capacity building activity and
- 53% were involved in CLD as either volunteers or committee members.
Outcomes
Overall, there was a positive distance travelled toward the defined outcomes by the respondents who completed the personal development indicators. This means that CLD is supporting people to become more confident individuals, more effective contributors, responsible citizens and successful learners. Equally, across a range of indicators, those involved in community capacity building provide evidence of improvement in how their groups function and the impact that this has on communities.
Conclusions
The instrument that was developed was tested in order to establish its validity, to identify ways in which it offers reliable results, and to understand the impact of CLD activity in the pilot populations. Scrutiny of the data results allowed testing of the instrument's validity and the reliability of findings.
Was taking part in the study a positive experience for participants and CLD staff?
With the support of workers in the field and despite the difficulties of some aspects of the design of the instrument, participants largely enjoyed the opportunity the instrument provided them in terms of personal reflection. Their efforts were supported by managers and CLD workers across a range of settings and sectors, all of whom were keen to understand if their work made a difference to the lives of those they support and the communities in which they live.
Did the study identify positive outcomes of CLD?
The data indicates that across a range of indicators CLD did make a difference. These indicators not only directly correlate to the requirements of Delivering Change, but are also directly relevant to the government's aspirations set out in the National Performance Framework.
Is the instrument that was developed a useful tool for providing evidence of the impact of CLD?
The study confirms that the instrument can assist in providing evidence that CLD is able to make a contribution to key national as well as local outcomes; and particularly that it dovetails well with the development work on implementing Curriculum for Excellence in Scotland's schools.
Can the instrument be used or developed as a practice tool?
The task was the development of an instrument which supports national research into the outcomes of CLD activity. There may be scope for using the instrument as the basis for developing practice tools which help with evaluation: assisting those involved in CLD activity to understand how to adapt approaches which will provide the best possible chance of achieving the desired outcomes.
What needs to happen next?
To date there is little national CLD outcome research and this instrument seeks to support the establishment of a body of evidence in this field. While the instrument needs to be further refined, the study suggests that if it were used alongside longitudinal research and more developed practice tools, CLD would be more able to evidence its impact. CLD should be informed by the same body of evidence and rigour as is provided by research in parallel professions.
Recommendations
Key recommendations are detailed in the report. They include:
1. The Scottish Government and other stakeholders, including Community Planning Partnerships, note that the instrument appears to be a valid measure of CLD outcomes and as such should be further developed;
2. The Scottish Government, CLD managers and workers, promote understanding of the instrument as one of a series of activities that support evidence of CLD outcomes. The instrument should be understood to be complementary to work which seeks to demonstrate the relationship of inputs and processes to the outcomes of CLD;
3. Lifelong Learning Directorate undertake research that seeks to demonstrate the relationship between CLD outcomes, inputs and processes, including a longitudinal study.
« Previous | Contents | Next »