« Previous | Contents | Next »
Listen
Executive Summary
Task
As part of national work identifying the links between the activity of Community Learning and Development ( CLD) and the difference that it makes, Learning Connections, which is part of the Scottish Government's Lifelong Learning Directorate, commissioned research that is the subject of this report.
The task was to develop and pilot an instrument that could test whether it is possible to identify and quantify retrospectively the outcomes of CLD for individuals and communities in Scotland.
Context
The work was set in the context of a range of work underway supporting evidence of outcomes in CLD. This includes:
- the recommendations of the former Learning Connections Performance Information Project Working Group, which identified the need for further research into CLD outcomes;
- the development of the Government's 2006 'Delivering Change: Understanding the Outcomes of Community Learning and Development' which sought to support more thoroughly the contribution of CLD to outcomes, by setting out for the first time the range of outcomes expected;
- How Good is Our Community Learning and Development?2 the HMIE self-evaluation framework supporting improved quality in CLD practice and other work underway supporting outcome based practice for CLD; and
- The November 2007 groundbreaking Concordat between the Scottish Government and the Convention of Scottish Local Authorities which set out a National Performance Framework with 15 national outcomes and 45 associated indicators and targets, many of which relate to the work of CLD.
Approach
The study took place in three distinct phases during 2007. The first phase saw the construction of an instrument to measure outcomes, which entailed both desk research together with the input of CLD stakeholders. The second was a pilot of the instrument in order to test validity and to identify ways in which it offered reliable results; as well as to understand the impact of CLD activity in the pilot population sites of Aberdeen and Aberdeenshire. The final phase entailed analysis of the findings to establish the contribution that the instrument can make and recommendations for the future.
Constraints
There were a number of constraints that had a bearing on the study, these include issues of attribution, retrospection, links to inputs and outputs, worker influence, numbers of participants in each phase, study timeline, participant feedback and measurement scale.
Findings
- 231 participants took part in the test of the instrument
- 74% of participants were female
- 24% were young people (up to the age of 25) and
- 33% were over 60 years of age.
- The majority (66%) of participants were engaged in adult learning activity
- 19% of participants were engaged in youth work
- 15% were engaged in community capacity building activity and
- 53% were involved in CLD as either volunteers or committee members.
Overall there is a positive distance travelled toward the defined outcomes by the respondents who completed the personal development indicators. This means that CLD is supporting people to become more confident individuals, more effective contributors, responsible citizens and successful learners. Equally, across a range of indicators, those involved in community capacity building provide evidence of improvement in how their groups function and the impact that this has on communities.
Conclusions
The instrument that was developed was tested in order to establish its validity, to identify ways in which it offers reliable results, and to understand the impact of CLD activity in the pilot populations. Scrutiny of the data results allowed testing of the instrument's validity and the reliability of findings. In effect, by asking questions of the data, analysing the answers and then asking questions of the answers it is possible to then test the validity of the instrument itself.
With the support of workers in the field and despite the difficulties of some aspects of the design of the instrument, participants largely enjoyed the opportunity the instrument provided them in terms of personal reflection. Their efforts were supported by managers and CLD workers across a range of settings and sectors, all of whom were keen to understand if their work made a difference to the lives of those they support and the communities in which they live.
The data supports their aspiration that across a range of indicators the work does make a difference. These directly correlate not only to the requirements of Delivering Change, but also those in areas of direct relevance to the Government's aspirations set out in the National Performance Framework.
Even given the constraints of causality, the statistical analysis of outcomes demonstrates a positive message. This is supported by the analysis of the data generated by completion of the instrument both in terms of what it can tell us about distance travelled and the patterns of responses.
The study therefore confirms that the instrument can assist in providing evidence that CLD is able to make a contribution to key national as well as local outcomes; and particularly that it dovetails well with the development work on implementing Curriculum for Excellence in Scotland's schools
It is important to note that the task was the development of an instrument which supports national research into the outcomes of CLD activity. It was not about evaluation. There may be scope for using the instrument as the basis for developing practice tools which help with that: working with those involved in CLD activity, assisting them to gain an understanding of how to adapt approaches which will provide the best possible chance of achieving the desired outcomes. Such outcome based practice also enables a direct correlation to be drawn between the outputs (what was done and how) and inputs (the resources deployed).
The practice that is adapted as a result of this information should flow through and be demonstrated in the results of national research, which will also support evidence of CLD outcomes. To date there is little national CLD outcome research and this instrument seeks to support the establishment of a body of evidence in this field.
While there is work to be done to further refine the instrument, the data that it can produce, suggests that were it to sit alongside a longitudinal research and more developed practice tools, CLD may be more able to evidence the impact of such work. Research data flowing from the use of such an instrument would provide important trend analysis over time. This is largely absent for CLD. Research in parallel professions is frequent, substantive and given considerable credence. CLD should be informed by the same body of evidence and rigour.
Recommendations
Key recommendations include:
1. The Scottish Government and other stakeholders, including Community Planning Partnerships, note that despite the constraints of the study, the instrument appears to be a valid measure of CLD outcomes and as such should be further developed.
2. The Scottish Government, CLD managers and workers, promote understanding of the instrument as it develops as being of use as part of a series of activities that support evidence of CLD outcomes. The instrument should be understood to be complementary to work which seeks to demonstrate the relationship of inputs and processes to the outcomes of CLD.
3. Directorate General Education undertake research that seeks to demonstrate the relationship between CLD outcomes, inputs and processes, including the longitudinal study noted as under consideration in the brief for this research and recommended by the PIP group.
4. Learning Connections, Aberdeen and Aberdeenshire undertake further analysis of the volume of data available from the test in order to maximise the available information for the two pilot sites.
« Previous | Contents | Next »