« Previous | Contents | Next »
Listen
5. STUDY FINDINGS AND CONCLUSIONS
5.1 INTRODUCTION
This Section draws together the main study findings and lessons. It begins by summarising the key findings and issues arising from the Pathfinder experience before going on to discuss some of the major lessons learned from the experience. Finally, it offers some alternative models derived from discussions with staff who took part in the interview process.
5.2 KEY FINDINGS AND ISSUES
This study has evaluated the progress and performance of both the Phase Two Adult Literacies Pathfinder projects and the framework put in place to support their design and delivery. From this review a number of main messages have arisen.
5.2.1 FOCUS AND TARGETING
The evaluation finds that the Challenge Fund process put in place by Learning Connections was successful in bringing forward a range of potentially valuable project actions. The Phase Two Pathfinder projects selected have been well targeted, and have addressed strategically significant literacies issues. As such they have been well placed to make an important strategic contribution to the issues and priorities set out in the original ALNIS report.
The Pathfinder projects have typically focused on designing and testing new structures and methods for attracting learners and supporting learners.
The strength of the literacies component has varied somewhat between projects, although literacies learning has been incorporated into all of the Pathfinder projects. In some cases this component could have been made more explicit or the outcomes from literacies learning more clearly monitored.
5.2.2 PROGRESS AND PERFORMANCE
Overall, the Pathfinder projects have progressed well, albeit subject to significant early delays. Difficulties during the set-up phase meant that projects have struggled in some cases to deliver agreed activities within a truncated timescale. Modifications to agreed project activity have been noted in most cases.
Common difficulties and constraints faced have included: recruitment difficulties; time taken to engage fully with the target group; low take-up of learning opportunities; technological difficulties; limited partner commitment or input; and a variety of resource constraints ( e.g. tutors, venues, crèche, and volunteers etc.).
Many of the challenges noted appear to have been unavoidable or have been a symptom of the learning curve that might be expected for organisations delivering new or innovative activity. However, in other cases a degree of over-ambition is noted in relation to the initial Pathfinder proposals.
At the close of the programme, the projects have all made satisfactory progress. Particular improvements in progress and performance have been evident during the final six months of implementation.
5.2.3 PROJECT MANAGEMENT AND DELIVERY
The evaluation indicates that the Pathfinder organisations put in place reasonably strong and supportive project management arrangements. Steering groups have proven to be particularly valuable in harnessing the skills, knowledge, and commitment of partners. A key issue identified, however, has been the limited management time that could be allocated to the Pathfinder from within host organisations, in light of available resources and competing priorities/demands.
The staff employed with Pathfinder funding have proven instrumental in the successful development and delivery of activities. However, the shortage of specialist literacies staff at the outset of the programme created significant recruitment difficulties for almost all of the Pathfinder projects.
Although a standard practice for the Scottish Executive, and agreed with project representatives at the outset, retrospective payment arrangements procedure caused cash flow difficulties in at least one case. It is important that arrangements for making payments 'in advance of need' should therefore be carefully considered in relation to voluntary sector grant recipients.
The delivery of activities in partnership with others has been central to the success of the Pathfinder projects. Joint working has typically broadened out over time, with staff successful in bringing new partners on board. That said, varying levels of 'buy in' from local partners have been evident and differing levels of connection to, or integration with, Adult Literacy and Numeracy Partnerships has been achieved.
5.2.4 PATHFINDER EVALUATION AND DISSEMINATION
Considerable effort has been devoted to putting in place a supportive framework designed to enable Pathfinder organisations to track, monitor, review activity and to share this learning with others.
From the perspective of the Pathfinder organisations, however, this effort was somewhat misdirected. The emphasis placed on generating and critically reflecting on performance information was limited both by the resources/time available to project staff and by the perceived weakness of the formative Evaluation Framework and tools put in place. In this respect further broad-based project management and capacity building support might have been more welcomed.
From a programme perspective, the Framework was not used to systematically synthesise and distil the learning from the Pathfinder projects or to share this learning with a wider audience. Greater emphasis on gathering and disseminating evidence might have been expected in a demonstration programme of this kind.
5.2.5 EFFECTIVE PRACTICE
Despite some difficulties in project implementation and monitoring, a variety of elements of effective practice have been identified as part of Pathfinder implementation.
Common elements of effective practice identified have included: methods of engaging with 'hard to reach' client groups; the use of community networks, resources and capabilities; learner involvement in the design of programmes; approaches to delivering learning and learner support; the use of various technologies, resources, and materials; approaches to partnership working and collaborative delivery; and methods of integrating literacies into wider learning/community activities.
Where learning did take place, learners welcomed the: gentle and informal means of introduction to literacies learning; learner-centred, responsive and flexible approach to courses; non-threatening and positive learning environment created; range of interesting and relevant activities for learners; opportunity to apply literacies learning in a real-world context; and the opportunity for learner progression through related forms of support.
5.2.6 OUTCOMES
On the basis of the research carried out it appears that outcomes are now emerging, to varying degrees, from all of the Pathfinder projects. These outcomes have been evident at four levels:
- learner outcomes - there have been wide ranging benefits for learners, although literacies learning has not always been to the fore and outcomes have not always been systematically captured;
- partnership outcomes - the Pathfinder projects have provided an important catalyst and focus for partnership activity locally;
- organisational outcomes - in some places the Pathfinder projects have introduced new ways of thinking and working locally; and
- infrastructure outcomes - in most cases tangible resources/materials are now emerging from the Pathfinder projects.
While these are positive outcomes, the evidence suggests that these are likely to have been strengthened with a longer timescale in which to set-up and deliver agreed activities. Due to early delays, much of the Pathfinder activity was only becoming fully embedded or reaching a point of maturity at the point of closure of the programme.
5.2.7 THE PATHFINDER LEGACY
The evidence indicates that exit strategies for the Pathfinder projects were initially underdeveloped and only seriously considered late in the life of the projects. For most of the Pathfinder projects there was no straightforward or managed transition from the end of funding. As noted below, however, 'exit strategies' were for the Pathfinder projects to arrange themselves (planning exit strategies was an important part of the Challenge Fund application process).
The Pathfinder projects, in all cases, have left some form of tangible legacy locally ( i.e. literacies materials, resources or activities). In five cases local partners have secured alternative grant funding from other sources to carry on the projects in a similar vein, if not always on the same scale. In three cases, local partners are attempting to mainstream the successful elements of activity into wider programmes of activity.
In each case it seems that partnership relationships and structures have been strengthened as a result of the Pathfinder experience. This bodes well for future adult literacies work and as such should be regarded as an important legacy from the Phase Two Pathfinder Programme.
A range of factors has determined the ability of the Pathfinder projects to sustain and build-on activities and outcomes. In particular, the extent to which the project: was established with an independent structure and identity; was closely linked into the structure, strategic priorities, and support of the Adult Literacy and Numeracy Partnership; and sufficient internal management capacity or external development support was in place to plan for continuation.
5.2.8 REPLICATION
The Pathfinder review undertaken as part of this evaluation reveals a range of interesting and effective practice that merits the attention of practitioners in other areas. The potential for learning lessons or replicating practice exists in each case.
Experience suggests, however, that it is not always possible to simply transplant project approaches or methods wholesale to other contexts. For each Pathfinder, a variety of local circumstances and pre-requisites have been identified as instrumental to project success and necessary as any attempt at replication elsewhere.
It is also noted that the process of transferring this learning and practice is still at an early stage. Dissemination to date has taken the form of a national dissemination conference in March 2005 hosted by Learning Connections, where all eight Pathfinder projects were given the opportunity to showcase their work. Dissemination has also taken place at a local level involving some of the Pathfinder projects.
The intention ultimately will be to disseminate the findings of the programme evaluation and the framework evaluation via Learning Connections' main website and also the website: www.adultliteraciesonline.com. Where possible and appropriate, the resources and materials produced by the Pathfinder projects will be posted on this site to aid dissemination.
5.2.9 VALUE FOR MONEY
In coming to an assessment of value for money achieved by the Phase Two Adult Literacies Pathfinder Programme there are three main perspectives: absolute; in terms of achievement against stated objectives; and in terms of cost-effectiveness when compared to similar programmes.
In absolute terms, the evidence indicates that the Pathfinder projects have achieved much with investment from the programme. They have undertaken project activity that has left a worthwhile legacy locally and in doing so produced a variety of learning. In this, programme investment has been additional and complementary to that allocated through Adult Literacy and Numeracy Partnerships. Moreover, in all cases but two (where projects had been piloted prior to the award of funding) the investment seems to have been instrumental in bringing forward new project activity.
In relation to objectives, the Pathfinder projects have helped to encourage innovative and collaborative literacies practice. In all respects the Pathfinder Programme has achieved the main objectives set for it.
In relation to cost effectiveness, there are no simplistic benchmarks. The Phase Two Programme was designed as a demonstration programme, and as such its ultimate success and cost-effectiveness will depend on the extent to which:
- the pilot activity is sustained and built upon locally;
- local learning is transferred or replicated elsewhere; or
- this activity results in new or improved literacies practice across Scotland.
5.3 CONCLUSIONS
5.3.1 LESSONS LEARNED
The Phase Two Literacies Programme itself was introduced as a bold and innovative initiative at the time, which has subsequently yielded positive results and learning.
The evaluation has indicated a number of important strengths of the Pathfinder approach. In particular, the approach has:
- enabled the targeting of strategically important literacies issues;
- induced organisations/agencies outwith the 'literacies field' to become involved in delivering literacies learning;
- provided the time and resources for the meaningful testing of new demonstration activity;
- offered the opportunity to gather real-time intelligence from innovative literacies practice; and
- uncovered a range of interesting and informative materials for those working in the literacies field and beyond.
On the basis of the evaluation evidence, and from the above discussion, a number of important lessons have arisen for Learning Connections and partners surrounding the delivery of any similar programme, including the importance of:
- establishing a robust programme governance framework, including input from an independent and expert Advisory Group, to ensure clear leadership and direction;
- maintaining tight management arrangements, to ensure a co-ordinated focus and sustained efforts from all of those tasked with implementing, and supporting the implementation of, the programme;
- establishing a straightforward project monitoring framework that lends itself to ongoing critical reflection and is tailored carefully to the needs, workloads, and capabilities of participating organisations;
- selecting projects with a clear and additional focus on literacies learning and providing adequate support to measure and track these agreed learning outcomes;
- building in adequate resources to project proposals from the outset to enable effective management, delivery, review, and dissemination of learning from the Pathfinder experience;
- providing ongoing, holistic, and flexible support to participating organisations, as necessary, to ensure the efficient and effective delivery of agreed project activities;
- providing sufficient time to organisations, including appropriate 'lead in' time to establish a realistic delivery plan and recruit staff, and adequate time to test project activities;
- facilitating connections between pathfinder activity and the work of local Adult Literacies Partnerships, to ensure that these activities are not delivered in isolation from local priorities and support structures; and
- ensuring a sustained focus and commitment on the part of participating organisations to the requirements of the Pathfinder programme and to progressing project 'exit strategies' against agreed milestones.
5.3.2 CHANGING INFLUENCES
In moving forward, it should be recognised that the literacies field has matured significantly since the establishment of the Pathfinder programme by the Scottish Executive in 2002.
Broad changes in this respect include:
- the continuing development of Learning Connections and its portfolio of activities designed to develop and promote innovative and effective practice;
- the evolution of Adult Literacy and Numeracy Partnerships and their increasing willingness and ability to stimulate and support innovation; and
- the breadth and scale of innovation in literacies practice that has now emerged naturally through investment from a variety of sources.
In addition, the current developments in the field of adult literacies are due to feed into a refreshed ALNIS report. This refreshed document will set revised priorities and actions.
5.3.3 ALTERNATIVE MODELS
In recognition of the above, and as part of this review, consultees have put forward various alternative models for Learning Connections to consider in stimulating innovative and effective practice in the future.
From these we have distilled four main options, which are not mutually exclusive, and include:
1. Thematic Pathfinder Projects
Under this option a further small number of large-scale pathfinder projects would be commissioned. This strategically targeted series of demonstration projects would be focused on a small number of areas where gaps in literacies practice still clearly exist, and which are in keeping with the priorities of the refreshed ALNIS strategy. The support framework for these projects would be redesigned to reflect the learning from the Phase Two Pathfinder Programme.
2. Action Research
Under this option a set of strategically focused small-scale action research projects would be initiated. These community-based projects would provide practitioners and partners with the time and resources for planning, action, reflection and learning. These projects might be initiated with support from action research mentors, and could be linked to or established in partnership with the Scottish Community Action Research Fund ( SCARF).
3. Seeding Innovation
Under this option a seedcorn fund would be established to support small-scale innovative actions within agreed themes. This would provide small amounts of grant funding to identify, develop and pilot innovative or effective ways of supporting literacies learning. This might be modelled on, or broadened out from, the existing Innovations in ICT and Literacies Project and would include appropriate support to extract and disseminate the learning from the pilots.
4. Research and Development
Under this option a programme of research and development activity would be initiated. This would focus on identifying and harvesting learning from existing literacies practice throughout Scotland, rather than initiating or supporting new project activity. This might involve: the commissioning of targeted research and studies into effective practice; targeted policy development work with partners in relevant fields; and the development of mechanisms to share effective practice.
Whatever the option or combination of options adopted it will be important to maximise the value of learning from the Phase Two Pathfinder experience. As noted, the ultimate success of the programme will depend on the transfer of learning and knowledge, and the use of this, to bring about improvements in literacies practice throughout Scotland.
« Previous | Contents | Next »