« Previous | Contents | Next »
Listen
SUMMARY
CONTEXT
Since 1999, the Scottish Executive has pursued a strategy to tackle poverty and facilitate social inclusion which complements that of the UK Government. The Scottish Executive supports the UK commitment to eradicate child poverty within a generation, has not used its tax-varying powers to diverge from UK policy, and has focused effort and investment on employability and a welfare-to-work strategy.
In 2004, a Cabinet review of the Social Justice Strategy ( SJS) led by the Minister for Communities resulted in the revision of Scottish Executive's social inclusion policy and the adoption of the Closing the Opportunity Gap ( CtOG) strategy. CtOG reflects the Scottish Executive's desire to focus on the most important issues and activities required to address poverty in Scotland, and concentrate on areas for which the Scottish Executive is responsible. The three basic aims of CtOG are to prevent poverty, provide routes out of poverty, and to sustain poverty-free lives. Six CtOG Objectives were announced in July 2004, and 10 CtOG Targets in December 2004. Within the Scottish Executive, the Social Inclusion Division is responsible for overseeing cross-Departmental social inclusion policy, and undertakes annual monitoring of progress of social inclusion indicators.
EVALUATING CTOG
The second Scottish Executive proposed a three-part framework for evaluating CtOG. This was to comprise:
- Part one: on-going monitoring of the 10 CtOG Targets
- Part two: assessing the effectiveness of Departmental activity contributing to the achievement of these 10 Targets
- Part three: assessment of the effectiveness of CtOG as a whole.
Part three - assessment of the effectiveness of the CtOG as a whole - was to comprise three phases:
- Phase one: overall evaluation for the CtOG Programme
- Phase two: analysis of CtOG impact on the opportunity structures available to people in Scotland
- Phase three: analysis of CtOG processes
This study deals with phase one of the third part of the CtOG evaluation, by analysing progress toward achieving the CtOG Aims, Objectives and Targets up to January 2007.
RESEARCH AIMS
There are three aims in this part three phase one evaluation.
First, to analyse CtOG progress on three levels:
i. towards achieving the overarching CtOG Aims of tackling poverty in Scotland
ii. towards fulfilling each of the 6 higher-level CtOG Objectives
iii. assessing the rate and direction of progress towards the 10 lower-order CtOG Targets.
Second, to advise on measuring, analysing and assessing progress towards Aims, Objectives and Targets across different social groups and local areas in Scotland.
Third, to comment on the impact of CtOG Targets and Objectives upon the three CtOG Aims in the context of Scottish Executive and UK policy interventions.
METHODS
The research on which this evaluation is based comprised three bodies of work.
First, stage 1 comprised three tasks: evaluation of progress towards CtOG Aims, evaluation of progress towards CtOG Targets, and appraisal of the extent to which CtOG is contributing towards the Scottish Executive's Equalities agenda. These evaluations made use of available data sources.
Evaluation of progress towards CtOG Aims involved analysis of the incidence and trends in poverty in Scotland over the course of the operation of CtOG; i.e. since the 2004 baseline. This drew upon UK-wide survey and other quantitative data, such as Households Below Average Incomes statistics ( HBAI). Where possible, analysis was disaggregated below the Scotland level to explore variation by geographic, social and demographic groups. Sub-group variations were also explored through interviews with representatives of equality groups (described further below).
Evaluation of progress towards Targets involved re-examining the CtOG monitoring data routinely collected by the Scottish Executive Departments and officials responsible for each respective Target. Interviews were also undertaken with these Scottish Executive 'Target Owners' to examine any distinctive features of the data and inform the overall understanding of the CtOG implementation and monitoring strategy.
The relationship between CtOG and the Scottish Executive's Equalities strategy 1 was examined by two means: firstly, examining evidence of socio-economic and demographic differences in national survey sources and Target monitoring data; secondly by undertaking interviews with representatives of equality groups to explore their perceptions of CtOG and estimations of any changes in the social and economic circumstances of minority and disadvantaged groups in recent years.
Stage 2 evaluated progress against the 6 CtOG Objectives for the general Scottish population, for specific populations of equalities groups, and for those living in areas of high deprivation. This evaluation involved analysis of nationally representative survey data, including Scottish Household Survey ( SHS) data, and other routinely collected Scottish Executive data. The suitability of data for the analysis of each Objective was determined by assessing the relative relevance, availability, coverage and frequency of different sources. A standardised analytical template was applied to these diverse datasets to ensure a consistent appraisal between Objectives. Recommendations for future stages of analysis were generated from these appraisals.
Finally, in stage 3, the preliminary analysis of progress towards CtOG Aims from stage 1 was undertaken again to examine the extent to which apparent trends in the data could be attributed to interventions encouraged by CtOG Targets and, more generally, the wider range of activities that contribute to CtOG Objectives.
Evaluating the impact of CtOG at this interim stage entails certain difficulties requiring careful contextualisation and interpretation of data. January 2007 is too early to assess progress for many CtOG Targets, almost all CtOG Objectives, and the overall CtOG Aims. Where an assessment of outcomes is not possible, this study has reviewed the progress of the work programme that underpins CtOG, advised on measurement issues, and provided baseline data against which future evaluations of CtOG can be made.
PROGRESS I - AIMS
Poverty in Scotland is falling. However, this trend pre-dates CtOG and the fall is experienced unevenly across social groups. Particular progress has been made in reducing child poverty and pensioner poverty. Given the UK Government and Scottish Executive's commitment to address child poverty, further reductions in the level of child poverty may be expected. However, the key challenge is whether the persistence of poverty for some groups of children can be addressed, and whether poverty among adults of working age can be reduced as CtOG impacts in the years ahead.
The Scottish Executive should continue to utilise HBAI data to monitor progress toward achieving CtOG Aims. However, further attention should be directed at utilising the Scottish Household Panel Survey and the Scottish Household Survey to monitor poverty in Scotland.
PROGRESS II - OBJECTIVES
It is premature to reach anything other than a qualified evaluation of progress towards CtOG Objectives. There are two reasons for this: CtOG Objectives are still in the process of being fully operationalised, and not all the necessary data has been established to assess progress. For example, it is premature to evaluate progress for Objective 6 as the CtOG work programme to improve services in rural Scotland did not start until 2005; there are plans to evaluate the outcomes of this by the end of 2007
Nevertheless, uneven progress across CtOG Objectives can be discerned - improvements are evident in health status, but difficulties persist in effecting change in both enhancing sustained employment and improving the confidence and skills of young people. Trends are also highly uneven across sub-populations.
This Report facilitates the evaluation of CtOG Objectives by offering recommendations for operationalising these and providing benchmark analyses against which progress can be gauged in subsequent evaluations.
PROGRESS III - TARGETS
While there is more evidence relating to progress in achieving CtOG Targets than Objectives, nevertheless, this evaluation was undertaken before the end dates for each Target, and it is premature at this stage to reach definitive conclusions on the success of meeting CtOG Targets.
Nevertheless, it is possible to evaluate the rate and direction of progress for those Targets whose work programme has been operational for more than one year. Notable progress has been achieved for Targets A, C, D and K. Notwithstanding, limitations with data available to evaluate Target B, there is some evidence of progress being less than would be expected for Targets B and G.
The quality of data available to evaluate CtOG Targets is highly variable as is the experience of progress across social groups: no single trend adequately describes the circumstances of all sub-populations - progress varies for different groups and for different Targets.
IS CtOG MAKING A DIFFERENCE?
CtOG comprises a wide-ranging programme of activities and the attempt to co-ordinate policy streams across a range of different areas into an integrated strategy to tackle social exclusion in Scotland.
CtOG Objectives address most of what are known to be some of the principal factors contributing to poverty and social exclusion in Britain, to the extent that these are within the scope and responsibility of the Scottish Executive. These factors include obstacles to labour market participation, educational underachievement among some young people, neighbourhood decline, poor health, and lack of access to or poor quality services. Inevitably, CtOG does not address matters reserved to the UK government which are associated with poverty and exclusion; i.e. benefit and tax credits levels, and labour market demand.
Difficulties remain in the availability of data to assist the evaluation of the impact of and value added by CtOG. However, this interim evaluation has identified an increasing sophistication of data and analyses which will considerably assist future evaluations of Scottish Executive social inclusion policy.
Some positive changes in the landscape of social exclusion in Scotland are evident, particularly with regard to progress towards CtOG Targets. It is reasonable to attribute some of these changes to CtOG, and provisionally conclude that they would not otherwise have occurred without this programme.
Interviews with key informants from Equalities groups identified progress in a number of areas of CtOG activity, and have highlighted certain CtOG measures which are valued and which may effect positive change in CtOG outcomes in the years ahead.
However, for the most part, it is too early in the development of CtOG to reach anything other than highly qualified comment on the rate and direction of progress. In particular, it is premature to comment with any degree of certainty on the impact of the work programme that underpins CtOG Targets on changes evident in CtOG Objectives and CtOG Aims.
The period to which this evaluation of CtOG pertains, is that of the second Scottish Executive.
« Previous | Contents | Next »