« Previous | Contents | Next »
Listen
CHAPTER THREE: INTERVIEWS WITH KEY INFORMANTS
OVERVIEW
3.01 The interviews with the ten Target Owners and support staff provided contextual information on CtOG and insights which informed the design of the synthesis report and models for evaluating CtOG as a whole. This chapter summarises the key findings that informed design decision-making, presenting an evidence base for the recommendations set out in chapters four and five.
3.02 Findings are presented in three sections: CtOG Target work programme, CtOG Target monitoring and CtOG Targets, Objectives and Aims.
CTOG TARGET WORK PROGRAMME
3.03 The interviews explored the work programmes that were being used to achieve the CtOG Targets. In addition to providing an insight into the nature of these work programmes, additional views were provided on the relationship of work to achieve the Target and the wider work of the Department and Executive, Target Owners' awareness of relevant work beyond their direct responsibility, delivery responsibilities, timescales and the effectiveness of CtOG population targeting.
CtOG Target and Departmental work
3.04 In some instances, CtOG Targets were perceived by Target Owners to be an integral part of a wider programme of Departmental work, as opposed to an activity added to a pre-existing programme of work. This was perceived by some to be one of the strengths of the CtOG Target.
3.05 In other cases the CtOG Target was not only considered to be an integral part of departmental work but was conceived as being reflective of, and dependent on, a wider range of departmental activity. For example, Target D was selected to be indicative of progress towards Scotland's overall Health Improvement Plan (Improving Health in Scotland: the Challenge 2003), which contains a wide range of different indicators. The Financial Inclusion Action plan is also slightly wider in its remit than the explicit CtOG Targets. Although this presents a challenge in terms of effecting change through policy development (ultimately the CtOG Target is dependent on a raft of departmental activity, rather than a narrowly specified intervention), it does provide the potential for CtOG Targets to be seen as indicators for progress within the department as a whole.
3.06 CtOG Targets are therefore a reflection of departmental activity, rather than a distraction from the core work of a department.
3.07 The evaluation of CtOG should thus consider not only progress against the delivery of each Target, but also document the links between wider departmental activities and poverty reduction. For example, Target A is a crucial target for the Employability Framework, in addition to the Framework supporting achievement of the Target. The role that these links play, and the means by which they develop and influence outcomes, should be considered throughout the evaluation.
Awareness of work programmes or joined-up working?
3.08 CtOG Target Owners demonstrated an awareness of how their Target impinged upon, and was impacted upon by, other CtOG Targets.
3.09 However, there is more evidence of awareness of the focus of other CtOG Targets than there is of their work programmes. Similarly, there is more evidence of awareness of the potential relevance between CtOG Targets than there is of truly joined-up working among CtOG Target Owners.
3.10 In some cases however, links are recognised explicitly in CtOG action plans. For example, there are clear working and delivery links between the Targets A and B and between Targets B and G.
Responsibility for delivering CtOG work programmes
3.11 The CtOG Target Owner, and lead staff within their Divisions, is responsible for implementing a programme of work in Scotland, which will lead to the CtOG Target being achieved.
3.12 All Target Owners also have responsibility for undertaking equality mainstreaming targets (including consideration of rural communities), to ensure that the impacts of CtOG upon key equalities groups are considered and measured.
3.13 CtOG implies partnership working at the level of the CtOG Target. Such partnership working takes a variety of forms: using Scottish Executive funding to effect changes in local practice (e.g. in local authorities) which have been specified by the Scottish Executive; using Scottish Executive funding to effect local change, which has been specified locally, whilst ensuring that it is consistent with CtOG; and working collaboratively within the Scottish Executive to ensure that CtOG can be monitored.
3.14 CtOG Target leads are frequently presiding over a range of localised delivery targets. Of particular note is Target H (rural area in service delivery) in which the Target is locally determined. Furthermore, other Targets, while centrally determined, measure outcomes for clearly specified localities. For example, local targets have/will be agreed for each Target A (Employability) local authority area, each Target D health board and each Target B NEET 'hotspot' area. Target K (Financial Exclusion) will have no overarching national targets, only locally specific agreements. The emphasis in all of these cases has been on obtaining agreement at a local level of targets that are ambitious, but realistic.
3.15 The key strength of the partnership working that CtOG necessitates, is that CtOG is a collaborative project. The necessity of local delivery and local targeting heightens the relevance of CtOG at the 'local' level.
3.16 The extent and nature of partnership working present challenges for CtOG. Co-ordinating activity across a number of target areas presents a workload challenge for many CtOG Target holders. More generally, the reliance of CtOG on 'local' delivery necessitates close co-operation between the Scottish Executive, local government and other agents. It also allows for the possibility of national target successes masking varied local level performance. Finally, the broader context of global or UK influences on CtOG outcomes presents a tier of contributors, who cannot be brought into the CtOG programme but are influential upon its outcomes.
Timescales
3.17 Ultimately, the timescales/work programme for CtOG Targets are determined by the overarching need to deliver CtOG within the current parliamentary session. Thus, CtOG is conceived as a programme of work and a series of Targets that must be achieved, in the main, by 2008. Target A is the exception, with an overall end date of 2010.
3.18 There is a variable pattern of whether work programme timelines are set. A minority of CtOG Targets have a clearly specified programme of work with interim deadlines and schedules, while the majority have defined an end point (the date by which the CtOG Target will be achieved) and programme of work without interim deadlines. Whilst a range of local level monitoring is in place ( see Chapter 2 for more detailed information) no local interim targets have yet been agreed.
3.19 Most CtOG Targets are working to schedule. However, some CtOG Targets are progressing at a slower rate. These Targets are generally based on 'devolved' target setting and heavily disaggregated implementation. Although there is no immediate danger that CtOG Targets will not be achieved by the specified end dates, operationalising disaggregated Targets is proving to be a more protracted process.
3.20 This variety in progress towards Target achievement is also reflective of the complexity of the work programmes linked to each Target. Whilst Target K (financial exclusion) will be met if improved service availability has led to increased service uptake by 2008, Targets A (employability) and J (regeneration) are examples of Targets that are dependent upon much larger programmes of work, and are contingent upon a wider range of external factors. It will be important to its success that the evaluation fully considers these issues.
CtOG Targeting
3.21 There is significant variation in the population base for the Targets. Target populations are defined geographically, socially, by a combination of social and geographical traits and by institutions e.g. Target J focuses on the 'most deprived neighbourhoods' (geographical); Target B focuses on 16-19 year olds who are NEET (socially); Target A focused on the workless in seven Local Authority areas (society-in-place) and Target C (jobs in the NHS) focuses on an institution ( NHS Scotland). Furthermore, several of the target populations are defined by multiple traits, e.g. Target H focuses on remote and disadvantaged rural areas.
3.22 The majority of the Targets have a geographical focus of some kind, even if this is not explicit in their target wording. Target B will, in practice, focus upon nine local authority areas that are identified as ' NEET hotspots'. Similarly, the final delivery stage of Target K will focus upon the 11 local authority areas identified as being in most need of improved financial services and Target C has focussed only upon the five health boards that have opted into the programme.
3.23 Target Owners consider that the Targets are directed to those most in need of intervention or, at least, are more targeted toward groups in need than the Social Justice Milestones.
3.24 Some Target Owners also perceive that data could be deployed more effectively in the future to facilitate a more nuanced identification of need. This, it was perceived, could lead to further refinement in target setting at a later date. Better understanding, for example, of levels of debt amongst people accessing credit unions, and of the longer-term destinations of Scottish school leavers, could improve the resource targeting for Targets K and B respectively.
CTOG TARGET MONITORING
3.25 Full information of Targets and monitoring plans are available on the Scottish Executive Closing the Opportunity Gap website: http://www.scotland.gov.uk/topics/people/social-inclusion/17415/opportunity . Information is updated regularly by Target Owners.
3.26 The interviews explored progress that was being made toward monitoring and achieving CtOG Targets. Insights were provided of baseline data availability, on-going monitoring and evaluation, and prospects for achieving CtOG Targets.
Baseline data
3.27 There was uneven availability of baseline data across CtOG Targets. However, all CtOG Targets without baseline data had established, or were working toward establishing, a set of baseline data.
3.28 The absence of baseline data for some CtOG Targets is problematic in that it hampers the potential to monitor change toward the CtOG Target before the end dates in 2008.
3.29 Precise monitoring arrangements for each Target are discussed in greater detail in Chapter 2.
On-going monitoring and evaluation
3.30 On the whole, there is limited formal and publicly released monitoring of progress toward achieving some of the CtOG Targets, i.e. annual or quarterly measurement of change from the baseline. This finding is not surprising given the challenges faced in operationalising CtOG Targets and establishing the means to measure the CtOG Target and in the limited number of interim CtOG Targets that have been set.
3.31 Data are however being monitored according to the frequency with which they are made available. In some cases data are available annually, in others availability is more regular. It is, for example, impossible to measure progress towards Targets D (health inequalities) and K (financial exclusion) on anything other than an annual basis. Progress towards Target C (Jobs in the NHS) is however recorded monthly, and DWP returns for Target A (employability) are received quarterly. The Scottish Executive has committed itself to making the most up to date Target monitoring statistics available online.
3.32 CtOG Targets have developed at different points in time - some owing their form to Social Justice Milestones, other being devised post SJM - and the difficulties faced in implementing work programmes to address CtOG Targets (such as devolved implementation and target setting) have rendered quarterly and annual monitoring an unrealistic goal for some CtOG Targets at present. However, should CtOG progress beyond the current parliamentary session, then a detailed programme of on-going monitoring and evaluation would be a desirable element of the programme for all Targets.
3.33 We consider that the lack of a formalised programme of monitoring, and of publicly available data on progress towards CtOG, is a weakness of the programme, making it harder for stakeholders other than that Target Owners to respond to variation in progress towards the Targets, and limiting public accountability.
3.34 We also note that the division of responsibility along departmental lines limits the potential for developing co-monitoring and evaluation strategies across departments, and suggest that such a strategy would enhance the position of CtOG as an integrated cross-cutting programme.
Prospects for achieving CtOG Targets
3.35 Most Target Owners delivered an unequivocal message that their CtOG Target will be met by the end date.
3.36 Although acknowledging the interdependency of CtOG Targets and the wider context of societal and UK policy contexts, no CtOG Target Owner opined that the destiny of their Target was 'contingent' on external factors - the destiny of the CtOG Target is understood to rest with their department and the work programme that has been specified.
3.37 In some instances, the CtOG Target could be achieved through prudent interpretation of CtOG Target specification. For example, Target E (integrated support for children and young people) will have been met if an integrated system of support is in place, regardless of the outcomes that such as system may have, and Target K (financial exclusion) will be met if advice is available regardless of the impacts that this availability leads to. Although this could be interpreted as a sleight of hand, it would be more appropriate to conceive of this as a measured response to the implementation timescale and the limitations of some of the available data.
CTOG TARGETS, OBJECTIVES AND AIMS
3.38 The interviews explored the role of each CtOG Target within the wider CtOG programme. Insights were provided of links between CtOG Targets, links between the CtOG Target and CtOG Objectives, and the overall CtOG Aims.
Joint working among CtOG Targets
3.39 As discussed above, the extent to which CtOG Target Owners have established cross-target links is variable. Where links exist, they tend to take the form of information sharing. There is little collaborative working across CtOG Targets, and no joint monitoring arrangements.
3.40 The primary forum for joint working across CtOG Targets is the joint meetings of the CtOG Target Owners (and analysts). Target Owners report valuing these meetings for the opportunity to share general concerns and to discuss experiences of the work of a Target Owner.
3.41 Outside formal meetings among all CtOG Target Owners, ad-hoc meetings are arranged to discuss shared concerns. These meetings tend to focus on matters of substance.
CtOG Targets and CtOG Objectives
3.42 Target Owners were less focused on CtOG Objectives. Indeed, of the three tiers of CtOG (Target, Objective and Aims), the Objective tier was the only one with which Target Owners expressed any reservation over its purpose and importance. In some cases the Targets were defined as realistic delivery measures of the Objectives.
3.43 Although each CtOG Objective can be aligned to a specific policy field, the impacts of individual Objectives are likely to extend beyond their 'core' policy fields. For example, Objective 2 (To improve the confidence and skills of the most disadvantaged children and young people - in order to provide them with the greatest chance of avoiding poverty when they leave school) not only enhances educational attainment, but may also improve employability prospects, thereby increasing the likelihood of avoiding financial exclusion. Furthermore, four of the CtOG Objectives are to be tackled to both enhance employability prospects and to improve quality of life.
3.44 Responsibility for CtOG Objectives was not perceived to rest with the Target Owners, even for those Objectives that were most closely aligned to a CtOG Target. Target Owners varied in their views of where that responsibility rested. Some felt it was with the Social Inclusion division whilst other felt it was with their wider Department or with the Scottish Executive, and its delivery agencies, as a whole.
3.45 It should however be noted that the Target Owners do not have overall responsibility for the delivery of the Objectives, and that other Scottish Executive officials, who it was not within the scope of this work to interview, may view this relationship differently.
3.45 The CtOG Objectives have been embedded into a range of different policy frameworks, for example, the Objectives are reflected in the Regeneration Outcome Agreements, the aims of the Enterprise Networks, Futurebuilders Scotland guidance and the new Big Lottery Fund priorities. CtOG is also a cross-cutting priority for the Enterprise Networks. However, none of the Target Owners discussed this directly.
3.46 The evaluation of the CtOG Objectives will therefore have to consider that the Objectives are seldom understood as delivery outcomes. The evaluation should therefore test the assumption that targeted delivery activity will lead to the national change that the Objectives aim to achieve.
CtOG Targets and CtOG Aims
3.47 Target Owners convey a strong belief, commitment to, and a shared sense of common purpose in CtOG.
3.48 Overall Target Owners believe that the success of CtOG would lead to a range of positive change for the Scottish population, including improved quality of life, higher employment rates, longer life expectancies and less debt.
3.49 Target Owners, understandably, perceived a key role for CtOG Targets in the overall CtOG Aims. Less clarity was expressed with regard to the role of CtOG Objectives.
SUMMARY
Targets are often clearly linked to a wider departmental programme of work. The success of some Targets is much more dependent upon wider work programmes than others. However, all of the CtOG work programme necessitates strong partnership working.
Whilst Target delivery activities have various impacts upon each other, there is little joint delivery and no joint monitoring.
The majority of the Targets have to be achieved by the end of 2008. Few Targets have specified interim delivery points.
Most Target Owners are collecting some form of interim data, not all of which is made publicly available.
There is variety across Targets in the quality and quantity of available baseline data.
A large proportion of CtOG monitoring is taking place at a local level, in response to locally agreed Targets that feed into the national picture.
There are no delivery streams linked to the six CtOG Objectives.
Target Owners believe that CtOG will have been successful if a range of poverty reduction aims have been met, including improved quality of life and increased economic activity across Scotland.
Achievement of these aims is seen to rest upon the delivery of a wide range of national and local initiatives and strategies.
« Previous | Contents | Next »