The Scottish Government

« Previous | Contents | Next »

Listen

ANNEX 2: FURTHER DETAIL ON TARGET MONITORING

CtOG Target A - Reduce the number of workless people dependent on DWP benefits in Glasgow, North & South Lanarkshire, Renfrewshire & Inverclyde, Dundee, and West Dunbartonshire by 2007 and by 2010.

2.01 Summary: CtOG Target A is characterised by the following:

• Strong baseline data
• Local level targets
• Extensive local level monitoring and research to be coordinated by the Employability Unit
• No consideration of they types of work that people move into, or of low pay
• A range of additional delivery issues that could be considered in the overall CtOG evaluation

2.02 Current monitoring data: Numbers of people in each local authority area who are in receipt of Incapacity Benefit, Jobseekers Allowance and Income Support.

2.03 Frequency of analysis: Analysis is undertaken quarterly.

2.04 Monitoring end date: Monitoring will end in March 2010, at the end of the Target delivery period.

2.05 Baseline data: A baseline was established in all authorities in August 2004. In Glasgow, the baseline is May 2003, as this was the point at which Glasgow developed its own worklessness targets (which now comprise its local level CtOG Targets - see 2.19).

2.06 Additional monitoring: Local level targets have been established in each of the 7 authorities receiving CtOG funding. There have been developed with guidance from the central CtOG Target lead. Local authorities have been encouraged to set targets that are aspirational, but also realistic in light of local economic circumstances. These targets are being monitored at a local level by each authority.

2.07 There are no local level interim targets and there are no plans to make progress towards local level targets public. If specific authorities do not meet their local level targets, but the national figures meet the CtOG Target requirements, the national target will be considered to be achieved.

2.08 Future plans for additional monitoring: The Scottish Employability Framework contains provision for the creation of an Employability Unit. From April 2006, the Unit will have responsibility for monitoring the progress of local authorities towards locally agreed worklessness targets. At this point responsibility for monitoring overall progress of the seven CtOG areas will pass to the Employability Unit. Each authority will be expected to sign a local level agreement outlining what action they are taking, and how it is being monitored and evaluated. The agreement will also be expected to highlight the structures they have in place to promote joined up working. The Unit may also decide to measure progress against other indicators or research other areas of progress (for example, partnership working).

2.09 Each of the seven CtOG areas will receive funding to develop their local plans. The funding will be managed alongside the Community Regeneration Fund and the ROA process, and will not be restricted (as other CRF funding is) to specified data zones. The regeneration outcome agreement ( ROA) monitoring will therefore also include additional employment targets.

2.10 Delivery issues currently outside of Target monitoring and evaluation: The local authorities receiving CtOG funding will also be central to the delivery of the Scotland's Employability Framework. A key aim of the framework will be the establishment of local partnerships setting out how local agencies should be working together. The work undertaken in the seven CtOG areas is expected to inform the national roll out of the Employability Framework. Research on the processes of partnership operation will therefore be important to inform this process. Such research will also allow for variations in the effectiveness of these partnerships in particular areas (for example engagement with employers) to be assessed.

2.11 The DWP benefits data, which are being used to measure Target progress, do not make provision for the measurement of distance travelled towards employment, including improvements in skill level amongst the local population, or researching the effectiveness of different local strategies as a means to achieve these ends (these issues may however be picked up by new monitoring initiated by the Employability Unit).

2.12 Need for further monitoring and evaluation: There is no further need to extend the monitoring of this target, as it is already comprehensive in its coverage. Given the recent DWP announcement that from November 2005 earnings information will be included in the work and pensions longitudinal study consideration could however be given to how this data could be incorporated into target monitoring. A range of additional delivery issues (highlighted above) should be considered in the evaluation of the CtOG Objectives, and overall CtOG strategy.

CtOG Target B - Reduce the proportion of 16-19 year olds who are not in education employment or training by 2008.

2.13 Summary: CtOG Target B is characterised by the following:

• Only high-level baseline data
• Local level targets with corresponding monitoring
• Further data sources in development
• Further issues to be monitored as Scotland's NEET strategy develops

2.14 Current monitoring data: There are ongoing difficulties with monitoring progress towards CtOG Target B. Current monitoring is via the Labour Force Survey ( LFS). Very significant changes in the numbers of young people who are not in education, training or employment (% of 16-19 year olds who are NEET) would however be required to impact upon LFS data. It is not clear if the proportional shifts likely to result from CtOG will be recorded by LFS data. Despite recent boosts to the LFS sample there are therefore concerns that there are not currently any measures in place to monitor local level changes in the numbers of young people who are NEET. Consideration is currently being given to combing LFS data with DWP benefits data, but it is unclear whether access difficulties with the latter datasets can be resolved.

2.15 The overall CtOGNEET strategy, which was being finalised at the time of writing of this report, will specify a range of precise measures of success. It is likely to set a national target of a reduction of between 5 -10% in the numbers of young people who are NEET. Local level targets will support the national target. These will be set in each Scottish local authority, according to specific local circumstances. It is likely that outcomes measures will include the numbers of young people who are NEET, school leaver destinations, and for individual sub groups, including looked after children and other groups who are assessed to be at risk of becoming NEET. Five NEET local authority 'hotspots' have been identified, where the process of local target setting will be accelerated. These areas will also receive additional funding (via the Regeneration Outcome Agreements ( ROAs)) to support work towards their target. It is not yet clear whether there are plans to make the achievements of local level targets available publicly.

2.16 Frequency of analysis: Analysis is undertaken annually.

2.17 Monitoring end date: Monitoring will end in December 2008, at the end of the Target delivery period.

2.18 Baseline data: The LFS provides a baseline for the numbers of young people who are NEET in Scotland. Local level data is not currently available.

2.19 A dditional monitoring: The local areas receiving additional funding will be subject to monitoring arrangements specified in their ROAs. There will also be overlaps with Employability Framework monitoring, as this will also include specific provision for young people who are NEET.

2.20 Future plans for additional monitoring: Consideration is currently being given to developing the Careers Scotland database as a means to improve monitoring. At the point of the interview with the CtOG Target Owner discussions between Connexions, DfES and the Scottish Executive were ongoing. Scottish School Leavers survey data may also be combined with these data. Were this option successful it would allow pupil level data from schools to contribute to Target monitoring. A full delivery plan on how this will be taken forward will be available from March 2006.

2.21 Delivery issues currently outside of Target monitoring and evaluation: Many issues will be flagged up in the full NEET strategy, which will be monitored and evaluated at a local level. These will include the building of local partnerships, including work with schools, and ensuring that outcomes for groups of particularly vulnerable young people are achieved.

2.22 Need for further monitoring and evaluation: To measure small local changes in the numbers of young people who are NEET there is a need for more specific local level information to be made available. As far as is possible the Scottish Executive is addressing this issue, but it is not yet clear when, if at all, this improved monitoring information will become available.

CtOG Target C - Public sector and large employers to tackle aspects of in-work poverty by providing employees with the opportunity to develop skills and progress in their career. NHS Scotland will set an example by providing 1000 job opportunities, with support for training and progression once in post, between 2004 and 2006 to people who are currently economically inactive or unemployed.

2.23 Summary: CtOG Target C is characterised by the following:

• Full monitoring of outcomes achieved in different health boards
• Further evaluation could consider wider range of delivery issues
• Locally specific developments could also be recorded by further research
• No work being undertaken by public sector and large employers other than NHS Scotland (and Target Owner who only has responsibility for the work being undertaken by NHS Scotland).

2.24 Current monitoring data: Five health boards (Fife, Forth Valley, Grampian, Greater Glasgow, and Lothian) are currently participating in the initiative. Of the participating boards, all have been signed up to the initiative for different lengths of time. Current monitoring arrangements cover how many people in each authority have been offered places on pre-employment training, how many people have completed the programme, how many people have entered employment with NHS Scotland, whether people are still in employment after 13 weeks and whether participants achieve any additional outcomes (e.g. further education or employment outside of the Health Sector). The numbers of people entering courses in each area claiming incapacity benefit (IB) and Job Seekers Allowance ( JSA) are also monitored. All remaining boards have a commitment to offer training and job opportunities to individuals claiming 'New Deal'. Every Board records the number of New Deal contracts its holds as part of the staff governance standard ( NHS Reform Act (Scotland) 2004). This information is reported on annually and will be considered within the Target.

2.25 Frequency of analysis: Local health board employer coordinators liaise with Jobcentre Plus to obtain local figures on a monthly basis. These are collated centrally.

2.26 Monitoring end date: Monitoring will be completed at the end of July 2006 (4 months following the end of the initiative in its current form), when sustained employment outcomes will be known.

2.27 Frequency of analysis: Returns on the numbers of pre-employment placements offered by each board are available monthly. Analysis is conducted annually to assess numbers of places offered, completed and translated into job entries.

2.28 Baseline data: No baseline data is necessary as this target relates to a new initiative.

2.29 Additional monitoring: The CtOG Targets are part of a wider equality and diversity work programme being undertaken by NHS Scotland. As each of Scotland's 23 health boards are individual employers they are responsible for their own equalities monitoring and evaluation work, so whilst further local monitoring may currently be in progress it is not being centrally collated. This is not however being collated

CtOG Target D - To reduce health inequalities by increasing the rate of improvement for under 75 Coronary Heart Disease mortality and under 75 cancer mortality (1995-2003) for the most deprived communities by 15% by 2008.

2.30 Summary: CtOG Target D is characterised by the following:

• Health monitoring in place
• Local level responsibility (highly devolved) for specific targets
• Potential for further research into the different approaches taken by community health partnerships

2.31 Current monitoring data: The overall health improvement aims are taken from the Improving Health in Scotland: The Challenge that incorporates 23 specific health inequality measures. Six of these are used as the health inequalities targets in Building a Better Scotland and two of them are being used for CTOG Target D. Progress is being monitored annually from data collected form Health Boards.

2.32 Frequency of analysis: Progress against the Targets is recorded annually.

2.33 Monitoring end date: Monitoring of current targets will end in December 2008, at the end of the Target delivery period.

2.34 Baseline data: These arrangements have been in place since March 2003, with full baseline data established by March 2004. Historic analysis of the Carstairs index of deprivation has allowed a baseline for CHD and cancer rates in deprived areas to be established from 2003. The Health Department are however planning to move to using the Scottish Index of Multiple Deprivation, and to create a new baseline. This baseline data should be available by December 2006. HDASD to confirm.

2.35 Additional monitoring: Community health partnerships are responsible for the development of their own local level targets. These are monitored at a local level. This information is not collated centrally.

2.36 Future plans for additional monitoring: There are no plans for any additional monitoring related to this target.

2.37 Delivery issues currently outside of target monitoring and evaluation: More research could be useful on the operation of community health partnerships, and their different approaches to tackling the national health improvement plan.

CtOG Target E - By 2008, ensure that children and young people who need it have an integrated package of appropriate health, care and education support.

2.38 Summary:CtOG Target E is characterised by the following:

• Not measurable by use of quantitative data
• Delivery by local agencies
• Wide range of different stakeholders involved in achieving target

2.39 Current monitoring data: The nature of the target means that it is not measurable by use of datasets. Progress towards developing integrated approaches to the provision, planning and delivery of children's services will be monitored through integrated Children's Services Plans and through the joint inspection of children's services (proposals being developed by a joint Children's Services Unit, based within Her Majesty's Inspectorate of Education). As part of the work on joint inspection of children's services, a coherent quality assurance framework relevant across services for children and young people will be developed. This will draw on and complement existing performance frameworks, including the National Priorities for Education, NHS Performance Assessment Framework and Integrated Early Years Strategy outcome measures.

2.40 Monitoring end date: All plans for monitoring and assessment are still in development.

2.41 Frequency of analysis: No set interim progress report dates have been set.

2.42 Baseline data: No baseline data is necessary as this target relates to a new initiative, progress towards which is not measurable by datasets.

2.43 Future plans for additional monitoring: There are no plans for additional monitoring of progress towards this target.

2.44 Need for further monitoring and evaluation: At present it is not possible to say whether further monitoring is required to measure achievement against the specified target.

CtOG Target F - Increase the average tariff score of the lowest attaining 20 per cent of S4 pupils by 5% by 2008.

2.45 Summary:CtOG Target F is characterised by the following:

• Strong baseline and ongoing monitoring
• Delivery at a local level throughout Scotland
• Wide range of local level variation likely, which could be a useful area for further research

2.46 Current monitoring data: Data on attainment in National Courses is obtained annually from the Scottish Qualifications Authority ( SQA). The SQA attainment information only includes those pupils who take National Courses, therefore data from the annual school census is also included to bring in those pupils who fail to gain an award.

2.47 Monitoring end date: Monitoring will end in April/May 2008 when the tariff scores from the academic year 2006/2007 are received.

2.48 Frequency of analysis: Analysis is undertaken annually. Calculation of the average tariff score of the lowest attaining 20% is made in April/May of each year. This is based upon the achievements of the pupils beginning S4 in the September of the two years previously.

2.49 Baseline data: A baseline was established on 30th September 2004 (based upon S4 pupil achievements in the 2002/2003 academic year).

2.50 Additional monitoring: The tariff score of the lowest attaining 20 per cent of pupils has been built into the school improvement framework Local education authorities have therefore set their own local targets towards this aim. These are not being centrally collated.

2.51 Future plans for additional monitoring: At present it is possible to measure any achievement gained by these pupils in S5, and it will also eventually be able to link to their leaver destinations.

2.52 Need for further monitoring and evaluation: No further monitoring is required to measure achievement against the specified target.

CtOG Target G - By 2007 ensure that over 50% of all "looked after" young people leaving care have entered education, employment or training.

2.53 Summary:CtOG Target G is characterised by the following:

• A wide range of different delivery streams impacting upon achievement of the target
• Monitoring still in development, including new monitoring pilot
• Local authority level monitoring of targets

2.54 Current monitoring data: A range of different work feeds into the achievement of this target including fostering, aftercare services, health education and enterprise. In addition, a range of new regulations concerning young people in care and after care service have recently (April 2004) come into force. To support local authorities in meeting their obligations under these Regulations the Executive has also funded a £1.2 million pilot Care Leavers Leadership Academy project (delivered by Columba 1400) to assist young people in developing the necessary life skills required to make a successful transition from the care system to independent living. All local authorities in Scotland are able to access places on this programme should they wish. The monitoring and evaluation of these initiatives feed into the CtOG monitoring arrangements.

2.55 Monitoring activities have been available since 31 March 2004 but there are currently attempts to develop this monitoring activity further. In early November 2005 local authority implementation officers met with the Scottish Executive to discuss monitoring arrangements for both the new regulations and the CtOG Target. Monitoring will be at a local authority level.

2.56 An improved system of local authority monitoring of looked after children is currently being piloted. This involves individual statistics for each looked after young person. There have however been some difficulties with the information recorded and compatibility between IT systems. It is not therefore yet clear whether or not the system will be implemented nationally and will be set up in time to contribute to CtOG monitoring activity.

2.57 Monitoring end date: Monitoring will continue on an annual basis, and the end date for monitoring this target is 31 March 2007.

2.58 Frequency of analysis: Annually.

2.59 Baseline data: A baseline was established on 31st March 2004.

2.60 Additional monitoring: The evaluation of Columba 1400, and of the implementation of the new regulations for looked after children, will complement the CtOG monitoring.

2.61 Future plans for additional monitoring: Attempts are being made to allow monitoring of this target by different children characteristics (e.g. gender, ethnicity, etc.). This is dependent on the success of the new pilot collection currently taking place.

CtOG Target H - By 2008, improve service delivery in rural areas so that agreed improvements in accessibility and quality are achieved for key services in remote and disadvantaged communities.

2.62 Summary:CtOG Target H is characterised by the following:

• No plans for overarching national targets, only for locally specific measures
• No targets, baseline data or monitoring arrangements currently in place
• Wide diversity of locally specific work being undertaken to achieve Target

2.63 Current monitoring data: Targets are currently being negotiated with each of the 22 Rural Service Priority Areas ( RSPA). All monitoring will be at a local level, based on the particular baseline situation in each Community Planning Partnership area. Targets will comprise a mix of quantitative and qualitative measures, specific to the local targets that are agreed. Some satisfaction measures will also be taken from the Scottish Household Survey. The possibility of using the Ordnance Survey dataset, Points of Interest, to map public infrastructure in rural areas is also being considered.

2.64 Monitoring end date: Monitoring activities will end on 31st March 2008.

2.65 Frequency of analysis: Analysis will be conducted at March 2006 and March 2008.

2.66 Baseline data: Consultation with the 22 RSPAs areas (and their respective Community Planning Partnerships) is being undertaken during November 2005, and will lead to the development of a baseline by March 2006.

2.67 Future plans for additional monitoring: The Scottish Executive is considering improving their baseline information by undertaking an initial survey of perceptions of service access in the 22 relevant RSPA areas. Monitoring activities will end on 31st March 2008, and analysis will be conducted in March 2006, and March 2008. Within a few months rural/urban analysis of the Households Below Average Income dataset will be possible, which will contribute to Target monitoring.

2.68 Delivery issues currently outside of Target monitoring and evaluation: There is no monitoring of how the delivery process is working across Community Planning Partnerships ( CPPs), which could be an area for further investigation.

2.69 Need for further monitoring and evaluation: Once the systems outlined above have been implemented, no further monitoring will be required to measure achievement against this target.

CtOG Target J - To promote community regeneration of the most deprived neighbourhoods, through improvements by 2008 in employability, education, health, access to local services, and quality of the local environment.

2.70 Summary:CtOG Target J is characterised by the following:

• A mix of local and national targets
• Majority of monitoring arrangements now in place
• Large volume of data to collate centrally
• Application of measures from Targets A and F to the 15% of most deprived areas

2.71 Current monitoring data: Success of Target J will be partially indicated by progress against the targets for employment, health and education (Targets A, D and F) in the 15% of most deprived areas. Monitoring for these Targets will therefore inform Target J. Success is also contingent upon measurable improvements to the quality of the local environment in the most deprived neighbourhoods. The Scottish Household Survey will comprise the key measure of this aspect of the Target. Consideration is being given to the use of the Ordnance Survey dataset (Points of Interest) as a means to measure access to local services. The Scottish Executive is also exploring if and how Local Environmental Audit and Management Systems ( LEAMS) could be used. Success if also contingent upon the delivery of the local outputs and outcomes set out in each Regeneration Outcome Agreement ( ROA). A National Performance Management Framework is being used to monitor the outcomes laid out in these agreements. Responsibility for its implementation sits with individual Community Planning Partnerships, and includes local surveys, use of Scottish Household Survey data, and use of national benefits data.

2.72 Monitoring end date: Monitoring will end on 31st March 2008.

2.73 Frequency of analysis: The employment deprivation aspect of Target J is measured through the employment domain of SIMD and will be updated when SIMD is updated (i.e. October 2006). Targets D and F are disaggregated to the 15% most deprived areas'. Assessment of the delivery of ROAs is particular to each CPP. No indication of the likely frequency of analysis of improvements to the local environment has been given. It is unclear whether this information is being collated centrally.

2.74 Baseline data: Baseline data has been established for the employment, health, education and neighbourhood satisfaction elements of Target J.

2.75 Additional monitoring: Communities Scotland is undertaking a formative and summative evaluation of the ROAs. The recently published Social Focus on Deprived Areas provided a wealth of indicator data with which to monitor progress. The Executive are currently working to provide time series data for a number of indicators.

2.76 Need for further monitoring and evaluation: Issues concerning the defining and monitoring of the 'access to local services' element of Target J are still outstanding.

CtOG Target K - By 2008 increase the availability of appropriate financial services and money advice to disadvantaged communities to reduce their vulnerability to financial exclusion and multiple debts.

2.77 Summary: CtOG Target K is characterised by the following:

• Ongoing monitoring of credit union membership (based on Annual Returns submitted by credit unions to the FSA)
• Limits to the extent that the impacts of CU membership are currently being, and can be, measured (as noted in recently published Scottish Executive research)
• Range of additional local monitoring sources
• Monitoring not yet established for Financial Inclusion Fund

2.78 Current monitoring data: There are three strands of Target K activity, which have been subject to a range of monitoring arrangements: Credit unions, money advice and financial education. Scottish CU membership figures are derived from the Annual Returns credit unions have to fill in for the FSA. The Credit Union Capacity Fund (set up in September 2003, first grant made in January 2004, last one to be made in April 2006; grants are either for 1 or 2 years) is being monitored through interim and annual reports which report back on Targets and outcomes; an evaluation is likely to take place after autumn 2006; a decision on whether this will be commissioned has not yet been taken. 11 money advice projects for vulnerable groups have been funded around Scotland. An independent evaluation is running parallel and will report in spring 2006 with interim findings and a final report is due in 2007. The increase in the number of money advisers is also relevant to this target. The third strand is financial education work. These projects are funded or co-funded by the Scottish Executive. Each is independently undertaking monitoring activities that have been agreed with the Scottish Executive.

2.79 Frequency of analysis: CU membership: annual returns, but the FSA had not received the complete 2004 data by November 2005. CU Capacity Fund: annual (interim reports only for 2-year projects). Money advice projects record all clients and other monitoring data, and the external evaluation team report to the Scottish Executive 5 times during the project duration of 2 years. On financial education, reporting is specific to each project. Scottish Executive research staff have had no involvement in this work beyond early comment on the projects' monitoring proposals. Regular financial returns are received on all funded projects in addition to the specified monitoring activities.

2.80 Baseline data: There is a clear baseline for the proportion of people who are members of credit unions in Scotland. Each CU project funded under the Capacity Fund has known baseline data. The money advice for vulnerable groups projects are not about baseline monitoring - instead they are about finding effective ways of developing services for groups to whom mainstream money advice access was problematic; each project is being developed with a view to mainstream it. The number of money advisers and the number of new money advisers are known. DN: do the financial education projects have a known baseline and are baselines appropriate here. There is neither clear baseline for levels of debt in Scotland nor any monitoring of indebtedness levels. A range of datasets currently in development/recently developed (see table 2.5) could however be used to develop a future baseline.

2.81 Additional monitoring: Many local areas are receiving/will also be receiving additional funding from their local authority and Communities Scotland (via Regeneration Outcome Agreements) for local money advice initiatives. These funding streams have separate monitoring requirements.

2.82 Future plans for additional monitoring: Spending of the Financial Inclusion Fund has not yet started as the proposals from LAs are not due until February 2006 (and will be discussed and finalised by 1 April 2006). This will be shared proportionally between the 11 local authority areas that have been identified as having significant financial exclusion problems (this was partly done through combining SIMD data with SHS data on savings and bank accounts). One rural area (Highlands and Islands) is included in the 11. The local level nature of the issues that will be addressed make it likely that local targets will be more effective than high level national monitoring.

2.83 Consideration is also being given to how the quality of data received nationally from money advice projects can be improved. At present there is no national case management system that means that only top-level data regarding the number of people accessing projects can be measured. No information is available on the outcomes of accessing that advice, or on local level changes in people's financial circumstances/levels of debt. On indebtedness, the SE is looking to the forthcoming ONS Wealth & Assets Survey to meet this information gap.

2.84 Delivery issues currently outside of Target monitoring and evaluation: Evaluation of local level operation, and of how the financial inclusion fund is tailored to specific local circumstances, would be useful for the development

2.85 Need for further monitoring and evaluation: As per the Scottish Executive's views a national case management system for money advice projects would make it possible to monitor outcomes of money advice and whether levels of debt are being impacted by greater access to financial advice.

« Previous | Contents | Next »