The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER FOUR: DESIGN OF SYNTHESIS REPORT

OVERVIEW

4.01 This chapter presents a blueprint for the proposed synthesis report.

OBJECTIVES OF SYNTHESIS REPORT

4.02 One of the two project aims was to identify appropriate contextual data and evidence to include in a synthesis report. The purpose of the synthesis report is to draw together individual CtOG Target monitoring data and relevant internal and external statistical data into a single report to present a picture of progress within the CtOG context.

4.03 The target market for the synthesis report was not specified in the project brief. However, given that CtOG is conceived as a collaborative project involving a wide range of stakeholders and clients, it is assumed that the synthesis report must attend to the needs of a wide readership of delivery agents, policy analysts and client groups.

4.04 The synthesis report will provide an opportunity for the Executive to reflect upon CtOG progress at a mid-point in the programme, before the final evaluation reports. Given that full monitoring data for the overall evaluation will not be provided until late 2008, it is unlikely that a full evaluation report will be available and agreed until 2009/2010. The publication of the synthesis report will therefore ensure that the Executive are able to assess progress towards the CtOG Targets, Objectives and Aims prior to the end of the programme, enabling decisions regarding the final stages of CtOG, and post- CtOG planning, are informed by comprehensive and accurate evidence.

4.05 The synthesis report will also provide key stakeholders from outside the Executive with the opportunity to inform their current practice, and future strategic decisions, with learning from the CtOG programme.

4.06 The synthesis should therefore be written and presented in such a way as to facilitate learning from key delivery lessons, and to inform all stakeholders (within and external to the Executive) about progress towards all aspects of CtOG delivery.

KEY ISSUES

4.07 It is envisaged that the synthesis report will review progress for each tier of CtOG activity.

4.07.1 It is recommended that the review of the ten CtOG Targets should comprise the following elements:

CtOG working processes

Progress towards CtOG Targets

Understanding of progress made towards CtOG Targets

4.07.2 It is recommended that the review of the six CtOG Objectives should comprise the following elements:

CtOG working processes

Progress towards CtOG Objectives

Understanding of progress made towards CtOG Objectives

4.07.3 It is recommended that the review of the CtOG Aims (as an Anti-Poverty Strategy) should comprise the following elements:

CtOG working processes

Progress towards CtOG Aims

Understanding of progress made towards overarching CtOG Aims

4.08 The synthesis report will draw upon robust and readily available sources of data that inform understanding of conditions of life in Scotland.

4.09 The synthesis report will inform the CtOG evaluation. The data that is utilised in the synthesis report will be an integral part of the evaluation of the three tiers of CtOG activity (Targets, Objectives and Aims). However, the objective of the synthesis report is to inform, whereas the evaluation aims to appraise; thus although utilising (at least in part) the same information, the presentation and focus of both reports will differ. The emphasis in the synthesis report will be on the effective communication of key findings.

4.10 It is recommended that the authors of the synthesis report retain a degree of independence from those responsible for delivering or implementing CtOG. This would enhance the credibility of the report. On the other hand, it is essential to draw from the experience of those responsible for CtOG in the synthesis report, in particular to inform the assessment of progress towards the CtOG Targets, and the qualitative analysis of working processes.

4.11 An Independent Review Panel should be set up to oversee the production of the synthesis report and the overall evaluation of CtOG.

4.12 The Editor (or Editorial team) of the synthesis report should be a suitably qualified and experienced consultant drawn from the commercial, voluntary or academic sector. The editor, or editorial team, whether independent consultants or Scottish Executive staff, should work under the overall supervision of this Panel.

4.13 Contributory authors should be drawn from the Scottish Executive (Target Owners). Given the breadth of the CtOG programme, it is unrealistic to expect an Editor (or Editorial Team) to possess the breadth of experience required to provide expert commentary on specific aspects of CtOG. Hence, it may also be necessary to budget for commissioning expert commentary on specific aspects of CtOG.

DESIGN RECOMMENDATIONS

Context

4.14 Four factors have informed the design recommendations for the synthesis report:

• The desirability of engaging a wide audience with different information requirements.
• The need to provide an integrated and comprehensive review of CtOG.
• The scale of undertaking (financial and other costs).
• The desirability of participative engagement with readers in which lessons learned may be incorporated in on-going and subsequent CtOG activity.

4.15 These factors have shaped the recommended design of the synthesis report in various ways:

• The variable information requirements of different audiences necessitate the presentation of the synthesis reports in a range of formats.
• The need to provide an integrated and comprehensive review of CtOG necessitates the publication of a comprehensive analytical report.
• The scale of undertaking mitigates against the publishing a full analytical report on an annual basis.
• The desirability of participative engagement suggests a role for a regular review of CtOG.

Key Recommendation

4.16 A three-tier approach is recommended, which comprises a comprehensive analytical report, a series of shorter thematic digests, published annually, each of which is aimed at more narrowly defined target audience and a series of educational resources and accessible summaries of key findings.

Analytical Report

4.17 Scottish Ministers, MSPs, policy makers (Scottish Executive and beyond), analysts (including academic observers) and service managers (those implementing the CtOG work programme) will require a comprehensive synthesis report which reviews CtOG.

4.18 The substance of this synthesis report is set forth in 4.07. It will comprise data analysis (quantitative and qualitative), descriptive commentary, critical analysis and conceptual reviews.

4.19 Although the analytical report we are recommending is a substantial and comprehensive review, it must be stressed that the objective is to inform and not to appraise CtOG. Hence, careful consideration must be given to the presentation of the report.

4.20 The report will provide a substantial addition to existing online progress reports published by Target Owners. As discussed in 4.07 it will: allow for extensive reflection on the processes by which the CtOG programme has been implemented; provide time-series monitoring of progress towards all CtOG Targets; provide reflection on the range of factors that have impacted upon the CtOG Targets; provide an indicator set for monitoring progress towards CtOG Objectives, and an assessment of progress towards the Objectives to date; provide an indicator set for monitoring progress towards the overarching CtOG Aims, and an assessment of progress towards the Aims to date; provide reflection on the different programmes that have contributed towards the CtOG Objectives and overarching Aims.

4.21 We consider that such a report is an essential mid point reflection on the CtOG strategy, drawing together progress to date, and allowing for understanding of reasons for varied progress to inform future policy developments and the final delivery stages of the programme.

4.22 The tension outlined in 4.13 between facilitating participative engagement and the 'costs' of producing a full analytical report should be resolved by publishing a full analytical report on a regular, but not annual basis. A publication cycle of two or three years between publication is recommended. In practice, this will mean publication of one synthesis report during the current round of CtOG Targets. Given the likelihood of the continuation of some form of Target activity beyond 2008/2010, the report should be produced in a format that will allow replication in future years.

Thematic Digests

4.23 The thematic digests should attend to the need of subject specialists for regular CtOG information to inform understanding, policy and practice in their area of interest.

4.24 They should be published annually, for each year of CtOG's operation.

4.25 Given that thematic digests are applied, in the sense that they are focused and geared toward policy and practice, it is not recommended that digests are published for CtOG Objectives, which are conceived as multi-dimensional. Rather, eleven thematic digests should be published, one each for the CtOG Targets and one for CtOG Aims (the programme as a whole, poverty rates and dynamics). Although the CtOG programme as a whole comprises a multi-dimensionality, there is long-standing concern with poverty in Scotland and an established group of voluntary sector organisations with an anti-poverty remit; a poverty thematic digest would attend to the information needs of the anti-poverty movement in Scotland.

4.26 As with the full analytical report, the desirability of communicating with a wide readership necessitates careful consideration of the presentation of the thematic digests. It is recommended that digests are no more than two A4 pages in length, are published in a CtOG house style and that they are updated on an annual basis.

4.27 The digests should include available data on progress towards CtOG Targets, and descriptive commentary on the CtOG work programme.

4.28 It would be necessary to maintain strong editorial control over the substance of the thematic digests. However, given the overarching objective to communicate more widely than, for example, through the analytical report, it may be preferable to employ a Publishing Consultant to work in conjunction with the Editor (Editorial Team) to produce user-friendly digests.

Educational Resources

4.29 The UN Convention on the Rights of the Child advocates that children are consulted on matters that pertain them (subject to the capability of the child). Given that much of CtOG concerns children and young people, there may be merit in disseminating CtOG directly to these populations, or through appropriate representative participation networks.

4.30 More generally, there may be merit in commissioning the production of educational resources that would facilitate the dissemination and debate of CtOG throughout the education sector in Scotland (secondary school, Further Education and Higher Education).

4.31 At this stage, no recommendation is made to the most effective and appropriate way to engage young people in Scotland. However, the Scottish Executive are encouraged to consider making an explicit statement / commitment to this issue and to further consider how the work of CtOG can be communicated to different groups of children and young people in Scotland.

Accessible summaries

4.32 Other population groups that are the focus of CtOG Targets, for example people with low skills and people in rural areas, and the key equalities groups, should also be provided with the opportunity to engage fully with the CtOG programme. Appropriate summaries of key findings and progress towards the CtOG Targets, Objectives and overarching Aims should therefore be made available in formats which are accessible to population groups who are often marginalised from such processes.

DATA FOR SYNTHESIS REPORT

Analytical report

4.33 The full set of data that could contribute to the analytical report has been presented in Chapter 2.

4.34 It is clearly important the timing of the synthesis report allows for the inclusion of the maximum volume of data measuring progress towards each CtOG Target and Objective to be included.

4.35 From our analysis of current Target monitoring, and potential data sources for monitoring progress against each Objective, we recommend that mid-2007 would be the best point at which to publish an analytical report into CtOG impacts. This will allow a record of progress made up to and including 2006 to be included.

4.36 The data that it would be possible to include in the analytical report is shown in the following table:

Table 4.1 Data availability for inclusion in the analytical report at August 2007

4.37 Overall, the quality and availability of data mean that the analytical report should be able to discern the extent of progress towards CtOG Targets, Objectives and overarching Aims.

4.38 As discussed in Chapter 2, there is some variation in data quality and quantity available for each CtOG Target.

4.39 As referenced in Chapter 2 there may also be a range of different local evaluation data that it is relevant to include in the synthesis report. Precise details about the reporting requirements upon local delivery bodies (Community Planning Partnership, local health boards and local authorities) are still in development for all CtOG Targets. Where possible, the Scottish Executive is incorporating any additional reporting requirements into the Regeneration Outcome Agreement reporting process.

4.40 To ensure that local level progress and Targets are documented the external consultant commissioned to undertake the CtOG evaluation work should continue to work with the CtOG Target leads to determine the precise reporting requirements that are placed upon local delivery agencies, and to consider how they can best be included in the analytical report.

4.41 Further developments in monitoring, for example the potential development of the Careers Scotland database to improve Target B (%16-19 year olds who are NEET) monitoring (for more information see the full Target summary on page 60), or the use of the Ordnance Survey dataset (for more information see the full Target summary page 66) for Target H (service provision in rural areas), should also be recorded and considered for inclusion in the analytical report.

Thematic digests

4.42 The full set of data that could contribute to the thematic digests has been presented in chapter 2.

4.43 It is important the thematic digests report on key progress indicators for each CtOG Target, and for the achievement of the CtOG overarching Aims. From our analysis of current Target monitoring, and potential data sources for monitoring progress against each Objective, we recommend the thematic digests are published annually each April of the operation of the main CtOG work programme (2006, 2007 and 2008).

4.44 This will allow a record of progress made up to and including 2006 to be included. The data that it would be possible to include in the thematic digests is shown in the following table:

Table 4.2 Data availability for inclusion in the annual thematic digests

4.45 Overall, the quality and availability of data mean that the thematic digests should be able to provide a meaningful overview of progress towards CtOG Targets and the overarching CtOG Aims.

SUMMARY

The synthesis report should take the form of one full analytical report, two annual sets of 11 thematic digests and accessible resources for children and young people.

By Summer 2007 sufficient data will be available to document change against each Target baseline, and progress towards the achievement of the overall CtOG Objectives and overarching Aims. We recommend that a full analytical report be produced at this point.

In a few specific cases there may be difficulties in discerning progress due to a lack of data availability. As noted previously, as far as possible CtOG Target leads are however already addressing these issues.

Ongoing monitoring of a range of local level targets will be underway. Although precise timescales are not yet available, it will be important that the synthesis report considers how local achievements can best be documented. Consideration should also be given to the inclusion of evaluation evidence from supporting work programmes that are not under the specific responsibility of CtOG Target leads.

By April of each year sufficient Target monitoring information will be available to produce short thematic digests, documenting progress towards each CtOG Target and the CtOG overarching Aims, and providing descriptive analysis of the work programme being undertaken and processes by which Target work programmes are operating.

« Previous | Contents | Next »