The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 5: CONCLUSIONS AND RECOMMENDATIONS

5.1 Through the rich array of evidence collective across the spectrum of government and stakeholder research interests and research resources, the study has revealed the value of, and pressing need, for enhanced longitudinal research resource development and use in Scotland. The parameters for such expansion are set by the new national performance framework ( chapter 1). The agreements around this framework offer opportunities to meet more effectively Scottish Government policy needs while lending the framework itself to evaluation through this expanded research resource.

5.2 The conclusions set out below draw together judgments drawn from the different sources of evidence examined, including the views of senior Government officials and other experts that we talked to. The documentary sources comprised the descriptions of the 29 longitudinal studies and administrative databases supplied by the teams responsible for them and examples of their use. These were summarised in relation to National Outcomes to which they bear most direct relevance ( Chapter 2) and formed the basis of the vignettes, which set out different scenarios for their use illustrated, where appropriate, by research exemplars ( Chapter 3). The vignettes and the summaries constitute a useful resource for identifying the datasets of most relevance to particular outcomes and the forms of analysis likely to be most useful, i.e. the report as whole can be viewed as supporting a clearing house for a series of research agendas. Issues that need to be resolved to get most value from the investment in longitudinal research resources. These come together in the proposed national strategy for longitudinal resource development and capacity building ( Chapter 4).

(a) Investment

5.3 Table 5.1 provides an overall summary linking studies/resources to outcomes and identification of gaps.

Table 5.1 Research relevance and use

5.4 Table 5.1 points to five kinds of longitudinal data resources of particular relevance to the evaluation of most of the National Outcomes grouped with the exception of reducing inequalities (applying to all of them) and developing national identity (not covered by any of them) in terms of 6 policy domains and merit sustained investment:

(a) the Scottish national studies, GUS and SSLS (replacement) and the SLS - education and life long learning, health, environment, inequalities
(b) the major Scottish area based studies, West of Scotland, Edinburgh, Go Well Glasgow, health, community and public services, environment, protection/safety, inequalities the
(c) Household Panel and Birth cohort Studies, BHPS/ UKHLS, BCS, NCDS - economy, education and Life long learning, health
(d) (potentially) the Scottish ELSA (the need for which is currently being scoped), health, economy, inequalities community and public services
(e) administrative data sources such as IDBR, WPLS, ASHE -(economy); SSA, careers guidance (education and life long learning); SCORE, (environment, community and public services)

5.5 Gaps are evident that need to be filled if all 15 National Outcomes are to be supported by longitudinal research

(a) longitudinal survey of businesses and employees building on the IDBR and linked to WPLS (b) longitudinal study of rural community life and migration
(c) (Scottish) modules on identity, rurality and crime ( UKHLS)

5.6 Give high priority to improved linkage between and access to administrative datasets in a national database and develop facilities for linking this dataset to the key longitudinal studies at national and local level.

(b) Strategy

5.7 Consciousness needs raising within all policy divisions of the value of longitudinal research for policy development and evaluation.

5.8 Develop a comprehensive evaluation strategy that embraces performance at all levels from national to local service delivery utilising developments at all levels to support continual review and updating of the whole structure.

5.9 As all National Outcomes can be linked directly or indirectly to all the surveys reviewed, there needs to be cross-government agreement about strategy for longitudinal resource development to share the costs in a coordinated plan.

5.10 At local area level base longitudinal research strategy on a few studies designed in terms of a multi-level framework to avoid dissipating resources over a large number of disparate studies.

5.11 Support UK national studies with a Scottish boost component principally for the testing of systemic and policy differences between Scotland and other countries in the UK, and general descriptive statistics. Use local longitudinal studies to understand the links between policy inputs, processes of delivery and achievement of outcomes.

5.12 Within Scotland build as much complementarity between national and local measurement as possible through the development of multi level design frameworks embracing, below national level, local authorities, school catchment areas and neighbourhoods, to ensure that institutional and contextual effects can be identified in the modelling of the delivery system.

5.13 Exploit to the full the research opportunities offered by the concordat around the development of Single Outcome Agreements through the production of accompanying longitudinal resources at national and local level and collaborative research agreements.

5.14 Develop further the operationalisation of the National Outcomes to make evaluation more rigorous and unambiguous. Thus certain major policy areas like housing need to be built more specifically into the outcomes as a basis for identifying a set of indicators that are most appropriate to the outcome.

(c) Capacity

5.15 Develop the capacity for local research and evaluation, including the recognition of the value of disaggregated national datasets and administrative data, for linkage to local indicators.

5.16 Strengthen links between academic research departments with longitudinal research capability and local Government as in the Glasgow based Go Well study.

5.17 Develop the capacity of the system for "natural experiments" by, wherever possible, staggering the introduction of new policy interventions in different places, to supply the basis for "waiting room" controls, in which over time everybody receives the intervention.

5.18 Ensure that baselines are in place nationally and locally via surveys and geographical and administrative data to ensure that all new policy developments and interventions start with prior knowledge of the outcome indicators.

5.19 Develop evaluation capability both within Scotland, but also contribute to its enhancement across the UK and internationally, by seeking collaborative arrangements to develop common cross-national datasets around themes such as ageing, child development and community development.

5.20 Invest in capacity building:

• in Government to develop and use longitudinal resources in the form of surveys and administrative data to support evaluation in all policy domains in the most effective way and ensure their accessibility to the research community as a whole.
• in the universities through linked studentships, i.e. through a Scottish-wide PhD programme in the social, developmental and economic sciences, for which the national PhD programme in Economics is the exemplar, linking these to the recently announced HEFCE support for one year MSc courses.
• in the research community more widely through the Hub-Node infrastructure of the National Centre for Research Methods by following the Welsh Assembly example of investing in a Scottish 'Node'

« Previous | Contents | Next »