« Previous | Contents | Next »
Listen
CHAPTER 8 DISSEMINATION, USE AND MANAGEMENT ISSUES
8.1 A longitudinal study represents a considerable financial investment and if the potential benefits of such an investment are to be realised, attention needs to be given to maximising the visibility and profile of the study, to the reporting and dissemination of its findings and to facilitating access to the data for further analysis. As we have reported, stakeholders felt strongly that much more attention and resource needs to be given to these aspects than has been the case with the SSLS, especially in more recent years. We suggest that a strategy and mechanisms to support the dissemination and use of the data should form an integral part of the contract for any new longitudinal study.
A strategy for analysis and reporting
8.2 In the past, the contract for SSLS has focused, to a large extent, on the data collection aspects with less attention paid to the analysis and reporting. The SSLS contract has included funding for descriptive analysis and reporting of findings at each sweep, mainly on a cross-sectional basis with little longitudinal analysis. While a few in-depth studies on particular topics have been funded by the Scottish Government, this has been on an ad hoc basis 20. We suggest that in a new study, a strategy for data analysis should be developed that would specify what is required in terms of basic reporting at each sweep but that would also set out the further longitudinal analysis of issues of interest to policy makers and stakeholders that might be undertaken. There needs to be a more structured process to identify and fund further analysis of such issues. The Government should consult internally with policy and analytical staff as well as externally with other policy makers and researchers on a regular basis about the key issues to be examined in more depth after each sweep and establish a programme of studies for the next few years. The Executive has now established a Longitudinal Research and Analysis Network ( LRAN), part of whose remit is to champion, promote and stimulate the exploitation of longitudinal datasets both within and outwith the Government, particularly in relation to their contribution to policy evaluation. Strategic direction of the longitudinal study, and its analysis, may be an appropriate role for this group.
8.3 The consultation process revealed a considerable lack of awareness both within and outside the Government about the SSLS, the nature and range of its data, the various analyses that have been carried out and the resultant publications. We have already recommended the creation of a study website as part of the strategy to retain young people's participation; we also regard a study website as fundamental to publicising the study to users and potential users and stimulating their engagement with it. It is now common practice for major surveys to have a dedicated website containing information about the survey, research findings and in some cases, data to access; users and potential users now expect such a resource to be available.
8.4 For a new longitudinal study to be successful, an active dissemination strategy would be required with staff to implement it. The Growing Up in Scotland study, for example, has a Dissemination Officer. The dissemination strategy could include the publication of findings in formats that would engage the interest of stakeholders, for example, summary briefings and short issues papers. These types of publications would facilitate the dissemination of findings to policy makers, practitioners and the media via, for example, the study website, press briefings, and perhaps a regular seminar series.
8.5 We have discussed the need to ensure that young people 'buy into' the study and this also applies to the range of organisations and groups with an interest in young people's transitions. As we have already suggested, securing the endorsement of national bodies, local authorities, professional groups and voluntary organisations would be an important element in ensuring that the study is viewed as high profile, relevant and useful. Such awareness and endorsement needs to be actively pursued.
Encouraging use of the data
8.6 Another way to capitalise on the investment in a longitudinal study is to encourage and enable others to use the data for analysis. A number of the large surveys in the UK have data available on their websites as well as depositing data with the UK Data Archive (eg LYPSE, the British Cohort Study, and the Scottish Longitudinal Study). LSYPE, for example, as an integral aspect of the study has a wide ranging data enhancement programme with the specific aim of maximising the availability and usability its data. Although SSLS data is deposited in the UK Data Archive, it was evident from the consultation that awareness of its availability for secondary analysis was limited while the limitations of the documentation of the large and complex SSLS datasets do not facilitate their use.
8.7 Clearly mechanisms have to be in place to protect respondents' anonymity and uphold confidentiality before the data can be accessed by a range of users. There are various ways of doing so such as anonymising individual level data, aggregating any other potentially disclosive data and having a system for regulating access to the data eg licensing arrangements. We would not envisage that there would be problems in this respect in relation to the data from a new study. As we have noted, data from the SSLS are already available for secondary analysis.
8.8 Administrative data form an important component of the proposed study providing detailed information on the outcomes of education and training that is difficult to obtain in short questionnaires or interview schedules. For the most part administrative data will be linked using the Scottish Candidate Number ( SCN) which will be removed from public datasets. Care is needed to ensure that a full range of variables are derived from administrative data and linked to the survey data before any process of anonymising takes place, so that the potential richness of these data can be utilised.
8.9 Further issues regarding anonymity relate to schools. Preferred practice is to replace the establishment codes with anonymous identifiers, so that analysts can control for the design effects of school membership, and analyse school differences. However, a number of school contextual variables need to be derived by the survey organisation before school identity is removed. Contextual variables include school denomination, school roll size, free-meal entitlement, local area deprivation, and characteristics of the travel-to-work area. These variables are very important for research, and impossible to derive once the school identity has been removed.
8.10 If the data collected by a new study is to be analysed extensively (and therefore be cost effective), it is essential to publicise its availability and to develop a strategy to support those accessing it. Critical factors include the provision of comprehensive and understandable documentation, the availability of key derived variables, and files that are easy to navigate (Martin et al 2006). As well as paper documentation, there are now additional possibilities, for example, the US National Longitudinal Study of Youth ( NLSY) includes a CD- ROM which provides interactive access to both documentation and data.
8.11 Training workshops that provide a hands-on introduction to accessing and using the data might also be considered to encourage the use of the data. The capacity in the research and policy communities in Scotland for quantitative analysis in general and longitudinal analysis in particular is an issue of concern to the Government, the Scottish Funding Council and the ESRC. Workshops and other measures to support the analysis of data on young people's transitions could be developed in conjunction with other measures to stimulate quantitative data analysis in Scotland (and elsewhere in the UK) such as the ESRC Researcher Development Initiative.
8.12 In Scottish education there is considerable interest in evidence-informed practice. For example, the Applied Educational Research Scheme has attracted teachers, head teachers and local authority staff to undertake collaborative research with university researchers. Potentially the longitudinal study provides a valuable resource for practitioner - research. For example, a subject teacher may be interested to analyse the career trajectories of young people specialising in her/his subject and the factors such as subject combinations and aspirations that affect their outcomes. Or staff from a local authority may be interested in the factors that influence the perceptions and aspirations of young people in their area. Providing a resource that is seen to be useful to Scottish educators can provide a counter-weight to the heavy demands made on their time by research studies.
8.13 An important aspect of a strategy to encourage the full utilisation of a new study is the provision of staff to respond to requests for data from those who are not able to analyse the data themselves. This was raised in the consultation process and there was some suggestion that organisations might be willing to contribute to the funding of such a post.
The management and governance of a new study
8.14 Underlying our discussion of the dissemination and use of a longitudinal study is the question of who would be responsible for developing and carrying out the strategy and activities suggested. Experience shows that the success of a longitudinal study is dependent on there being a committed group or organisation that will promote it, 'nurture' it and develop its potential (Martin et al 2006). A clear message from the consultation process is that this has been lacking in respect of SSLS since the early 1990s.
8.15 Responsibility for SSLS within the Government has moved between the former Departments of Education and of Enterprise and Lifelong Learning over the years; since 2003 the remit for the project management of the survey has lain with Education Research. It has formed one element of a number of responsibilities of the staff concerned and the role of the Scottish Government has essentially been to manage the contract rather than to promote and develop the survey. Since the early 1990s, the Government has put the conduct of the survey out to tender. The scope of the contract has focused on data collection, with most resource directed to collecting the data with a limited amount of funding in the contract for descriptive analysis and providing an overview report of findings at each sweep. The SSLS has an Advisory Group made up of academics, policy makers and practitioners but the Advisory group as constituted has not been able to play an active role in supporting and developing the survey.
8.16 On the basis of the experience of other longitudinal studies and the consultation process we suggest that different arrangements would be required for a new longitudinal study, a central aim of which would be to ensure that there would be specific individuals or groups who would have a central responsibility (and the time) for its active development, promotion and use.
8.17 Several stakeholders felt that the Scottish Government does not have the critical mass of staff to support and develop a longitudinal quantitative study and that individuals or groups outwith the Executive would need to play a central role. Another factor to take into account is the relatively rapid turnover of staff in the Government - but the effective management and development of a longitudinal study, by its very nature, requires continuity and stability in staffing. One approach might be for the Government to contract with an appropriate organisation (or possibly a consortia) that would then have the responsibility for the direction of the study, its promotion and development; such an organisation or group could be overseen by an advisory or steering committee. The ESRC is currently developing a model for the governance of longitudinal data that could be a helpful starting point in considering the most appropriate management and governance arrangements for a new longitudinal study. This model is based on the concepts of 'ownership' by the funders of a study and 'stewardship' by some appropriate organisation with various arrangements in place to safeguard the rights of the funders and to support the longitudinal study.
8.18 The consultation identified the issue of the place of a study of youth transitions alongside the other research that the Government funds, both in relation to surveys of school pupils and its other longitudinal studies. In this respect the creation of the Longitudinal Research and Analysis Network ( LRAN) which aims to support the strategic development of large scale, cross-cutting longitudinal surveys across the Government, including co-ordinating investment and identifying and addressing gaps, is to be welcomed.
Co-funding
8.19 A new longitudinal study would not only provide data to meet the policy needs of the Scottish Government but would also provide a data resource for the research and academic community as well as other public and voluntary bodies. As a general resource of this nature it would be appropriate to consider approaching the ESRC for some element of co-funding. This would be in keeping with the ESRC's National Data Strategy. The ESRC recognises the need for collaboration with other bodies, including the devolved administrations, in developing a strategic approach to data resources for the social sciences. One of the identified roles of its UK Data Forum is to consider shared responsibilities where the development of a particular data resource is 'beneficial to a range of users' - governmental and non-governmental, academic and non academic (p. 17, ESRC 2007). A new longitudinal study would fall into this category of being beneficial to a range of users.
8.20 The possibility of co-funding from the ESRC for a longitudinal study of young people's transitions is also suggested by the need that the Council has identified to fill the gap in UK longitudinal data on children born in the early 1990s. The earlier birth cohorts followed children born in 1946, 1958 and 1970 but there is then a gap of thirty years until the Millennium Cohort study of children born in 2000. In considering how to bridge this gap, the Strategic Review of Cohort and Panel Studies, recommended an investigation of the feasibility of using the SSLS to do so for Scotland (p 57, Martin et al 2006). A new longitudinal study which first contacted young people aged around 14 in 2008 (Design A) ie those born in 1994 would meet the need the ESRC has identified for data on the birth cohort from the early 1990s.
« Previous | Contents | Next »