The Scottish Government

« Previous | Contents | Next »

Listen

EXECUTIVE SUMMARY

Chapter One Introduction: Aims and Objectives of the Options Appraisal

1. The Scottish Government Schools Directorate and Lifelong Learning Directorate commissioned the Centre for Educational Sociology and BMRB to carry out an appraisal of the options for meeting the needs for longitudinal data on young people's experiences in secondary school and subsequent transitions to further/higher education, training and employment.

2. The Scottish Government has funded the Scottish School Leavers Survey ( SSLS) since the mid 1970s. The SSLS is a longitudinal study of nationally representative samples of young people in Scotland. The SSLS has given Scotland an enviable resource for the longitudinal study of young people's transitions, but young people's transitions, survey methodology and the policy environment have all changed since its basic design was developed. In recent years the SSLS has also suffered from declining response rates with consequent issues of bias and small numbers. The most recent contract for the SSLS ended in 2007 and it was seen as timely to review how the evidence needs on young people's transitions could best be met in the future.

3. The options appraisal was therefore commissioned to undertake a critical examination of a range of options on how to meet the needs for longitudinal data on young people and make recommendations on any future longitudinal study. Early in the project it was agreed with the Scottish Government that the project would focus on developing detailed options.

4. This options appraisal has included: consultation with external stakeholders and the Scottish Government; investigation of administrative data; investigation of longitudinal studies; a review of the latest survey design literature, and a focused review of the youth transitions literature. Based on these activities, options for longitudinal data collection, analysis and reporting study were developed, assessed and costed.

Chapter Two Background: Longitudinal Studies, Young People's Transitions and the Policy Context

5. Longitudinal studies are concerned with the behaviour of individuals over time, and offer a way to analyse change and dynamic behaviour. Repeated contacts enable analysis of individual's transitions, including their movement in and out of education, employment and unemployment, and capture the sequential ordering of events and influences in their lives.

6. Longitudinal studies are important for policy analysis because they document change over time, and also because they enable the influence of policies and practice to be isolated from confounding influences such as social background and context.

7. While a longitudinal study offers the best approach to understanding young people's transitions, it poses several challenges, in particular, the potential for the attrition of participants over the course of the study. Other issues include cost and the need for a long-term commitment to funding.

8. Young people's transitions are becoming more protracted, more diverse and more complex in a context where pathways in education and in the labour market are becoming more flexible. Initial labour market destination is no longer an adequate indicator of their longer term position in the labour market, and labour market careers must be followed for a number of years before stable 'outcomes' of different educational pathways can be observed. In addition, understanding young people's pathways requires longitudinal data on 'soft skills', their goals and motivations, and on their perceptions of the available educational and labour market options.

9. There is increasing polarisation between those who remain in education and gain qualifications and those who leave school as soon as they can. Poorly qualified young people run a high risk of marginalisation in the labour market and in society. The concept of young people not in education, employment or training has gained prominence over the last two decades as a key indicator of an unsuccessful post-school transition. The antecedents of this are varied but include young people's earlier experiences of education and previous transitions such as the move from primary to secondary school.

10. Social origin continues to influence young people's transitions after they have entered the labour market. The transition from education to work and adult life is a critical phase when inequalities may be challenged or reinforced so that a longitudinal study needs to be able to monitor inequalities in both the transition process and in transition outcomes.

11. Scottish policy aims to create more flexible and individualised provision within an 'integrated landscape of learning' that will enable all young people to access a wider range of learning opportunities, to become lifelong learners, to develop their skills and, by doing so, help to raise the skills base in Scotland and contribute to economic success at an individual and societal level. These themes underpin key Scottish Government policy initiatives and provide the policy context for a longitudinal study of young people's transitions.

Chapter Three The Experience of Longitudinal Studies Internationally

12. There are a number of longitudinal studies around the world that focus on the pathways young people take as they complete compulsory education. The options appraisal examined their key design elements and how successful they have been in retaining study members.

13. Relevant features identified from these studies were:

• high initial response rate by making a first contact within the school setting;
• mixed-methods of administration provide better response rates than postal questionnaires;
• the importance of regular contacts to maintain a study identity;
• the benefits of surveying parents, and gaining active parental support;
• the use of incentives to increase response rates;
• the use of endorsements of the aims of the study by respected organisations in order to gain support of schools and raise the profile of the study.

Chapter Four Consultation on a Longitudinal Study of Youth Transitions

14. External stakeholders were unanimous in their view that a longitudinal study of young people's transitions in Scotland is necessary. They supported a study that would track individuals' routes through different activities and statuses and enable a better understanding of young people's pathways and outcomes. Overall, Scottish Government staff supported a longitudinal study although a minority wondered if their information needs could be met from other data sources or from a different approach.

15. Policy and analytical staff at the Scottish Government focused on the need to provide the Government with the capacity to evaluate the impact of its policies and interventions.

16. There was a demand for more data on young people's wider experiences at school (including in relation to the four capacities set out in a Curriculum for Excellence), on their attitudes, aspirations and motivations, the guidance and support they had received at school and beyond, as well as data on wider aspects of their lives including participation in leisure, cultural and volunteering activities, on health and on housing and living arrangements.

17. Better data on government-supported training and other training was identified as a need as was more data on finance and funding, especially the impact of different funding policies on young people's attitudes to, and participation in education and training. There was interest in tracking the destinations and outcomes of young people who followed different post-school learning routes.

18. Certain groups of young people were identified as being of particular policy interest including those who are not in education, training or employment, or at risk of being so, and sub-groups within this category. The recruitment of sufficient numbers of young people in disadvantaged or vulnerable groups and their retention during the study were seen as crucial aspects that a new study must deliver.

19. Common themes expressed by stakeholders and Scottish Government staff were:

• Scotland needs a longitudinal study to follow young people's transitions through an increasingly diverse education and training system and a more complex economic and social world.
• As youth transitions become more complicated, prolonged and differentiated, a longitudinal study is necessary to enable understanding of these transitions.
• Respondents identified the lack of a continuous picture of individuals' movement through the system and a longitudinal study was seen as necessary to identify and track individuals' transition pathways, including alternative and cross-sectoral routes.
• It is essential that a longitudinal study includes all young people, including those in the independent sector. Without this comprehensive coverage it would be impossible to compare the outcomes of different groups of young people and to assess whether inequalities are declining or not, or to compare the alternative routes that are increasingly available to young people.
• Most thought that a new study should contact young people earlier, when they are still in compulsory education, to track and assess the effect of the reforms of compulsory schooling and the growing curricular flexibility and differentiation within this stage on young people's later experiences and outcomes.
• There was substantial support for a survey sweep at around the age of 27, 28 or 29, when graduates would be more likely to have started to establish themselves in a career, and to enable the study to pick up on those returning to education.

20. There is a need for better and more active dissemination of a new longitudinal survey than is currently the case in respect of the SSLS. Concise briefings covering the findings and highlighting their policy relevance were suggested rather than longer descriptive reports of each sweep. A new name for the study would be required.

21. There was limited awareness of SSLS including of the research that has used SSLS data. Respondents suggested that a study website would be helpful including the facility to carry out analyses. There was also support for the idea of staff (whether internal or external to the Executive) with a remit to undertake analyses as required by policy teams.

22. There was a strong feeling that for a new study to be successful new management arrangements are necessary, a central element of which would be the existence of specific individuals or groups with a remit to develop and promote it.

Chapter Five Options for the Design of a Longitudinal Study

23. The options appraisal examines a number of design issues relevant to a longitudinal study and considered their advantages, disadvantages and implications.

24. The key design issues considered are:

• The age range to be covered.
• Timing and frequency of contact(s) in compulsory education.
• The timing and frequency of contacts after S4.
• Frequency of the recruitment of new cohorts.
• Interim measures.
• Survey methodology, including the potential for carrying out the first sweep in school.
• The design of the sample, including whether it should be random, clustered and/or boosted.
• The sampling frame.
• The extent to which information can be derived and linked from existing administrative sources including legal issues.

Chapter Six Design and Guideline Costs

25. We recommend consideration of two main design options:

• Design A: first survey young people while they are still in compulsory education with subsequent surveys at key points up to their mid or late 20s. 1st contact to be carried out in a sample of schools. Use of mixed methods in post-16 survey sweeps. Measures to maintain contact with sample members between sweeps.
• Design B: first survey young people in the year after compulsory education as in SSLS but change the data collection strategy to mixed methods and boost sample numbers in certain sub-groups of young people. Subsequent surveys at key points up to their mid or late 20s, with interim measures to maintain contact with sample members.

26. We recommend the use of administrative data as an integral part of both designs to: provide background details for selecting a cohort sample; track sample members at later stages of the study to alleviate problems of non-response; and to link information from administrative sources to survey data.

27. Ballpark costings are provided for these options, based on an achieved sample of 10,000 at sweep 1.

28. Design A provides the optimum design for a longitudinal study of young people's transitions to meet the needs expressed by policy and analytical staff, and stakeholders.

29. If Design A is commenced immediately, by 2008 it can provide valuable information on early secondary school experiences. However, under Design A, data on 16/17 and 18/19 transitions will not become available until at least 2010 and 2012. To secure data on these transitions in the short term we recommend that Design B be conducted as an interim measure in parallel with Design A.

Chapter Seven Sample Maintenance and Attrition

30. Problems associated with non-response were a major issue for the former SSLS, and the resolution of these problems is a major aim for the proposed longitudinal study of young people.

31. The proposed design for the new study has a gap of two years between the initial interview conducted in school and the first follow-up interview conducted in the respondent's home. Contact must be maintained with respondents during this time so that they will not forget about the survey. The young people will be becoming more mobile at around this time and maintaining regular contact will increase the likelihood of tracing those who move. Accordingly, we recommend that a number of steps are taken to maintain contact with participants and build a study identity between sweeps, including the creation of a study website.

Chapter Eight Dissemination, Use and Management Issues

32. A longitudinal study represents a considerable financial investment and attention needs to be given to maximising the visibility and profile of the study, to the reporting and dissemination of its findings and to facilitating access to the data for further analysis.

33. We recommend a study website to publicise the study to (potential) users and stimulate their engagement with it. It should contain information about the survey, research findings and possibly data to access.

34. An active dissemination strategy, with staff to implement it, is essential for a successful study. The dissemination strategy should include the publication of findings in formats that would engage the interest of stakeholders, for example, summary briefings and short issues papers.

35. A strategy for data analysis should be developed that would specify what is required in terms of basic reporting at each sweep but that would also set out the further longitudinal analysis of issues of interest to policy makers and stakeholders that might be undertaken.

36. A range of users should be encouraged to use the longitudinal study data for secondary analysis. This requires attention to deriving variables, including contextual variables, anonymising records, and comprehensive documentation. Another aspect is the provision of staff to respond to requests for information from those who are not able to analyse the data themselves.

37. Workshops and other measures to support the analysis of data on young people's transitions could be developed in conjunction with other measures to stimulate quantitative data analysis such as the Economic and Social Research Council ( ESRC) Researcher Development Initiative.

38. Securing the endorsement of national bodies, local authorities, professional groups and voluntary organisations is a way of ensuring that the study is viewed as high profile, relevant and useful. Such awareness and endorsement needs to be actively pursued.

39. Experience shows that the success of a longitudinal study is dependent on there being a committed group or organisation that will promote it, 'nurture' it and develop its potential. The ESRC is currently developing a model for the governance of longitudinal data based on the concepts of 'ownership' by the funders of a study and 'stewardship' by some appropriate organisation. This could be a helpful starting point in considering the most appropriate management and governance arrangements for a new longitudinal study.

40. The ESRC could be approached for some element of co-funding of the proposed longitudinal study. This would be in keeping with the ESRC's National Data Strategy which recognises the need for collaboration with other bodies, including the devolved administrations, in developing a strategic approach to data resources for the social sciences.

Chapter Nine Conclusions and Recommendations

41. The consultation process found strong support for a longitudinal study. Other approaches such as cross-sectional surveys and administrative data would not meet the data needs in respect of young people's transitions. Only longitudinal data enable understanding of transitions, provide a continuous picture of individuals' movement through the system ('learning journeys') required for policy development, and enable an assessment of the cumulative impact of government initiatives on young people's outcomes.

42. We make the following recommendations:

• The Scottish Government should continue to fund a longitudinal study and it should consider exploring the possibility of some element of co-funding with the ESRC.
• Design A which first surveys young people when they are still in compulsory education best meets the need for longitudinal data on their transitions; it is our preferred option.
• Within Design A, the number and timing of contacts within the compulsory stage would need further consideration in the light of available resources and developments in a Curriculum for Excellence and other related initiatives.
• We recommend that there should be at least four further survey sweeps after S4: at ages 16/17, 18/19 with subsequent sweeps possibly at 22/23 and 26/27. The timing of these later two sweeps would be best decided in relation to relevant policy developments nearer the time of these sweeps.
• If the Scottish Government decides to proceed with Design A, it should seriously consider carrying out an interim study to fill the gap in data between that collected by SSLS and the data that will be collected under Design A. The design and methodology of this interim study could follow Design B (first survey young people in the year after compulsory education using a mixed methods approach with a boosted sample).
• New cohorts should be recruited on a 4-yearly basis.
• A longitudinal study should aim to include different groups of young people and should cover both publicly-funded and independent schools.
• The potential of administrative data to enhance the survey should be utilised. In this regard, measures to support and take forward the work of the Managing Information Across Partners ( MIAP) project would be extremely helpful.
• To make the most of its investment in a longitudinal study, the Government should develop a strategy to maximise its profile, disseminate the findings widely and encourage others to use the data. This strategy should include a number of elements.
• The Government should review the management and governance arrangements for a longitudinal study. A key consideration should be to ensure that arrangements support the active promotion and development of the study and provide continuity and commitment in its management.
• The Government should ensure a strategic approach to the longitudinal research it commissions, for example, through developing the work of its Longitudinal Research and Analysis Network.

« Previous | Contents | Next »