The Scottish Government

Technical Note:

Follow Up Survey Of Leavers From Scottish Schools: Leavers From School Year 2007/08

Background

Information on the latest known destination of school leavers in September each year is supplied to the Scottish Government by Skills Development Scotland (SDS). This is referred to as the school leaver's 'initial destination' roughly 3 months after they left school (although some may have been Christmas leavers thus representing a longer follow up period). In 2006/07 this data was supplied at individual pupil level for the first time, and since then it has been possible to match it to information from the Pupil Census for the relevant year. This means that we were able to provide much more detailed analysis of the characteristics influencing different destinations. For example, pupils from more deprived areas (based on the Scottish Index of Multiple Deprivation 2006) are less likely to go on to Higher and Further Education than those from less deprived areas.

During March and April 2009, Careers Scotland contacted the 2007/08 cohort of leavers from publicly funded schools to update their destination from the initial survey. The results will be used to monitor the national indicator on increasing the proportion of school leavers in sustained positive destinations. This paper sets out the process through which the final estimate for the national indicator was derived.

Data matching

In 2006/07, matches were based on using the Scottish Candidate Number (SCN) first, and if that was missing, combinations of school code, date of birth, postcode and gender, since these four variables were common to both datasets. Full details of the methodology can be found in 'Methodological changes to the 2006/07 Destinations of School Leavers Survey' ( http://www.scotland.gov.uk/Publications/2007/12/07093501/26).

For 2007/08, SDS also supplied the Student ID (where known), which is not unique across schools. The methodology used in 2006/07 was applied to the 2007/08 data for the initial destinations publication ('Destinations of Leavers from Scottish Schools: 2007/08' http://www.scotland.gov.uk/Publications/2008/12/08090751/0). Prior to analysing follow up destinations, we revisited the matching process for 2007/08 data as part of a continuous programme of data improvement.

One consideration was that the original matching had excluded status 04 pupils (those on the roll of a school but attending a course of Further Education elsewhere) from the exercise, and SDS had since confirmed to us that their data included such pupils. Furthermore, the follow up survey of leaver destinations which is the true focus of National Indicator 10 uses pupil characteristics in the process of imputing destinations for leavers who could not be contacted. This consideration lent weight to the need to improve the matching of leavers to pupil census records.

For the rematching, status 04 pupils were included and tighter constraints on what was considered a good match were applied. Rematching was performed on the follow up data provided by SDS at the beginning of June 2009.

The following table shows the results of the matching.

• The total number shown (58,844) differs from that published in December 2008 (58,791). The reason for this is that leavers are only included in the published figures if they leave from a publicly funded secondary school, so changing the match may result in changing the school concerned.
• For pupils where no match was found on the census (1,266), the data reported by SDS on school, gender and postcode was accepted and no attempt was made to assign additional characteristics. A non-match may result from a lack of information from SDS (not all fields were supplied for all leavers) or because the leavers were not on that year's pupil census (pupils may be omitted if they are not in the school on census day, for instance if they had left school but subsequently returned).

• Of the 58,844 leavers:

o 56,657 are status 01 (on roll and based in school)

o 921 are status 04 (on roll but attending full time Further Education elsewhere

o 1,266 are unknown (no match on pupil census, due to limited information or absence of the leaver from that year's census as noted above)

• The percentage in positive destinations compared to the figure published in December 2008 decreases slightly from 86.5 to 86.4 This would have made no difference to the arrow direction for National Indicator 10 - unchanged.
• The percentages in the destinations categories (HE, FE etc) does not change when taken to one decimal point.
• Total numbers and percentages in destinations categories have changed slightly by local authority. This is a result of matching to different schools in the rematched dataset compared to the original dataset.

Rematching was also carried out on 2006/07 data to test comparability. Again, the number of leavers in publicly funded secondary schools that would have been reported changed slightly, from 57,364 to 57,387. Headline figures for positive initial destinations would remain unchanged at 86.6%. The 2006/07 figures reported in 'Follow Up Survey Of Leavers From Scottish Schools: Leavers From School Year 2007/08' ( http://www.scotland.gov.uk/stats/bulletins/0000738) are based on the original published totals, while the 2007/08 figures are based on the rematched data.

We are confident that this rematched data is an improvement on the original. The matching methodology will be kept under review, and the possibility of using specialised data linking software is being explored.

National Indicator Estimate

Creating the dataset

The following table outlines the steps taken to arrive at the final dataset for deriving the national indicator estimate. Firstly, the matched data gives the number of pupils in valid schools - that is, in publicly funded and grant-maintained secondaries. Leavers whose original destinations was Moved outwith Scotland were excluded, as problems with following up such leavers would be anticipated.

In the follow up survey, those leavers whose destination was recorded as school pupil have been excluded from the results. The surveys are intended to focus on young people of school leaving age who left school during or at the end of the school year (which is taken to run from 1 August to 31 July). In practice many young people do not decide whether or not to leave school until after they have received the results of external examinations, have tried to get a job or achieve entry to further or higher education or training. For many pupils, therefore, it is only possible to identify at a later date whether they have effectively left school by 31 July. Where the follow up has established that pupils originally thought to be school leavers are now back in school, such pupils are excluded from the analysis. Other leavers have been excluded from the original data because they are deceased or have moved outwith Scotland prior to the follow up survey.

Imputation for non-follow up

In the initial survey SDS was able to establish the destination of 98.9% of school leavers, and those results were published with the remaining 1.1% stated as 'Unknown'. For the follow up survey, destinations could be established for 94.5% of eligible cases, an improvement on the 91.1% sucessfully followed up in 2006/07. However, the higher rate of non-response compared to the original destinations survey suggested that follow up destinations should be imputed if analysis suggested that certain groups would be under-represented by only considering cases where a follow up destination is available.

The cases without a valid follow up destination comprised those not followed up: 2,913 leavers with a known initial destination and and unknown sustained destination, and 337 with both intitial and follow up destinations being unknown. These latter cases are treated as unknowns for sustained destinations.

Analysis of the follow up survey showed that several variables might be related to the likelihood of a successful follow up. The variables considered for use in the imputation were i) match type - i.e. whether a match to the pupil cenus was possible or not; ii) original detailed destination data; iii) deprivation as measured by the Scottish Index of Multiple Deprivation 2006 (SIMD); and iv) stage of leaving school.

Several other variables were examined, and either did not appear to significantly affect the likelihood of being followed up (e.g. gender) or were not regarded as being suitably robust indicators, due to small numbers (e.g. ethnicity) or doubts about the suitability of the data(e.g. looked after status). Free school meal registration, despite being used in the imputation exercise in the previous year, was excluded from consideration due to concerns about the quality of the data available in the relevant year's census, 'Pupils in Scotland 2007' ( http://www.scotland.gov.uk/Publications/2008/07/28100032/0) (see background note 5.9 in the publication). Although the SIMD variable did not appear to be as strongly related to the likelihood of being sucessfully followed up as the other chosen variables, it was felt that having another socioeconomic indicator in place of free school meals was desirable.

These variables were chosen after examing the percentages and numbers followed up by classes of the variable, followed up by chi-square tests of association. Logistic regression was then used to assess models including the chosen variables. At this point the variable related to stage of leaving school was dropped, as its influence was subsumed into the effects of original destination and SIMD indicators once all were included in the model.The following tables shows the rate of follow up for each initial (detailed) destination, SIMD category and match type.

Table 3: follow up rates

The final set of characteristics was thus:

• Initial destination

• Match type - those not matched to a record in the pupils census as opposed to those matched
• SIMD ranking recoded to 20% most deprived datazones and other 80% datazones

As the percentage of known destinations was very high, it was decided that a simple imputation method would provide a sufficient adjustment to improve the representativeness of the results.

Where the initial destination was known, respondents and non-respondents were split into groups by destination, and then again by whether or not matched to the census and by SIMD indicator. This made it possible to determine the distribution of follow up destinations of respondents within each group and randomly allocate follow-up destinations to non-respondents with the same characteristics in the same proportions.

Effects of imputation

The effects of this imputation are shown in the following tables which provide the final published data together with the comparable tables if imputation had not been carried out and only the known follow up destinations had been analysed.

Table 1 (as published - with imputation)

Table 1 (known destinations only)

Table 2 (as published - with imputation)

Table 2 (known destinations only)

Table 3 (as published - with imputation)

Table 3 (known follow up destinations only)

This comparison indicates that the imputation of follow up destinations has had a relatively small effect on the results of the survey. It can be seen from Table 2 that the imputation has produced a small increase in the destination categories which the analysis of follow up rates suggested may have been under-represented in the data, i.e. the various employed and unemployed categories.

Tables of the distributions of follow up destinations by intital destinations for imputed cases and non-imputed cases are given below.

We are confident that by adjusting the data to be more representative of the overall cohort, this imputation has improved the quality of our estimate of the proportion of school leavers in positive and sustained destinations. If you have any comments or questions about the imputation method used in this survey, please contact school.stats@scotland.gsi.gov.uk.