The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 6 DESIGN AND GUIDELINE COSTS

6.1 In the light of the options discussed in the previous chapter, we recommend two main designs for a new longitudinal study:

• Design A: first survey young people while they are still in compulsory education with subsequent surveys at key points up to their mid or late 20s. Interim measures to maintain the continuity of data collection need to be considered.
• Design B: first survey young people in the year after compulsory education as in SSLS but change the data collection strategy to mixed methods and boost sample numbers in certain sub-groups of young people.

Design A

6.2 The key features of Design A include:

• young people surveyed while in compulsory education (one or two sweeps);
• a sample of publicly-funded and independent schools;
• administrative data for linkage by Scottish Candidate Number to be provided by ScotXed;
• within the sample of schools, all pupils in the year group OR a random sample;
• survey to be conducted in school using paper or web-based questionnaires;
• measures to follow-up absentees and coverage of young people in alternative education provision;
• an additional element of telephone interviews with parents to establish background information;
• follow-up sweeps conducted with young people in their own homes using a mixed mode approach;
• boosted sampling of young people in sub-groups of policy interest;
• incentives to be used at follow-up sweeps to maximise response;
• administrative data for linkage by SCN to be provided by SQA, Careers Scotland and other MIAP organisations;
• regular communication with respondents between sweeps to increase sense of survey identity; study website.

Figure 6.1: Summary of Design A

* provisional

Costs for Design A

First sweep of research - in schools

6.3 As discussed in Chapter 5 there are two main options for data collection in schools - paper based questionnaires and web-based questionnaires. In addition, there are two options in respect of the proportion of pupils to be surveyed: the entire year group or a sample. Guideline costs are included for each approach and potential sample size. All costs exclude VAT and are based on 2007 rates so inflation may need to be added. Costs are given for one survey sweep in schools between S1 and S4, and the cost should be doubled for two sweeps.

6.4 Figure 6.2 presents the costs for Design A, outlining the assumptions on which these costs are based. Figures 6.3 and 6.4 then summarise the costings.

Figure 6.2: Costs for Design A

Figure 6.3: Summary of costs for Design A - one sweep in compulsory education

NB: Costs exclude VAT

Figure 6.4: Summary of costs for Design A - two sweeps in compulsory education

NB: Costs exclude VAT

Design B

6.5 The second potential design for the new study would be to continue the SSLS approach of first surveying young people in the year after compulsory education but with major changes to the sampling frame, to the sample, to the data collection strategy and with greater linkage of administrative data. As we have suggested, this approach could also be used as an interim measure if it was decided to proceed with Design A as the longer term strategy.

6.6 The key design features of Design B include:

• 1st contact in the year after compulsory education (16-17)
• sample frame ScotXed
• sample - boosted by eg attainment, deprivation
• 1st sweep achieved sample = 10,000
• contacted at home using mixed methods - telephone, web, postal, face-to-face
• contacted again at: 18-19; 22-23?; 26 -27?
• incentives to encourage response
• administrative data for linkage by SCN to be provided by ScotXed, SQA, Careers Scotland and other MIAP agencies
• website and other measures to maintain contact

6.7 In Design B, the first contact with young people would be in the first year after the completion of compulsory education with subsequent sweeps as outlined for Design A.

Figure 6.5: Summary of Design B

* provisional

6.8 A key difference of Design B compared with Design A is that in Design B, the contact with young people in the year after compulsory education will be the first contact with them whereas in Design A they would already been surveyed one or more times in school. This will impact on costs and may affect response rates.

Costs

6.9 One issue is that without the prior contact in school, telephone numbers may not be available for as many respondents as in Design A. In Design B the survey would be reliant on what schools can provide and what can be found via telephone number matching. As such we have assumed that only 50% of the initial sample would have available telephone numbers in our costs compared with our assumption of 80% in Design A.

6.10 The second difference is that since this would be the initial sweep of research it would therefore lack the recognition that could have been built up by continuing "keep in touch" exercises following the school based survey. This may affect response rates.

Data collection methods

6.11 In putting together our guideline costs we have assumed that the data collection strategy in Design B will be the same as that proposed for the post S4 sweeps of Design A. Thus in Design B all sweeps of research would be conducted with respondents at their home address using a mixed methodology of telephone, postal, web and face to face interviewing.

6.12 Respondents would initially be assigned to either telephone interviewing or a combined postal / face to face data collection strategy depending on whether telephone numbers are available. In addition to this, all respondents would be able to complete the survey online. This design would then be repeated for each of the subsequent sweeps.

Web-based data collection

6.13 In the first instance all respondents should have the option of completing the follow-up survey online. Where e-mail addresses are available respondents should be invited to participate via a hyperlink and password/study number sent out by e-mail. In addition to this, the existence of the online completion option could be highlighted in all advance correspondence sent to respondents along with the password/study number they would need in order to access the site.

6.14 Respondents who complete the survey online would then be removed from the sample for the other data collection strategies to which they had been assigned. We have assumed that around 500 respondents (5%) would choose to complete the survey online for the purpose of these guideline costs.

Telephone data collection

6.15 As noted above, we have assumed that telephone numbers would be available for approximately 50% of all sample members in our guideline costs. For costing purposes we have assumed that 5,000 respondents would be assigned to CATI data collection and that 3,000 (60%) would complete the interview over the telephone or have completed the interview online. We have also assumed that 1,000 respondents would be eligible to be re-assigned to the combined postal and face to face data collection strategy due to an incorrect telephone number or failure to make contact.

Combined postal and face to face data collection

6.16 A combined postal and face to face data collection strategy would be used for those for whom no telephone number is available or who were not contactable during the telephone interviewing stage. We have assumed that 6,000 respondents would be eligible for this data collection strategy.

6.17 In the first instance a postal data collection strategy would be used for these respondents with non-responders followed up by face to face interviewers using Computer- Assisted Personal Interviewing ( CAPI). In our guideline costs we have assumed that 2,400 respondents (c.40%) assigned to this strategy would return a postal questionnaire or have completed the survey online. The remaining 3,600 sample members would then be transferred to face to face data collection.

6.18 Face to face interviews would be used with the most difficult to reach young people. In our costs we have assumed that around 1,800 interviews would be conducted face to face from an initial start sample of 3,600 contacts.

6.19 In total 7,200 interviews would be completed at Sweep 1.

Figure 6.6: Guideline costs for Design B

Figure 6.7: Summary of costs for Design B - 1 ^st sweep after S4

Guideline costs for a 'Keep in Touch' Exercise

6.20 In order to maintain response between sweeps and minimise attrition we would recommend conducting a number of "keep in touch" exercises with respondents and these are outlined in the next chapter. Here we provide a unit cost for a single "keep in touch" exercise based upon an achieved sample of 10,000 respondents; this can be pro-rated for smaller sample sizes.

6.21 Guideline costs for a single "keep in touch" exercise £8,500 + VAT.

Guideline costs for data linkage and derived variables

6.22 We have recommended the linkage of administrative data as an integral part of the longitudinal study design and below give some guideline costs for this (Figure 6.8). These costs exclude overheads. It should also be noted that the costs are based on the assumption that the MIAP strategy will have been successfully implemented and that data will be available for linkage.

Figure 6.8: Data linkage and derived variables

Figure 6.9: Summary of costs for data linkage and derived variables (excludes overheads/ VAT)

Costs for analysis and reporting

6.23 For the sake of completeness, we also include here the tasks and costs for analysis and reporting. This is discussed further in Chapter 8.

Descriptive reporting

6.24 For each survey sweep there will be:

• data manipulation, including joining of survey sweeps;
• descriptive cross-sectional analysis (frequencies, crosstabs etc);
• descriptive longitudinal analysis (eg comparing destinations by timepoints);
• reporting of key topics (eg attainment, attitudes/aspirations, career progression, migration, family formation - each of which will need to take account of progression between sweeps, and differences by gender, social class and area);
• writing Briefing papers.

Special studies of policy areas

6.25 As the longitudinal study develops over time, there will be areas of policy interest that need in-depth longitudinal analysis. These will probably require statistical modelling to explore causal relationships. It is very difficult to estimate the amount of time required for such analysis and reporting but we have attempted to give an indication of this.

« Previous | Contents | Next »