Paper SPSCC/06/03 - LONG TERM STRATEGY FOR POPULATION SURVEYS IN SCOTLAND
Proposed research in to implications of unclustered and modular design on large scale surveys
This document is also available in PDF format
1. Members are asked to comment on the draft research specification on unclustered and modular designs.
2. The responses to the survey of surveys (including draft responses for SCVS and SHeS) were considered at meeting of survey managers in October 2005. Together with the thinking emerging from the survey reviews have helped to set out the research questions around the implications of unclustered and modular designs. Answers to these questions will influence the future design of surveys and will be valuable to procurement process when evaluating the large scale contact bids.
3. OCS plan to contract out this research on unclustered and modular designs. The Scottish Household Survey is the main focus of the research but there would be real read across to all large scale surveys. A draft research specification is set out at Annex C.
Robert Williams / OCS / January 2006
ANNEX A - DRAFT RESEARCH SPECIFICATION - UNCLUSTERED AND MODULAR DESIGN
1. The Scottish Household Survey is the main focus of the research but there would be read across to all large scale surveys.
2. The definition of reliable and examples of reasonable confidence intervals for different proportions should be discussed.
- What sample size is required to give reliable estimates for the 2, 6 and 8 fold urban rural classifications?
- What sample size is required to give reliable estimates for the remote accessible split on the 6 and 8 fold urban rural classifications?
- What sample size is required to give reliable estimates for the 15% most deprived data zones, and SIMD quintiles?
- What sample size is required to give reliable estimates for local authorities?
3. What are the benefits of unclustered and clustered designs.
4. Estimates will be required for the different geographies for different frequencies, the effect of the overall sample size ranging from 15 to 20 thousand plus households would have on the design of the survey should be discussed.
5. The effect the requirements for estimates for different geographies over different frequencies would have on the design of the survey should be discussed (for example clustered and unclustered).
6. If the Scottish Executive's large scale surveys (including the Scottish Household Survey, Scottish health Survey, Scottish Crime and Victimisation Survey, and Scottish House Condition Survey) had a core set of socio-economic questions asked on all surveys, what changes to survey design would be needed to allow unit responses for the core questions be brought together to provide estimates for smaller geographies.
7. The impact of varying sample size will have on survey costs should be considered. As the sample size increases does the difference in interviewer travel costs in remote and accessible areas converge and so make unclustered sampling more cost effective. Would unclustered design mean fewer interviews providing estimates of equivalent confidence and what impact would this have on costs.
8. The different types of modular design should be explored, with the view of a survey that has a socio-economic, communities, local government, and transport questions asked of all the sample, and communities, local government, and transport questions asked of a sub-sample. How does the design of the modules interact with the requirements for estimates for the different geographies for different frequencies?
9. What impact will changing individual questions, and or adding new modules have on the design of modular surveys, and in particular could new modules and extra sample be added to the survey design during a sweep of the survey, or what would be an appropriate lead time? For example, what are the implications of increasing the sample size to allow an additional module including questions for a sub-sample with respect to the overall design and sample in individual local authorities etc.
10. What are the implications of local boosts on modular and unclustered design.
11. The implications of modular and unclustered design on weighting strategies should be explored.
12. What impact will the length of the survey sweep and sample selection have on the survey design?