On this page:

Paper 26th January 2006 - SPSCC-06-04

Paper SPSCC/06/04 - LONG TERM STRATEGY FOR POPULATION SURVEYS IN SCOTLAND

SURVEY OF SURVEYS

This document is also available in PDF formatAdobe Acrobat (pdf) Icon

1. Members are asked to note the information on survey of surveys.

2. Part of the long term strategy involves mapping out the content, coverage, geography, frequency and cost of the Scottish Executive's large scale surveys. This work is been taken forward through the survey of surveys. The survey of surveys covers three broad areas including survey design and contracts, outputs and management. Responses from Scottish Household Survey and Scottish House Condition Survey are included at Annex A.

3. Responses from the SCVS and SHeS should be completed in time for the next SPSCC.

Robert Williams / OCS / January 2006



ANNEX A - SURVEY OF SURVEYS

SURVEY DESIGN AND CONTRACTS

Scottish Household Survey

Scottish House Condition Survey (SHCS)

National Statistic

YES

No

Contractor

MORI and TNS

Office for National Statistics and Miller Mitchell Burley Lane

Contract Length

Jan 2003 - Dec 2006

2003 - 2006 (new contracted to be awarded in Spring 2006)

Contract Cost

£4M

£5m approx

Start of next tender process

Spring 2006

November 2005

Period of next potential contract

July/August 2006-July/August 2011

Fieldwork period: January 2007-December 2010

2006 - 2009

Survey Reference Period

Calendar years

October to September

Survey Structure

Continuous and cross-sectional

Continuous and cross-sectional

Interview Mode and specialised components

Face-to-face CAPI

CAPI interview

Building survey

Sampling Frame

Postal Address File

Postal Address File

Sampling Design

Systematic random sample where fieldwork conditions allow (areas of high population density). Unclustered in local authorities where population density of 500 or more persons per square kilometre. Clustered in those local authorities where population density of 500 persons or less per square kilometre.

Stratified random sample designed to give minimum number of paired surveys per local authority area

Primary reporting units

Households and random adults

dwellings

Secondary reporting units

Children

households

Response rate

1999/2000 - 66%

2001/2002 67%

2003/2004 69%

70% response to social survey

83% conversion rate to physical survey

Target Sample size (households)

15.5K per calendar year (31,000 over the two year sweep)

3000 paired (social and physical) cases per year

Target Sample size (random adults)

Target is to complete random adult interview in 90% of households in which a household interview is carried out.

n/a

Target Sample size (children)

We don't have an explicit target sample size for any attribute other than the expectation that a random sample with a high response rate will yield a sample with attributes that reflect the population in private households with the limits of sampling variability.

n/a

Door step routine

Interviewers are required to make up to six calls at an address (an initial visit plus five 'call-backs'). In addition to the immediate reissue of contact sheets that have been wrongly completed or where the required number of call-backs has not been made, there is an on-going programme of reissuing 'non-contacts' in a bid to maximise the response rate. At the end of each fieldwork year a significant number of valid but 'non-contact' addresses remain 'live'.

Interviewers make minimum of six calls (at least two at evening and weekends) and use kish grid to select households where there are multiple addresses. Physical surveyors make four visits - at least two at evenings and weekend

Field force size

Each month there are about 100 interviewers working on the SHS from the two companies. Not all will work each month so over the course of a year, approximately 200 (214 in 2004) will work on the SHS.

ONS employ approx 60 interviewers on SHCS - MMBL approx 50 surveyors

Groups excluded from sample

Communal establishments

Communal establishments

Sample coverage

Scotland

Scotland

Questionnaire quality assurance

Individual questions are discussed and agreed between the contractor and the Executive. These are rarely subject to piloting (the only example I can think of is the volunteering questions which were piloted although not by us). They are often drawn from other government surveys so have been piloted in that respect.

The research team on the contractor side specify the questions for scripting. These are checked by the person responsible for the questions within the SE.

Scripted questions are checked on paper - reading them and the associated programming - and comparing against spec.

The full CAPI script is tested by a number of researchers in the contractor team, with each route in the questionnaire checked to ensure that the script works as specified.

Checking is generally limited by the fact that questions tend to be agreed very close to the point where the questionnaire needs to go into the field.

Early in the fieldwork, data is extracted from the CAPI servers and checked to ensure that the scripted questions are working properly.

Hard and soft checks built in to CAPI script - validation of physical survey is done by MMBL and Communities Scotland

Dataset quality assurance

Most of the quality assurance on the data is built into the script, which checks the logic of responses, especially in the HA section, which collects age, sex, relationship and occupational data for each household member. These checks are confirmed as part of the data processing. In addition, checks are carried out on all data as part of the process of publication.

Data validation carried out by ONS and Communities Scotland

Frequency of data delivery from contractor

Quarterly

Quarterly



SURVEY OUTPUTS

Scottish Household Survey

Scottish House Condition Survey

Areas for which results are available

Scotland every quarter

Groups of LAs every year

All LAs every two years

Urban Rural every year

Deprivation quintiles every year

Scotland

Local authorities

Urban/ rural

Health boards

Frequency for key results

Every quarter

Annual at national level

2-3 years for Local Authority level

Time between collecting survey data and releasing results

5 months for Quarterly Bulletins

7 months for Annual Report

Although at the end of each two-year cycle, the time between closing of datasets and publication of the annual report is about 5 months (mopping up on 2004 continued until 5 Feb 2005 and AR was published in August).

18 months between fieldwork beginning and survey results being available

Weighting

No additional corrective weighting has ever been applied to the data beyond that required to account for sample design and differential response rates between local authorities.

Weighting carried out my main contractor. Weights account for differential non-response

Access to data sets for ASDs

SAS and SPSS data sets available

SPSS data sets available

Access to data sets for External Users

Data Archive

Data archive

Key policy divisions supported

Transport, communities and local government

Social Housing Strategy and Finance

Fuel Poverty and Supporting people

Housing Bill Team

Communities Scotland Quality and Transfer Delivery Unit

Main reports

Annual Report

Annual Technical Report: Methodology

Annual Technical Report: Field Outcomes

Annual Technical Report: Questionnaire

Quarterly Bulletins

Main report (2002)

Local Authority Report (2002)

Specialised reports

Scottish Transport Statistics

- as published in August 2004

Key Transport Statistics card

- February 2005 version

Household Transport: some Scottish Household Survey results

- as published in November 2004

Transport Across Scotland: some SHS results for parts of Scotland (biennial)

- as published in February 2004

SHS Travel Diary results

- as published in March 2005

Bus and Coach Statistics

- as published in March 2005

Accessibility and Transport

Modal Shift

Volunteering in Scotland: Evidence from the Scottish Household Survey 2000 (7 January 2001)

Older People in Scotland: results from the first year of the Scottish Household Survey -Charlotte MacDonald with Helen Storkey and Gillian Raab, Napier University

Older people in Scotland : results from the Scottish Household Survey 1999-2002 Raab, G (Napier University) and MacDonald, C

The Social Focus publications make extensive use of SHS data

The above are listed on the SHS website.

Follow-up reports on Health, Disrepair and Fuel Poverty

Results by gender

YES

Yes

Results by Religion

SOME

Yes

Results by Disability/LLTI

Some - depends on how disability is defined

Some

Results by age (including older people)

YES

Results by Minority Ethnic Groups

Collect but do not include results as part of standard outputs due to small sample size

Collect but small sample size means results available at 'White/Non-white' level only

Results by Sexual Orientation

NO

No



SURVEY MANAGEMENT

Scottish Household Survey

Scottish House Condition Survey

Process for agreeing content

The questionnaire revision process to date has consisted of a call for bids for new questions. These were then assessed by the SHS team and decisions were made within the Development Department.

The current questionnaire consultation requires lead analysts (owners of questions) to consider their current questions in relation to policy needs and plans for dissemination. They are required to submit a draft package for 2007, justifying every question, whether existing or new, in terms of policy links and dissemination plans.

Consultation every two years with key stakeholders

Survey Technical Management

The SHS has a technical group which meets every two months. Its remit is as follows:

1. Review progress of contract, short-term on workloads and long term on consequences of other Scottish social surveys

2. Advise project manager on contract issues

3. Advise on data issues

4. Advise on dissemination of survey

5. Advise on publications and approve inclusion of questions

6. Monitor progress within working groups

7. Advise on reviewing and re-tendering of survey

8. Individual members to take forward areas of work agreed by technical group.

Survey managed by House Condition Surveys team at Communities Scotland

Survey Strategic Management

Strategic direction of the survey to date has been governed by Neil Jackson, John Tibbitt and Duncan Gray. Currently, the SHS Review Board will make strategic decisions on the future direction of the SHS based on the findings of the SHS Review.

Survey Contract Management

Contractors managed by SHS Project Manager.

Contract managed by House Condition Survey Manager

Survey day to day Management

Project manager: B3 Key tasks:

Group Management:

Contract Management:

People Management:

Change Management

Publications Management

Oversee annual publications process

Data/Enquiry Management:

Overall view of datasets and enquiries

Oversee Quality Assurance, initiate new developments and dissemination of main dataset

Dissemination/Presentations

Maintain high-profile and widespread dissemination policy

Questionnaire Development

Establish and maintain relationship with other survey teams

Keep up to date ONS survey protocols

Liaise with OCS on Scottish Survey developments

SHS Review

Research Officer B1

TASK

Group Secretariat:

Data/Enquiry Management:

Contract Management

Publications Management

Project management of SHS annual publications

Questionnaire Development

Co-ordinate questionnaire revision

Website Management and Development

SHS Review

House Conditions Survey Team at Communities Scotland

Internal Stakeholder Involvement

Annual user day (March 2006)

Tailored seminars on request (transport, health and housing in 2005)

SHS News e-mail

External Stakeholder Involvement

Annual user day (March 2006), SHS News e-mail scotstat distribution lists, Regularly up-dated SHS internet site, some involvement in short life groups, User community database on SHS website

Limitations of the survey

Limited analytical resources attached to the main team. Hinders topic areas without SHS statisticians. Limited number of short, focused papers tailored to policy customers. Frustration at questionnaire revision process to date. Lack of transparency in decision-making. Huge pressure on questionnaire content.

Small sample size means analyses of sub-groups not available every year

Income data is not complete - gather data from HIH and spouse/partner only

Page updated: Monday, February 6, 2006