The Scottish Government

« Previous | Contents | Next »

Listen

2. An overview of the surveys

This chapter provides a brief overview of the design of the two surveys and discusses differences in survey administration.

2.1 The face-to-face survey

The face-to-face survey was designed to replicate the previous sweeps of the Scottish Crime Survey, the most recent of which was carried out in 2003 by MORI Scotland and TNS Social. The main features of the sample design were as follows.

• The sample was drawn from a geographical area that covers mainland Scotland and the larger inhabited islands.
• Within this area, sample was proportionately allocated to local authorities.
• Primary sampling units ( PSUs) based on combinations of Census output areas were selected within each local authority with selection probability proportionate to size.
• Addresses were sampled within PSUs based on a target average response rate of 70%, with assumptions for the proportion of ineligible addresses in each local authority based on the average percentage ineligible observed in the Scottish Household Survey ( SHS) between 1999 and 2002. ⁷
• Addresses were randomly allocated as either short or long interviews and, within the long interviews, as either version A or version B. ⁸
• In households with more than one adult aged 16 years or over, the respondent was selected using a Kish selection procedure.
• Interviewers made up to 6 attempts to secure an interview at sampled addresses.
• Interviews were carried out between July and November 2004.

2.2 The telephone survey

The design of the telephone survey illustrates some of the methodological attractions of telephoning interviewing.

• The sample covers the whole of Scotland, with no geographical exclusions.
• The sample is unclustered and has the potential to reach residences that might be excluded from samples based on the small user PAF. ⁹
• Interviews are disproportionately spread across the nine Police Force Areas ( PFA) in Scotland, with a minimum of 3,000 interviews in each, allowing geographical disaggregation of the data at PFA level for household victimisation and fear of crime.
• The telephone survey involves continuous data collection. To be eligible for the survey, telephone numbers must be those of the main residence of a private household in Scotland. Within each eligible household contacted, only one adult aged 16 or over can be interviewed and a random selection process is used to select one person in households containing two or more adults.
• Rather than 6 calls at sampled addresses, the telephone survey makes a minimum of 10 calls (and often many more) to attempt to make contact. These calls can be spread over several weeks.

2.3 Survey administration

Overall, the surveys were designed to be as similar as possible, consistent with normal practice for each mode of interview.

2.3.1 Respondent selection

Respondents to the face-to-face survey were selected by Kish grid whereas respondents in the RDD survey were selected according to which of the adults in the household had the most recent birthday. The key advantage of 'last birthday' over Kish is that the birthday method is less intrusive, requiring no substantive information in advance of respondent selection. It was therefore expected to lead to lower initial refusal than if Kish selection were used. This is borne out by previous research ¹⁰ although some studies have found that birthday methods have resulted in various types of bias. Of particular interest, for reasons that will become apparent below, birthday selection methods have, among other things, been found to result in:

• over-representation of women ¹¹
• people selecting themselves because they want to do the survey. ¹²

2.3.2 Survey instruments and administration

The survey instruments were kept as similar as possible although some differences were inevitable to reflect the requirements of the two modes of interview. Some differences were retained - such as show cards on the face-to-face survey - to retain consistency with normal practice on face-to-face surveys.

The face-to-face sample received advance letters from the Scottish Executive advising them of the survey. Obviously, the telephone sample could not receive an advance letter but the RDD telephone script contained a detailed section of designed to establish the validity of the survey and reassure potential respondents, including offering to send a letter explaining more about the survey.

Broadly, the substantive questions in each survey were the same, consistent with the mode of interview. The main difference was that the face-to-face survey used show cards for some questions such as Question 1 - "I would like to start by asking you about some general social issues in Scotland today. First, do you think unemployment is a problem in Scotland today or not?". Respondents were shown a card with the options 'a big problem', 'a bit of a problem' and 'not a problem'. In the telephone survey, the question was slightly different, with two stages: is the issue a problem or not and if so, how much of problem. Respondents were asked:

"I would like to start by asking you about some general social issues in Scotland today. First, do you think (problem) is a problem in Scotland today or not? IF A PROBLEM: Would you say it is a BIG problem, or a BIT of a problem?

The CATI script had the distinct advantage of reducing the scope for interviewer error by automating question routing and managing tasks such as identifying and selecting the appropriate incidents of crime for follow-up in the victim modules. These aspects of the face-to-face survey required detailed checking of forms as they were returned for coding and data entry.

Overall, although there were differences in the administration of the two surveys, these were not expected to be significant in terms of the survey measures. The surveys were not designed to test the impact of using or not using show cards. This would have required split samples in the face-to-face survey to try isolate the impact of survey mode on responses to show card questions.

« Previous | Contents | Next »