The Scottish Government

« Previous | Contents | Next »

Listen

Scotland's People: results from the 2001/2002 Scottish Household Survey
Volume 8: Technical Report

4. Response rates, achieved sample profile and weighting

4.1 Achieved sample size

The nature of the SHS as a survey with continuous fieldwork means that separating the data into fieldwork periods and data files for analysis involves simply establishing cut-off dates and any data submitted to the CAPI servers by a particular date is assigned to one data period or another. In each dataset there are a few interviews that strictly speaking belong to a different data period. For example, in the 2001/2002 data 75 of the 30,639 interviews were carried out on sample issued before 2001. Similarly, there are 160 interviews carried out on 2001/2002 sample that were not on the CAPI servers when the 2002 data file was closed. These interviews are carried into the file for 2003.

When calculating response rates based on the sample issued in 2001/2002, we need to exclude the 75 interviews from the previous sample and include the 160 interviews that will be in the 2003 data. There is, therefore, a difference of 85 interviews between the records in the 2001/2002 data file and the number of interviews achieved on 2001/2002 sample, as shown below.

As Table 4-1 shows, the 30,724 were spread quite evenly across the eight quarters covered by the survey within local authorities. On a month-on-month basis, the variation is greater. There are likely to be a variety of reasons for this fluctuation, including the unavailability of interviewers and potential respondents because of sickness or holidays, periods of bad weather which disrupt fieldwork, and slight variations in the fieldwork allocation by month within individual local authority areas.

Table 4-1 interviews by local authority by quarter in which sample was issued: SHS 2001/2002 (based on quarter of issue not interview)

4.2 Household response rate

The following table shows the detailed outcomes of issued addresses for the 2001/2002 SHS as a whole. Across the two years, 91% of issued addresses were valid and within these the response rate for the survey (where at least a 'householder' interview was achieved) was 67%. The number of interviews achieved over the two years is 99% of the target of 31,000 but the response rate is 3% below the target of 70%. This reflects a number of additional factors:

• the proportion of invalid addresses is slightly lower than the projected 10% used to sample addresses. This tends to reduce the response rate
• about 3% of addresses appear to interviewers to be unoccupied but this has not been confirmed by neighbours so these addresses have been counted as valid addresses, although some of them might be vacant (and therefore invalid), which again tends to depress the response rate
• A further 3% of contacts are either missing (lost in the post or simply not returned by interviewers) or have been returned without an outcome coded. Again, some of these might be invalid addresses but because we do not know, we include them all as valid addresses. This too tends to depress the response rate.

Table 4-2 Summary of outcomes at issued addresses for the 2001/2002 SHS

Examination of overall household response rates by local authority area reveals considerable variation both in the level of 'deadwood' in the sample and in the achieved response rate. The combined effects of these mean that the original projections of annual sub-sample size have been surpassed in some local authority areas but not reached in others. The response rate is highest in Eilean Siar (81%) and Orkney (80%) and lowest in Edinburgh (60%) and Clackmannanshire (62%). The following table shows this in detail.

Table 4-3 Household response rate, targets and invalid addresses by local authority: 2001/2002
Change in response rates between 1999/2000 and 2001/2002

In 2001/2002 the worst performance was in Edinburgh and Glasgow, both of which have achieved 91% of target. This is an improvement on 1999/2000 where the worst performance for achieving interviews against target was in Glasgow and Dundee, both of which stood at 87% of target when we last reported. The best performances were in Eilean Siar, with interviews 12% over target and a response rate of 81% and Orkney with interviews 15% over target and a response rate of 80% of valid addresses.

4.3 Random adult response rate

Since there is no substitution if the selected 'random adult' is unavailable or unwilling to take part in the second part of the interview, there is inevitably a degree of attrition in response rates between the first and second parts of the interview. This means that the 'random adult' response rate is lower than the household response rate. While a successful part one (highest income householder) interview was carried out at 67% of valid addresses, a part two (random adult) interview was achieved at only 62%. This is the same as the 1999/2000 figure of 62%. This means that a random adult interview was achieved at 93% of the addresses at which a part one interview had been completed. This is higher than the original projections for the survey, which estimated that a successful 'random adult' interview would be completed at only 90% of households participating in the first part of the survey. However, while the household response rate has stayed more or less the same over the first four years of the survey there has been a gradual decline in the proportion of random adult interviews achieved.

Table 4-4 shows the number of completed 'random adult' interviews expressed as a proportion of households that took part in the first part of the interview and as a proportion of valid addresses. Again, there is some variation by local authority, especially if the combined effects of household and individual non-response is considered.

Clearly, if non-response was distributed evenly throughout the population, it would have no impact on the representativeness of the achieved sample. However, since non-response tends to concentrate in particular sections of the population, there is good reason to examine the profile of the achieved sample to ensure that no systematic bias has been introduced. This issue - and the related question of weighting the data - is discussed below.

Table 4-4 Random adult response rate as percentage of valid addresses and of achieved household interviews: SHS 2001/2002

« Previous | Contents | Next »