The Scottish Government

« Previous | Contents | Next »

Listen

2. Fieldwork targets and outcomes

The requirements of the sample for the survey are as follows:

• it should provide an achieved national sample of 31,000 interviews over two years
• interviews should be spread evenly across the 24 months of interviewing
• the sample should be fully national in character ( i.e. covering the whole of mainland Scotland and the Islands) and each quarter should produce nationally representative results
• results as reliable as those of a simple random sample of 500 should be available for the larger local authorities on an annual basis and for all local authorities (regardless of size) after 2 years
• the sample should be capable of producing data representative both of Scottish households and the adult (aged 16+) population resident in private households.

With the sample designed to meet these objectives, these represent the key performance criteria for the survey. The survey's administration procedures are designed to minimise the impact of problems such as potential respondents not being at home or being unable to take part because of communication difficulties. Interviewers are required, for example, to make a minimum of 6 calls at each address on different days and at different times before it is considered 'no contact'. Even then, addresses will be reissued at a later stage in the fieldwork. Similarly 'soft' refusals such as 'too busy' or 'going out' are reissued.

Where interviewers are unable to conduct interviews in English or because the respondent is blind or partially sighted, these addresses will be revisited by an interviewer accompanied by an interpreter and, if appropriate, with showcards printed in a range of languages to minimise the extent to which language and communication barriers prevent people from taking part.

Nevertheless, participation in surveys is voluntary and some potential respondents refuse to take part. Similarly, no contact may be made at an address. This often reflects combinations of household types (single adults of working age), lifestyles (long working hours, active social lives), particular types of neighbourhoods and passive refusal (reluctance to open doors to strangers).

Sample performance

The sampling for the survey ¹ makes assumptions about the proportion of addresses that will be ineligible for interview in each local authority. Ineligible addresses would include derelict, vacant and non-residential addresses. The allowance for ineligible addresses is based on the proportion that was actually identified in the course of the 1999/2000 SHS fieldwork, the most recent two-year sweep when the sample was being designed in 2002. The extent to which these assumptions are accurate has an important bearing on the survey outcomes. If there are more 'deadwood' addresses, the interviewers have a smaller pool of addresses from which to achieve the target number of interviews. Conversely, a smaller proportion of 'deadwood' addresses should make it easier to achieve the target number of interviews but this target will be met with a lower response rate. Thus, overall, if the proportion of deadwood differs from the sampling assumptions this might have some impact on achieving the interview target and the target response rate.

Table 2 1 shows the proportion of deadwood addresses assumed in each local authority when sampling for 2005 and compares this with the proportion recorded by interviewers in the field.

Table 2 1: Deadwood rate assumptions and actual deadwood

Sorted in descending order of deviation (absolute value) between actual and assumption

This shows that overall, and in many local authorities, the level of deadwood recorded by interviewers was close to that used as the basis for the survey sampling. There is, of course, some deviation from the assumptions, reflecting sampling variability in both the base data used for sampling and the sampled addresses and the passage of time since 2000. In spite of the deviation from assumptions, using different deadwood assumptions in individual local authorities rather than previous practice of a uniform 10% in all areas improves the structure of the sample and should contribute to meeting fieldwork targets. Where the experience differs from the assumptions this is likely to reflect a combination of factors such as:

• housing regeneration and redevelopment, which leads to demolition and vacant properties (increasing deadwood) and properties being brought back into use (lowering deadwood).
• expansion of holiday properties and second homes, which are ineligible for inclusion in the survey, raising deadwood.

The profile of the sample selected and the level of deadwood are primarily qualities of the sampling frame and the assumptions used to sample. Inaccuracy and bias in these can have a knock-on effect on fieldwork performance. The other elements of fieldwork performance reflect:

• survey administration procedures and interviewer performance
• the availability of members of the public to be interviewed
• the ability of members of the public to participate in the interview
• the willingness of members of the public to participate in the survey.

Performance on each of these elements (as well as deadwood) is recorded as part of interviewers' attempts to secure interviews although there is, inevitably, interaction between these different aspects of performance. Overall, performance is summarised in the survey response rate and this is shown below for the 2005 sample. This takes account of the continuous nature of the survey. The data file for each year will contain a small proportion of interviews conducted on samples drawn in previous years. Similarly some of the addresses issued during any year will not be carried out until after the data file has been closed for analysis. These interviews are carried into the next data file. The response rates therefore report the outcomes for addresses sampled for a given period regardless of when the interview was carried out.

Table 2 2: Summary of outcomes at issued addresses for 2005 sample

Trends in response rates

The response rate of 69.1% in 2005 is very slightly higher than the 68.9% achieved in the 2003/2004 sweep of the survey and also higher than the previous two-year sweeps of the SHS. In 1999/2000 the overall response rate was 66% and in 2001/2002 was 67%. The table below shows the rates for each local authority.

Table 2 3: Trends in SHS response rates 1999 to 2005

The most notable change between 2003/2004 and 2005 is the decline of 10 percentage points in the response rate achieved in Renfrewshire. The issue in this particular local authority appears to be a few batches of addresses with very high rates of non-contact and/or refusal which have reduced the overall response rate.

The highest response rates were achieved in island and largely rural authorities, with Orkney achieving a response rate of 81% and four local authorities (Shetland, Eilean Siar, Scottish Borders and Fife) with a rate of 78%. The lowest response rates were in Glasgow City (58%), Renfrewshire (61%), East Renfrewshire (63%), the City of Edinburgh and North Ayrshire (both on 64%). Glasgow stands out as the only local authority with a consistently low and declining response rate.

Achieved interviews compared with targets - household interviews

The number of interviews compared with the target, and the corresponding response rates, are the principal measures of survey performance although issues of data quality and bias also need to be considered. The table below compares interview targets and achievement in each local authority.

Table 2 4: Household interview targets and numbers achieved in each local authority, 2005

The highest levels of under-achievement were in Glasgow City and the City of Edinburgh - Scotland's largest cities and this has been a consistent feature of the SHS since it started, reflecting the concentrations of household and neighbourhood characteristics that make refusal and non-response more likely. Taken together, these authorities account for 49% of the total shortfall ( i.e. the sum of all the authorities where the interviews achieved is short of target). The highest percentage shortfalls were in North Ayrshire (83% of target achieved) and Glasgow, Renfrewshire and Eilean Siar (88% achieved).

Achieved interviews - the random adult

The two-part structure of the SHS interview requires the selection of a random adult within the household who completes the second half of the interview. This represents a second opportunity for potential respondents to withdraw from the interview either because they refuse to take part or are unable, unavailable or not contactable for interview.

There is inevitably a degree of attrition between the household and random adult sections of the interview, especially where the person selected is not the same as the household respondent. This aspect of the survey has deteriorated since 1999/2000 when a random adult interview was achieved in 94% of households in which a household interview was completed. In 2005, random adult interviews were achieved at 91% of households where a household interview was completed. The participation rate varied from 84% in Dundee and 85% in Aberdeen to 98% in Orkney and 96% in Shetland. A participation rate of 91% means that while the overall household response rate was 69%, the random adult response rate was 63%. As with the household response rate, this is still higher than the rates of 62% in both the 1999/2000 and 2001/2002 sweeps although slightly lower than the 2003/2004 rate of 64%. Across local authorities, there is some variation in the random adult response rate, with rates of 51% in Glasgow and 57% in Renfrewshire and Edinburgh. This compares with rates of 75% in Fife and Shetland and 80% in Orkney.

Table 2 5: Random adult ( RA) response rates, 2005

« Previous | Contents | Next »