The Scottish Government

« Previous | Contents | Next »

Listen

Scottish Household Survey: Fieldwork outcomes 2003

2. Fieldwork targets and outcomes

The requirements of the sample for the survey are as follows:

• it should provide an achieved national sample of 31,000 interviews over two years
• interviews should be spread evenly across the 24 months of interviewing
• the sample should be fully national in character (i.e. covering the whole of mainland Scotland and the Islands) and each quarter should produce nationally representative results
• results as reliable as those of a simple random sample of 500 should be available for the larger local authorities on an annual basis and for all local authorities (regardless of size) after 2 years
• the sample should be capable of producing data representative both of Scottish households and the adult (aged 16+) population resident in private households.

With the sample designed to meet these objectives, these represent the key performance criteria for the survey. The survey's administration procedures are designed to minimise the impact of problems such as potential respondents not being at home or being unable to take part because of communication difficulties. Interviewers are required, for example, to make a minimum of 6 calls at each address on different days and at different times before it is considered 'no contact'. Even then, addresses will be reissued at a later stage in the fieldwork. Similarly 'soft' refusals such as 'too busy' or 'going out' are reissued.

Where interviewers are unable to conduct interviews in English or because the respondent is blind or partially sighted, these addresses will be revisited by an interviewer accompanied by an interpreter and, if appropriate, with showcards printed in Braille, Gaelic, Urdu or Arabic to minimise the extent to which language and communication barriers prevent people from taking part.

Nevertheless, participation in surveys is voluntary and some potential respondents refuse to take part. Similarly, no contact may be made at an address. This often reflects combinations of household types (single adults of working age), lifestyles (long working hours, active social lives), particular types of neighbourhoods and passive refusal (reluctance to open doors to strangers).

Sample performance

The first stage in assessing the performance of the sample is to determine the extent to which the sample selected for the survey matches the population from which the sample was drawn. Although it is unlikely, there is a theoretical possibility that a random sample does not match the population. This possibility is increased by clustering the sample as well as disproportionate sampling between local authorities.

The table below therefore assesses the extent to which the sample drawn for 2003 reflects the Mosaic profile of Scotland, after taking account of disproportionate sampling.

Table 2-1: Mosaic profile of sampled addresses and all Scottish households

Sample data is weighted to take account of disproportionate sampling between local authorities

Deadwood

The sampling for the survey ¹ makes assumptions about the proportion of addresses that will be ineligible for interview in each local authority. This is based on the proportion that was actually identified in the course of the 1999/2000 SHS fieldwork. The extent to which these assumptions are accurate has an important bearing on the survey outcomes. If there are more 'deadwood' addresses, the interviewers have a smaller pool of addresses from which to achieve the target number of interviews. Conversely, a smaller proportion of 'deadwood' addresses should make it easier to achieve the target number of interviews but this target will be met with a lower response rate. Thus, overall, if the proportion of deadwood differs from the sampling assumptions this might have some impact on achieving the interview target and the target response rate.

Table 2-2 shows the proportion of deadwood addresses assumed in each local authority when sampling for 2003 and compares this with the proportion recorded by interviewers in the field.

Table 2-2: Deadwood rate assumptions and actual deadwood

Sorted in descending order of deviation between actual and assumption

This shows that overall, and in most local authorities, the level of deadwood recorded by interviewers was very close to that used as the basis for the survey sampling. In some rural areas such as Argyll and Bute and Angus, however, the actual proportion was higher than expected and similarly, lower in the rural and Island areas of Highlands, Eilean Siar, Shetland and Orkney.

Fieldwork performance

The profile of the sample selected and the level of deadwood are primarily qualities of the sampling frame and the assumptions used to sample. Inaccuracy and bias in these can have a knock-on effect on fieldwork performance. The other elements of fieldwork performance reflect:

• survey administration procedures and interviewer performance
• the availability of members of the public to be interviewed
• the ability of members of the public to participate in the interview
• the willingness of members of the public to participate in the research.

Performance on each of these elements (as well as deadwood) is recorded as part of interviewers' attempts to secure interviews although there is, inevitably, interaction between these different aspects of performance. Overall, performance is summarised in the survey response rate and this is shown below for the 2003 sample. This takes account of the continuous nature of the survey. The data file for each year will contain a small proportion of interviews conducted on samples drawn in previous years. Similarly some of the addresses issued during any year will not be carried out until after the data file has been closed for analysis. These interviews are carried into the next data file. The response rates therefore report the outcomes for addresses sampled for a given period regardless of when the interview was carried out.

Table 2-3: Summary of outcomes at issued addresses for 2003 sample

Eligible and unknown eligibility are treated as valid addresses for calculating the response rate

This shows a response rate of 66.5% at the time of publication although as sampling is conducted for a two-year period, addresses remain 'live' and might be converted to interviews in the course of 2004. The 2003 response rate will, therefore, be interim and continue to increase until all addresses for the 2003/2004 sweep of the SHS are closed at the end of 2004.

Trends in response rates

The 2003 response rate is broadly consistent with the rates achieved in both of the previous (two-year) sweeps of the SHS. In 1999/2000 the overall response rate was 66% and in 2001/2002 was 67%. The table below shows the rates for each local authority.

Table 2-4: Trends in SHS response rates 1999 to 2003

The highest response rates were achieved in Scottish Borders and Orkney, both with a response rate of 78% and in Shetland and Eilean Siar, with response rates of 77% and 76% respectively. In general, the highest response rates were achieved in predominately rural areas. The lowest response rates were in Glasgow (57%), Edinburgh (61%) and Aberdeen (also 61%). As can be seen below, these authorities are also those where there are the largest number of interviews outstanding against target and where, with mopping up continuing over 2004, there should be an expectation of response rates increasing.

Achieved interviews compared with targets - household interviews

The number of interviews compared with the target, and the corresponding response rates, are the principal measures of survey performance although issues of data quality and bias also need to be considered. The table below compares interview targets and achievement in each local authority.

Table 2-5: Household interview targets and numbers achieved in each local authority, 2003

The highest levels of under-achievement were in Glasgow, Edinburgh and Aberdeen - Scotland's three largest cities and this has been a consistent feature of the SHS since it started, reflecting the concentrations of household and neighbourhood characteristics that make refusal and non-response more likely. Taken together, these authorities account for 65% of the total shortfall (i.e. the sum of all the authorities where the interviews achieved is short of target) and 86% of the overall shortfall (i.e. the difference between total target and total achieved). The highest proportionate shortfalls were in Perth and Kinross and Glasgow.

Achieved interviews - the random adult

The two-part structure of the SHS interview requires the selection of a random adult within the household who completes the second half of the interview. This represents a second opportunity for potential respondents to withdraw from the interview either because they refuse to take part or are unable, unavailable or not contactable for interview.

There is inevitably a degree of attrition between the household and random adult sections of the interview and overall, random adult interviews were achieved at 94% of households where a household interview was completed. The participation rate varied from 86% in Aberdeen to 99% in some authorities. This means that while the household response rate was 67%, the random adult response rate was 63%. As with the household response rate, this is almost the same as the rates of 62% in both the 1999/2000 and 2001/2002 sweeps. Across local authorities, there is some variation in the random adult response rate, with random adult response rates of only 53% in Aberdeen, 54% in Glasgow and 55% in Edinburgh. This compares with rates of 75% in Eilean Siar and Orkney and 76% in Shetland.

Table 2-6: Random adult (RA) response rates, 2003

In this table the addresses whose eligibility is not known in Table 2-3 are treated as valid

1. See ( Scottish Household Survey: Methodology 2003/2004) for further information.

« Previous | Contents | Next »