The Scottish Government

« Previous | Contents | Next »

Listen

4 Fieldwork targets and outcomes

The requirements of the sample for the survey are expressed in terms of the ability of the sample to deliver the number of interviews required to meet particular requirements at particular points in time. There are also requirements in terms of the geographical scope of the interviews and their representativeness in terms of the types of people and areas included in the survey. Overall, the survey is expected to be an unbiased sample of Scottish households and adults and this is most likely to be achieved by maximising the level of participation in the survey.

With the sample designed to meet these objectives, these represent the key performance criteria for the survey. The survey's administration procedures are designed to minimise the impact of problems such as potential respondents not being at home or being unable to take part because of communication difficulties. Interviewers are required, for example, to make a minimum of 6 calls at each address on different days and at different times before it is considered 'no contact'. Even then, addresses will be reissued at a later stage in the fieldwork. Similarly 'soft' refusals such as 'too busy' or 'going out' are reissued.

Where interviewers are unable to conduct interviews in English or because the respondent is blind or partially sighted, these addresses can be revisited by an interviewer accompanied by an interpreter and, if appropriate, with showcards printed in a range of languages to minimise the extent to which language and communication barriers prevent people from taking part.

Nevertheless, participation in surveys is voluntary and some potential respondents refuse to take part. Similarly, no contact may be made at an address. This often reflects combinations of household types (single adults of working age), lifestyles (long working hours, active social lives), particular types of neighbourhoods and passive refusal (reluctance to open doors to strangers).

4.1 Sample performance

4.1.1 Ineligible addresses

The sampling for the survey makes assumptions about the proportion of addresses that will be ineligible for interview in each local authority. Ineligible addresses would include derelict, vacant and non-residential addresses. The allowance for ineligible addresses is based on the proportion that was actually identified in the course of the previous year's SHS fieldwork, the most recent two-year sweep when the sample was being designed in 2006. The extent to which these assumptions are accurate has an important bearing on the survey outcomes. If there are more 'deadwood' addresses, the interviewers have a smaller pool of addresses from which to achieve the target number of interviews. Conversely, a smaller proportion of 'deadwood' addresses should make it easier to achieve the target number of interviews but this target will be met with a lower response rate. Thus, overall, if the proportion of deadwood differs from the sampling assumptions this might have some impact on achieving the interview target and the target response rate.

Table 4-1 shows the proportion of deadwood addresses assumed in each local authority when sampling for 2005/2006 and compares this with the proportion recorded by interviewers in the field.

Table 4-1: Deadwood rate assumptions and actual deadwood

Sorted in descending order of deviation (absolute value) between actual and assumption

This shows that overall, and in many local authorities, the level of deadwood recorded by interviewers was close to that used as the basis for the survey sampling. There is, of course, some deviation from the assumptions, reflecting sampling variability in the data used for sampling and the sampled addresses. In spite of the deviation from assumptions, using different deadwood assumptions in individual local authorities rather than previous practice of a uniform 10% in all areas improves the structure of the sample and should contribute to meeting fieldwork targets. Where the experience differs from the assumptions this is likely to reflect a combination of factors such as:

• housing regeneration and redevelopment, which leads to demolition and vacant properties (increasing deadwood) and properties being brought back into use (lowering deadwood).
• expansion of holiday properties and second homes, which are ineligible for inclusion in the survey, raising deadwood.

4.2 Fieldwork performance

The profile of the sample and the level of deadwood are primarily qualities of the sampling frame and the assumptions used to sample. Inaccuracy and bias in these can have a knock-on effect on fieldwork performance. The other elements of fieldwork performance reflect:

• survey administration procedures and interviewer performance
• the availability of members of the public to be interviewed
• the ability of members of the public to participate in the interview
• the willingness of members of the public to participate in the survey.

Performance on each of these elements (as well as deadwood) is recorded as part of interviewers' attempts to secure interviews although there is, inevitably, interaction between these different aspects of performance.

4.2.1 Survey response rates

Overall, performance is summarised in the survey response rate and this is shown below for the 2007 sample. This takes account of the continuous nature of the survey. The data file for each year may contain a small proportion of interviews conducted on samples drawn in previous years. Similarly some of the addresses issued during any year will not be carried out until after the data file has been closed for analysis. These interviews are carried into the next data file. The response rates therefore report the outcomes for addresses sampled for a given period regardless of when the interview was carried out.

Table 4-2: Summary of outcomes at issued addresses for 2007 sample

This is the response rate for the whole sample issued in 2007 - both the main sample and the Culture and Sport module. Separately, the response rate for the main sample was 65% and 67% for the Culture and Sport module.

4.2.2 Trends in response rates

The response rate of 65.7% in 2007 is the lowest response rate achieved on the survey since 1999 and reflects the combined impact of the late start to 2007 fieldwork, the errors in the scripts and the excessive length of the interview at the start of the year.

The table below shows the response rates for each local authority for each two-year period between 1999 and 2006 and compares the 2007 rates with the average in each local authority in that period.

Table 4-3: Trends in SHS response rates 1999 to 2007

The most notable differences between the 1999-2006 average and the 2007 response rate are in Shetland, Renfrewshire and Glasgow (down 6 percentage points), East Ayrshire and Edinburgh (down 7 percentage points) and West Lothian, where the 2007 response rate is 11 percentage points lower than the average for 1999-2006. Some areas have higher response rates than the 1999-2006 average, although fewer and with more modest differences. The largest comparable positive difference is in Fife, where the 2007 response rate is 5 percentage points higher than the average.

In 2007, the highest response rates were achieved in island and largely rural authorities. Response was highest in Eilean Siar (81%), Orkney (80%), Stirling and Fife (76%). Inverclyde - an urban authority - also achieved a response rate of 76% The lowest response rates were, as they generally have been, in Glasgow City (55%), Edinburgh (56%), West Lothian (57%), Renfrewshire (59%) and East Renfrewshire (61%).

4.2.3 Achieved interviews and fieldwork target

Compared with the total target of 17,977 interviews that the sampling for the survey was designed to yield, the 2007 total of 17,093 represents a total shortfall of 884 or 5% of the interview target. As mentioned earlier, in response to the longer interview at the start of 2007, the Scottish Government accepted that the interview target should be reduced by 500 to reflect the lower productivity of fieldwork in the first quarter of the year. This means that the achieved interviews represent a shortfall of 384 interviews or 2% of this revised target.

The revised target was not related to specific local authority targets so the percentage achieved in each local authority shown below relates only to the original target that the sample was designed to achieve.

4.2.4 Achieved interviews compared with targets - household interviews

The number of interviews compared with the target, and the corresponding response rates, are the principal measures of survey performance although issues of data quality and bias also need to be considered. The table below compares interview targets and achievement in each local authority.

Table 4-4: Household interview targets and numbers achieved in each local authority, 2007

The highest levels of under-achievement were in Glasgow City, West Lothian, Renfrewshire and the City of Edinburgh. Taken together, these authorities account for 45% of the total shortfall (i.e. the sum of all the authorities where the interviews achieved is short of target). The highest percentage shortfalls were in West Lothian (83% achieved), Shetland Islands (85%), Edinburgh (88%) and Renfrewshire (88%).

4.2.5 Achieved interviews - the random adult

The two-part structure of the SHS interview requires the selection of a random adult within the household who completes the second half of the interview. This represents a second opportunity for potential respondents to withdraw from the interview either because they refuse to take part or are unable, unavailable or not contactable for interview.

There is inevitably a degree of attrition between the household and random adult sections of the interview, especially where the person selected is not the same as the household respondent. This aspect of the survey has deteriorated since 1999/2000 when a random adult interview was achieved in 94% of households in which a household interview was completed. In 2007, random adult interviews were achieved at 92% of households where a household interview was completed. The participation rate varied from 86% in Dundee and Aberdeen to 97% in Orkney and 96% in Inverclyde. A participation rate of 92% means that while the overall household response rate was 65%, the random adult response rate was 60%.

The combination of lower response rates to the household survey and attrition at the random adult stage means that the random adult response rates varied from 49% in West Lothian and 51% in both Glasgow and Edinburgh to 78% in Orkney and 73% in Eilean Siar.

Table 4-5: Random adult ( RA) response rates, 2007

The random adult response rate for the main SHS interview was 59.9% and for the Culture and Sport module was 62.4%.

« Previous | Contents | Next »