The Scottish Government

« Previous | Contents | Next »

Listen

Appendix 1 - Estimating e

Response rates in telephone surveys are complicated by the fact that a large proportion of the telephone numbers generated in the sampling process are not valid numbers, being either business numbers, fax and computer lines or non-working lines. A number of measures can be taken to eliminate invalid numbers such as pre-dialling the numbers to test the validity of lines (known as 'pinging'), matching the generated numbers against published directories and dialling for interview. These identify the majority of non-working and ineligible lines. However, a proportion of numbers remain that ring when dialled but are never answered or where the line is constantly engaged. These lines may fall into one of five categories:

• they are non-working lines but dialling does not result in the characteristic 'number unobtainable' tone
• they are working lines but are non-residential, including telephone boxes, business lines, fax machines etc
• they are working residential lines but the property is between occupants or not a main residence and therefore regularly unoccupied
• they are working residential lines but are regularly connected to the internet or used for calls for long periods or the occupants are always out
• they are working residential lines but are behind a call screening device such as caller ID.

The additional potential category of the number representing a working residential line but the dwelling containing no eligible respondents does not apply in this case since all households would be eligible to participate in the SCVS.

Understanding the composition of these lines is important in determining the response rate for the survey. As the American Association for Public Opinion Research argues, "In estimating e, one must be guided by the best available scientific information on what share eligible cases make up among the unknown cases and one must not select a proportion in order to boost the response rate." ³²

This section attempts to estimate the proportion of eligibility unknown lines from the June-September sample of the SCVS that should be considered eligible for calculating the survey response rates.

The unadjusted response rate for the June-September sample was 38.4% although a high proportion of the sample would fall into the category of unknown eligibility. Depending on the extent to which unknown eligibility lines are actually eligible, the response rate for the survey might be up to 54%.

The key to estimating the proportion of unknown eligibility that should be considered eligible is the proportion of the sample registered with the Telephone Preference Service. The logic of the estimation is as follows.

• The proportion of all residential lines registered with TPS is known (at April 2005) to be 42%.
• Given the sampling method used for the SCVS, the sample, after deadwood has been removed, should be a random subset of all residential lines in Scotland and should therefore have a TPS registration rate of 42%. We know that:
• the sample drawn for January 2005 did not have TPS numbers screened out and TPS listed numbers made up 19% of the initial sample
• after the January 2005 sample had been fully worked, 42% of the sample was identified as deadwood.
• This implies that TPS numbers comprised 33% of the sample that was not identified as deadwood ( i.e..
• If the non-deadwood sample is a random subset of all residential lines, the only reason for the proportion of TPS lines to be lower than the known national proportion is that the non-deadwood sample still contains some ineligible lines. The real response rate is given by multiplying the apparent response rate (38%) by a factor equal to that required to bring the non-deadwood TPS rate into line with the known TPS rate.

• This gives a 'real' response rate of 48%

This can be expressed more generally in the following equation, based on the AAPOR terminology.

Assuming partial interviews are counted in the total, the minimum response rate is RR2

where I = Complete interviews, P = Partial interviews, R = Refusals, O = Other eligible households, UH = Unknown if an eligible household, UO = Other unknown eligibility

This is the same as saying that the response rate is the number of interviews (I = I+P) as a proportion of the potentially eligible sample, which is the issued sample (S) minus known ineligible sample (i).

Using the TPS registration rates we can express the relationship between RR2 and RR3 in terms of three values - the TPS rate in the population (Tp), the TPS rate in the issued sample (Ti) and the proportion of the issued sample that is identified as ineligible (i).

By this method, there is no need to identify e in order to calculate the response rate. However, since response rates are generally reported alongside the proportion of the sample that falls into different categories of eligibility and ineligibility, it is useful to calculate e in order to reapportion sample between the potentially eligible and ineligible categories.

If we call I + P + R + O the known eligibles (Ek) and UH + UO the unknown eligible sample (Eo) we can say that the response rate RR3 is

therefore

« Previous | Contents | Next »