The Scottish Government

« Previous | Contents | Next »

Listen

PUBLIC ATTITUDES TO THE ENVIRONMENT IN SCOTLAND - TECHNICAL REPORT

Chapter 4 - FIELDWORK ISSUES

4.1 BRIEFING

Prior to the main fieldwork stage, all interviewers who were due to conduct the interviews were briefed in person at a day-long session. One briefing session was conducted on 30 ^th January 2002 in East Kilbride, the other was conducted on 31 ^st January 2002 in Aberdeen. The briefing sessions covered the following :

• Introduction and explanation of the survey

• The interviewer briefing pack

• Sampling

• Questionnaire A

• Test interview using Questionnaire A

• Questionnaire B

• Test interview using Questionnaire B

• Survey administration

Executive staff of George Street Research hosted the briefing sessions. In addition, the briefings were attended by members of the Scottish Executive project team who addressed the field interviewers on the aims and objectives of the research.

4.2 QUALITY CONTROL AND SUPERVISION

On-going checking was implemented on the quality of the interviews completed and to ensure that the sampling procedure was being properly followed. This was in addition to accompaniment in the field by supervisors as part of the regular programme of quality checking and accompaniment implemented over George Street Research field interviewers.

When returned to the office, the Sample Issue Sheets (SIS) were manually inspected first of all to ensure that they were fully completed, that a minimum of 5 calls had been made at each property (according to the specified criteria set for addresses which were valid and where no interview had occurred) and that the respondent was correctly selected. The outcome data from each SIS was entered into a survey database which tracked the outcomes for each address. This was later used to identify points where addresses could be reissued to a different interviewer and has been used to produce detailed outcomes for the sample (detailed later under response rates).

Manual checks were also instigated on completed questionnaires to ensure that they were fully completed and accurate. Where mistakes occurred, information was passed to the interviewers concerned as soon as possible. To correct missing or inconsistent information, the interviewer recontacted the household, or if the query was quite straightforward, this was handled by telephone using office based staff.

A 100% manual check was implemented on the first batches of questionnaires returned by interviewers. Thereafter, checks were made as part of the on-going checking and coding of the questionnaires.

Back-checking (where staff contacted the respondent and confirmed that the interview had been properly carried out and checked the answers recorded on an agreed number of questions) was conducted by telephone. Some 10% of the interviews completed were back-checked by telephone.

Throughout the course of the fieldwork period regular updates were provided to the Scottish Executive detailing, for each of the 10 weeks during which sample points were allocated:

• The number of points allocated

• The number of points completed

• The number of valid addresses issued

• The number of addresses used and

• The number of interviews achieved.

In addition, profiles were run on the achieved interviews on a regular basis and provided to the project managers showing an analysis of interviews completed:

• By local authority area

• By Mosaic code and

• By rural/urban indicator.

The target number of interviews in each of these sub groups was also depicted showing how well the interviewing was performing in terms of achieving the expected pattern of response. As the fieldwork progressed, this information helped to identify which areas and types of address to prioritise for reissues.

4.3 REISSUES

Addresses were reissued where it appeared that the level of response was lower than desired.

Initially any point returned as completed and where no more than 2 addresses had been converted to an interview was reissued to another interviewer. As many 'still valid' addresses as possible were reissued, including many of those identified in the first visits made to an address as being invalid, to check that this was so.

Where further improvements to response rates were seen as being achievable, a further series of points were reissued, with the selection made on the basis of points where 3 up to 8 addresses remained as valid.

Throughout the monitoring process, the main gap was in terms of converting addresses in specific local authority areas. The response rates by rural/urban indicator and by Mosaic code were less variable. This was an influence on the final stage of reissues, when monitoring revealed that there were some locations (especially in the 4 primary city local authority areas), where there was still an interview shortfall. In these locations all valid addresses across each city area were reissued in an attempt to increase response rates.

Throughout the process, 126 points had addresses reissued to interviewers.

4.4 RESPONSE RATES

All the fieldwork was conducted between the 4 ^th February 2002 and 26 ^th June 2002. The months of May and June were used essentially for reissued points.

Interviews with 4,119 people aged 16 years and over, across 800 sample points on the Scottish mainland and islands were completed. The following table displays the breakdown of interview outcome codes for the survey.

Thus, the overall response rate achieved compares with the response level of 64.7% achieved for the Scottish Household Survey, 1997. The Scottish Crime Survey, 2000 achieved a response level of 70.5%.

Comparison with these other surveys also reveals that:

• The percentage of invalid addresses was broadly similar for the SCS and the EAS.

• Level of contacts with households differed. The proportion of households contacted, but who refused to take part in the studies were:

• However, when these figures are added to those relating to households that were not contacted across the calls made at each address, the overall figures are more similar and the difference may just be down to how interviewers record these. For example, one interviewer might deem an address where a household member is at home but who refuses to answer the door as a non-contact while other might take the resident's decision not to answer the door as a refusal.

The proportion of households where no contact was made is shown in the first column of the following table, with the second column showing the combined household/no contact figures.

The level of response in the EAS 2002 varied across each of the rural/urban areas and appendix 4 gives full details of the outcome codes for each of the rural/urban categories. As would be expected the number of invalid addresses varied across each category, ranging from 3.5% in the 'other urban' category to 19.6% in the 'remote rural' category. In particular, high proportions of empty properties (6.6%) and second homes (9.1%) were found within the sample for the 'remote rural' areas. In 'accessible rural' areas, where the level of invalid addresses was 8.2%, the incidence of not found addresses (3.6%) was more than twice the level found across the sample as a whole.

The availability of substitute addresses made it possible to replace most of these invalid addresses in all areas with the exception of in the 'remote rural' areas, where the active sample size was reduced by 5.4%.

The extent to which contact was made with a member of each household in the active sample varied across the rural/urban categories as well, ranging from 69.4% in the 'primary cities' to 82.4% in 'small remote towns'. In the more urban areas (primary cities and other urban), lack of contact with anyone living at the address was the key issue, affecting around 1 in 6 of the addresses in each area. Refusal rates were also slightly higher in urban areas (9.4%) than rural areas (7.1%).

The overall level of response to the EAS 2002 was 65.2%. Table 4.2 shows the response rates achieved for each of the rural/urban areas. The table also indicates the extent to which contact was made with individuals at each address in the active sample (as discussed above). There is a relatively close match between the final response rate and the contact rate.

Steps taken to maximise response rates

Calls were to be made to each address at different times of the day and on different days of the week in order to obtain a contact interview. Interviewers were instructed to plan their initial approaches for interview for weekends and evenings to maximise the chances of achieving a contact interview. Interviewers were instructed to make:

• at least 5 calls to each address;

• at least one call during a weekend;

• at least one call during a weekday evening;

• at least one call during the day on a weekday.

Interviewers were also instructed to leave a 'while you were out letter' if nobody was at home on the first visit to an address. A telephone number was provided on the letter so that potential respondents could call and arrange interviews over the telephone.

The other steps used to maximise the survey response rates was the reissue of addresses as detailed earlier.

4.5 POPULATION PROFILE, SAMPLE DESIGN AND ACHIEVED PROFILE DATA

The following tables display data on the overall unweighted pattern of achieved interviews across a range of dimensions and in both sample A and sample B. Of the 4,119 questionnaires completed, 1,989 interviews were completed on questionnaire A and 2,130 interviews were completed on questionnaire B. As the total number of interviews achieved from sample A and sample B differs, the data analysed in percentage terms provides a more meaningful comparison of outcomes.

As this analysis indicates, the achieved sample profile very closely mirrors the designed sample profile.

Rural-urban split

As a proportion, interviews within the 'primary cities and other urban' categories were slightly under-represented compared with the designed profile. Those from the rural categories were slightly over-represented.

Local authority groupings

With respect to the split by local authority groupings, the figures again show that the achieved sample closely matched the designed sample, with the exception of shortfalls in Edinburgh and Aberdeen.

Mosaic code

Only slight variations were evident between the designed sample and achieved sample across the Mosaic code classifications.

« Previous | Contents | Next »