The Scottish Government

« Previous | Contents | Next »

Listen

3. RESEARCH DESIGN AND METHODS

3.1 SUMMARY OF RESEARCH DESIGN

The following research design was utilised, although there may yet be minor modifications to the outcome phase.

There were two main types of data gathered during the evaluation:

• quantitative data on all attempts at test purchases
• the views of samples of all major categories of stakeholder concerning various aspects of the pilot scheme and its impact.

The following major categories of stakeholder were or will be included in the evaluation:

• Volunteers involved in test purchasing exercises
• Parents/guardians of these volunteers
• Licensees who are targeted for test purchasing (different types of outlet, e.g. off licenses, hotels, supermarkets, public houses, restaurants etc)
• Licensees in the same areas who are not targeted for test purchasing. It is likely that every outlet will be targeted within Fife, therefore it is imperative that the researchers work with the police in order to organise a wave of interviews before the actual test purchases at baseline and (if relevant) outcome. It is possible that at outcome nearly every premise will have been targeted.
• Police officers and Procurators Fiscal.
• Licensing Board representatives, and members of community councils and the Fife Alcohol and Drug Action Team.

The three main phases of the work will be:

• Baseline: June - July 2006
• Impact: November 2006 - December 2006
• Outcome: Probably April - May 2007.

Fife Police intended to carry out c900 test purchases ( i.e. in every possible outlet where it is deemed safe to do so) over the course of the whole pilot exercise.

3.2 DATA ON TEST PURCHASES

Data on all test purchases was collected in a consistent and uniform manner. At baseline the researchers were made aware of which premises the police intend to target and on which occasion, in order to inform their own sampling strategy for licensee interviews.

The data sheet prepared by ScotCen and agreed with the Fife Police Force covered the:

• type of outlet (supermarket, off sales, licensed grocers, pubs etc)
• name and address of retail business
• date of visit
• code for police area and code number for test purchaser
• gender of young volunteer carrying out test purchase
• month and year of birth of young volunteer
• Type of test purchase (first visit, revisit)
• whether visit resulted in attempt to purchase (if not, the reason why)
• whether proof of age information requested
• outcome of attempt to purchase (purchase made, purchase refused, attempt abandoned, other)
• status of person informed (owner, manager, assistant etc)
• status of person from whom child attempted to purchase (owner, manager, assistant etc.)
• awareness of radio advertising in relation to pilot

As a result the following information was gathered and presented in this report:
- the total number of visits to premises to make test purchase attempts
- the proportion of visits abandoned before any purchase attempt, and
reasons why
- the proportion of purchase attempts (first visits and revisits) that were successful, refused, or resulted in other outcomes
- the proportion of purchase attempts at which proof of age was requested.

3.3 YOUNG VOLUNTEERS

Data on all potential young volunteers was and will be collected in a consistent and uniform manner. It will be important to maintain strict standards of confidentiality. Also, consent will be sought from the young person and parent/guardian (if the volunteer is aged under 16 years) for participation in the evaluation.

3.4 KEY STAKEHOLDER VIEWS

The three phases of data collection of stakeholder views will be:

Phase 1: BASELINE: June to July 2006
The purpose of this phase was to gauge the views of licensees, volunteer test purchasers, parents/guardians etc as far as possible BEFORE the first test purchasing visits take place.

Phase 2: IMPACT: November to December 2006
The purpose of this phase was to gauge the immediate impact of the test purchasing visits on the licensees targeted by it, and a more general overview of the operation of the scheme from police officers.

Phase 3: OUTCOME: Approximately April - May 2007
The purpose of this phase will be to gauge the longer term effect of the test purchasing both on licensees who have been targeted by test purchasing and on others in the area who have still to be visited (if feasible at this stage), and a more general overview of the operation of the scheme from police officers with the benefit of longer experience. Procurators Fiscal, young volunteers and their parents and carers will also be interviewed, and members of the licensing board, community council representatives and members of the Fife Alcohol and Drug Action team will be invited to take part in focus groups.

3.4.1 Stakeholder views - baseline

Views were sought as far as possible before the first test purchase was attempted from:

• Licensees and managers of premises (100 interviews)
• Young volunteers who were trained as test purchasers (6 interviews)
• Parents/guardians of these children (6 interviews)

Fife Police started carrying out test purchases on the 30 ^th of June 2006. However, it was advisable to carry out as many of the baseline interviews with licensees before test purchasing actually started. The research team had carried out the majority of the licensee interviews before the 30 ^th of June. After liaison with Fife Police Force the sample of licensees selected for baseline interview were not to be targeted by police officers during the initial test purchase, therefore all of the licensees interviewed by the research team after June 30 ^th had not been visited for test purchasing at that point.

ScotCen carried out a random stratified sample of the list of licensees provided by Fife police in order to identify a sample of premises which covered all of the five outlet types. After this a baseline structured interview schedule was prepared. These were short interviews, were conducted on a face-to-face basis and were with the license holders or managers of the respective outlet; junior staff members were not interviewed. The interviews covered knowledge and awareness of test purchasing, licensees' practice in relation to selling to young people, their training of staff members, their views about and practice in relation to proof of age schemes, their perceptions of test purchasing in principle, and their perceptions of its anticipated effect on their own practice and those of other license holders generally, and their perceptions of risk re being targeted by such an intervention.

In terms of young test purchase volunteers and their parents, the baseline interviews were semi-structured, addressed their expectation and knowledge of the initiative, any concerns they might have had, the volunteers' views of the training or information provided by the police, etc. These interviews took place on a face-to-face basis, in the respondents' houses during the same visit. Six interviews were conducted with both volunteers and their parents and carers. No volunteer had been involved in test purchase visits at the time of the baseline interview.

3.4.2 Stakeholder views - impact

Views were sought after the test purchasing visits had become established from:

• Licensees/managers of outlets who had already been targeted with test purchase visits (30 interviews, 20 with those that sold alcohol and 10 in which no sale resulted)
• Police Force representatives: Interviews were held with those that had either a supervisory or an operational role (n=4).
• Volunteers and parents interviewed at baseline. (Also, five new volunteers and parents were recruited and interviewed; this group will be followed-up at outcome.)

The interview schedule at impact was similar to the baseline schedule, but with an additional component to address test purchase visits. The police officers were interviewed using an in-depth schedule; these interviews were conducted on a face-to-face basis.

3.4.3 Stakeholder views - outcome

Views will be sought towards the end of the pilot from:

• Licensees (n=100; not the same as those who have been sampled at either the baseline or the impact phases)
• Volunteers and parents first interviewed at impact (n=5)
• Police Force representatives: Interviews will be held with those that gave their views at Impact (n= 4).
• Procurator Fiscal (1 interview)
• Focus groups (n=3) with Licensing Board representatives, community council members and the Fife Alcohol and Drug Action Team.

3.4.4 Stakeholder views - summary

The Table below shows the numbers of interviews/focus groups which were or will be carried out during the evaluation.

*The exact number of licensee interviews with targeted and non-targeted licensees at baseline and outcome will depend upon the approach agreed with Fife Police Force.
** 6 volunteers and parents/guardians were interviewed at baseline and followed-up at Impact (Phase 2). Also, researchers recruited a new a sample of volunteers and parents at Phase 2 and followed this second group in Phase 3.
*** Licensing Board representatives, Community councils and Alcohol and Drug Action Team will participate in focus group research at the outcome phase.

Therefore about 286 interviews will be carried out, the vast majority of which will be held with licensees.

3.5 CONSENT AND INTERVIEWER SCREENING

In order that young volunteers participated in the evaluation of test purchase pilots ScotCen researchers sought their consent directly as they were over the age of 16 years of age at the time of contact. We negotiated with Fife Police as to how to approach the volunteers and parents in the first instance. All ScotCen interviewers and research staff undergo regular Disclosure Scotland checks.

3.6 ANALYSIS

All open-ended questions with licensees were recorded by the interviewer taking full notes at the time of interview, before preparing a near-verbatim account as soon as possible after the interview had been completed. This account was transcribed and entered on to Microsoft Access to aid analysis. A similar approach was utilised to analyse the responses of volunteers and their parents and guardians.

In-depth interviews with police officers and a procurator fiscal were or will be taped, transcribed and qualitative thematic analysis carried out using N6 or Framework. Focus groups will be taped, transcribed and entered on to N6 or Framework to allow thematic analysis to be carried out.

Data collected by police officers during visits was and will be collated, processed and analysed using SPSS. Statistical significance testing will be carried out, where appropriate.

« Previous | Contents | Next »