The Scottish Government

« Previous | Contents | Next »

Listen

3. RESEARCH DESIGN AND METHODS

3.1 SUMMARY OF RESEARCH DESIGN

There were two main types of data gathered during the evaluation:

• quantitative data on all attempts at test purchases
• the views of samples of all major categories of stakeholder concerning various aspects of the pilot scheme and its impact.

The following major categories of stakeholder were included in the evaluation:

• Volunteers involved in test purchasing exercises
• Parents/guardians of these volunteers
• Licensees representing all different types of on-sale and off-sale outlets, e.g., off licenses, hotels, supermarkets, public houses, grocers etc
• Police officers and Procurators Fiscal
• Licensing Board, community council and Fife Drug and Alcohol Action Team representatives.

3.2 DATA ON TEST PURCHASES

Data on all test purchases was collected in a consistent and uniform manner. The data sheet prepared by ScotCen and agreed with the Fife Police Force covered the:

• name and address, type of outlet, (supermarket, off sales, licensed grocers, pubs etc)
• date of visit, code for police area and code number for test purchaser
• gender and month/year of birth of young volunteer carrying out test purchase
• type of test purchase (first visit, revisit)
• whether visit resulted in attempt to purchase (if not, the reason why)
• whether proof of age information requested and outcome of attempt to purchase
• status of person from whom volunteer attempted to purchase
• status of person informed of test
• awareness of radio advertising in relation to pilot.

3.3 YOUNG VOLUNTEERS

Data on all potential young volunteers was collected in a consistent and uniform manner. Strict standards of confidentiality were maintained. Consent was sought from the young person and their parent or guardian for participation in the evaluation.

3.4 KEY STAKEHOLDER VIEWS

The three phases of data collection of stakeholder views were:

Phase 1: BASELINE: June to July 2006

This phase gauged the views of licensees, volunteer test purchasers, parents/guardians as far as possible BEFORE the first test purchasing visits took place.

Phase 2: IMPACT: November to December 2006

This phase examined the immediate impact of the test purchasing visits on the targeted licensees, and perceptions of the operation of the pilot of the police officers.

Phase 3: OUTCOME: April - May 2007

This phase looked at the longer term effect of test purchasing on targeted licensees, and the police officers' views of the operation of the whole pilot. Procurators Fiscal, volunteers and parents were interviewed. Members of the licensing board, community councils and Fife Drug and Alcohol Action Team ( DAAT) took part in focus groups.

3.4.1 Stakeholder views - baseline

At baseline the researchers were made aware of which premises the police intended to target and on which occasion, in order to inform their own sampling strategy for licensee interviews. Views were therefore sought before the first test purchase visits from:

• Licensees and managers of premises (100 interviews)
• Young volunteers who were trained as test purchasers (6 interviews)
• Parents/guardians of these children (6 interviews)

Fife Police started carrying out test purchases on the 30 ^th of June 2006. The research team had carried out the majority of the licensee interviews before the 30 ^th of June. ScotCen carried out random stratified sampling of the list of licensees provided by Fife Police in order to identify a sample of premises which covered all of the five main outlet types. After this a baseline structured interview schedule was prepared. These were short interviews, were conducted on a face-to-face basis and were with the license holders or managers of the respective outlet; junior staff members were not interviewed. The interviews covered knowledge and awareness of test purchasing, licensees' practice in relation to selling to young people, their training of staff members, their views about and practice in relation to proof of age schemes, their perceptions of test purchasing in principle, and their perceptions of its anticipated effect on their own and other licensees' practice.

In terms of young test purchase volunteers and their parents, the baseline interviews were semi-structured, addressed their expectations and knowledge of the initiative, any concerns they might have had and the volunteers' views of the training or information provided by the police, etc. These interviews took place face-to-face in the respondents' houses during the same visit. Six interviews were conducted with both volunteers and their parents or carers.

3.4.2 Stakeholder views - impact

Views were sought after the test purchasing visits had become established from:

• Licensees/managers of outlets who had already been targeted with test purchase visits (30 interviews, 20 with those that sold alcohol and 10 in which no sale resulted)
• Police Force representatives: Interviews were held with those that had either a supervisory or an operational role (n=4).
• Volunteers and parents interviewed at baseline gave their final interviews.
• Five new volunteers and parents were recruited and interviewed for the first time.

The interview schedule at impact was similar to the baseline schedule, but with an additional component to address test purchase visits. The police officers were interviewed using an in-depth schedule; these interviews were conducted on a face-to-face basis.

3.4.3 Stakeholder views - outcome

The following respondent types took part at outcome:

• Licensees (n=100; not the same as those who had been sampled at either baseline or impact; licensees who passed (n=58) and failed (n=42) the test were sampled)
• Volunteers and parents first interviewed at impact (n=5) gave final interviews
• Police Force representatives: Interviews were held with those that gave either their views at impact or were able to comment on the operational aspect of the pilot (n= 4)
• Procurator Fiscal (1 interview)
• Focus groups were convened (n=3) with Licensing Board representatives, community council members and the Fife Drug and Alcohol Action Team.

3.4.4 Stakeholder views - summary

The Table below shows the numbers of interviews/focus groups which were carried out during the evaluation. Therefore 283 interviews and 3 focus groups were carried out, the vast majority of the interviews were held with licensees.

* 6 volunteers and parents were interviewed at baseline and followed-up at Impact (Phase 2). Also, researchers recruited a new a sample of volunteers and parents at Phase 2 and followed this second group in Phase 3.
** Licensing Board representatives, Community councils and Fife DAAT took part in focus groups.

3.5 ANALYSIS

All open-ended questions with licensees, volunteers and parents were recorded by the interviewer taking full notes at the time of interview, before preparing a near-verbatim account as soon as possible after the completion of interview. This account was transcribed and entered on to Microsoft Access to aid analysis. All in-depth interviews and focus groups were digitally recorded, transcribed and qualitative thematic analysis carried out using the N6 analysis package. Structured data (test purchase data sheets and licensee interviews) were collated, processed and analysed using SPSS, a statistical analysis package. Statistical significance testing was carried out, where appropriate.

« Previous | Contents | Next »