Scottish Government

Evaluation of the Masterclass Initiative

CHAPTER 3 : STUDY APPROACH

The programme of research will be phased with final data being produced in June 2005. At this stage, initial data has been gathered on attitudes to Masterclass to date. This report provides information gathered by two complimentary research approaches.

3.1 QUALITATIVE RESEARCH

The qualitative research has involved face-to-face interviews with a sample of Masterclass participants across Scotland and Masterclass co-ordinators from five local authorities.

Masterclass participants from the following educational fields were interviewed:

• primary, secondary, nursery and special schools
• local authorities
• Teacher Education Institutes (TEIs)

When interviewing a participant at their work place, when possible we also spoke to one or two of their colleagues. The aim was to collect evidence of and changes in practice in the use of ICT that might have occurred as a result of Masterclass, to obtain additional insights into the impact of the initiative.

In total we conducted the following interviews:

We also spoke with five of the eight Masterclass on-line facilitators, four by telephone and one face-to-face. These were Masterclass participants who were asked to assist in facilitating the website by using it regularly and developing materials and activities for it.

3.2 QUANTITATIVE RESEARCH

This element involved surveys of participants and co-ordinators, with interviewers from George Street Research administering the questionnaire by telephone. An attempt was made to complete an interview with all Masterclass participants and in total 506 out of the eligible population of around 600 took part, representing a very high response rate of around 85%. This provides results that are accurate to +/- 1.8% at the 95% level of confidence.

Some 18 of those interviewed perform the dual role of being both participant and co-ordinator and 7 others who act as co-ordinators only were interviewed. Thus the total number of co-ordinators interviewed was 25 (out of a possible population of 32). Results for co-ordinators are accurate to within +/- 9.3% at the 95% level of confidence.

Different versions of the questionnaires were developed to make them suitable for both participants and co-ordinators. Copies of these questionnaires have been provided separately to the Scottish Executive. Where possible questions were kept consistent for both groups and this is reflected in the reporting of the information which presents the data from both audiences in tandem.

The questionnaire was piloted with 20 participants and the main phase of interviewing then commenced and ran over the period 11 May to 10 June 2004.

To facilitate the second phase of the research next year, all those taking part in our survey were asked if they would be willing to be recontacted. Almost all, (95%), agreed.

Table 3.1 provides details of the structure of the sample of individuals interviewed across this phase of research. As we interviewed such a high proportion of all Masterclass participants, this provides information on some of the characteristics of those chosen to take part in Masterclass. Key points are as follows:

• Most teaching staff are from the primary (44%) and secondary education (49%) sectors
• The majority of participants are in promoted posts (46%) or are teachers (31%)
• Most participants are aged 45-54 years (47%) or are aged 35-44 years (23%)
• School based participants are drawn from schools of all sizes. However, most are in schools of over 1000 pupils (25% - essentially secondary schools) or schools of 100-300 pupils (28%, with three quarters being primary schools)
• Although reported later, it is worth noting that 61% of participants regard themselves as being 'very ICT literate' prior to being involved in Masterclass.

Table 3.1

Sample profile

Local authority co-ordinators are the group most likely to have a complete understanding of who participated in Masterclass. The survey interviewed 25 of the 32 co-ordinators and each was asked to detail how many staff in the authority had taken part in Masterclass and to provide a breakdown of who was selected.

• The average number of Masterclassers within these 25 local authorities is 22
• This ranges from 4 local authorities with 15 or fewer Masterclassers to 3 with more than 30
• 67% of these Masterclassers are part of the first intake
• 5 local authority co-ordinators are unable to break their initial intakes of Masterclassers down by workplace. In the following calculations, they have been assumed to have the average number of Masterclassers in each category
• 19 of the 25 local authority all co-ordinators indicate that secondary and primary school staff became Masterclassers in the first intake
• Secondary school staff account for 36% of Masterclassers and primary school staff account for 46% of the first intake.
• Only 4 co-ordinators report that some nursery school staff were involved and overall this accounts for just 2% of the first intake
• Only 4 co-ordinators report having staff from special needs schools, accounting for 1% of Masterclassers in the first intake
• 5 local authorities did not include any education authority staff in their initial Masterclass allocation. These individuals account for 13% of the total.
• 1 co-ordinator included a university lecturer in the first intake. Librarians were included by 9 local authorities in the first intake