The Scottish Government

« Previous | Contents | Next »

Listen

Evaluation of the SchoolsOutGlasgow.net project - Final Report

SECTION ONE: BACKGROUND TO THE RESEARCH

The remit

The Scottish Executive, as a contributor to the SchoolsOutGlasgow.net pilot project, commissioned an evaluation in May 2003, of the first stages of the project. The stated aims of the evaluation, as agreed between the commissioners (the Scottish Executive) and the Moray House School of Education researchers were:

• to assess the impact of the project on the socialisation/isolation of vulnerable and interrupted learners ¹
• to identify any differential impact of the project for pupils from the targeted groups and from girls and boys
• to assess the effectiveness of on-line learning for vulnerable and interrupted learners
• to identify factors which facilitate on-line learning and those which disrupt on-line learning.

The importance of SchoolsOutGlasgow.net

It has been recognised for some time that not all pupils' learning needs are met within current mainstream schools' provision. For many pupils a variety of life chances and circumstances have contributed to their exclusion from regular school attendance with consequent significant periods of interruptions to the continuity and coherence in their formal curriculum related learning (Dobson and Henthorne, 1999). Such interrupted learning has generally been associated with marked underachievement and a subsequent reduction in life chances, little access to job opportunities and a reduced likelihood of participation in life-long learning initiatives (Dobson, Henthorne & Lynas, 2000). For many such pupils participation in state education has ended before their legal leaving date and usually with no formal qualifications.

In a Scotland committed to equality of opportunity for all, the 5 National Priorities set within the framework of the Standards in Scotland's Schools Act, signal a governmental intention and a duty on Local Authorities and schools to make provision more relevant to pupils' needs. The SchoolsOutGlasgow.net project is one Local Authority's response to the recognition of the specific learning needs of several groupings of pupils who cannot continue to attend schools regularly. It can be seen as a move to ensure equality of opportunity in learning for some of those most vulnerable pupils within the present construct of school education and the content of the school curriculum (Scottish Executive, 2003)

Outline of the project

SchoolsOutGlasgow.net is an on-line learning project specifically designed to support a range of learners with significant interruptions to their attendance at school or for whom school based learning is not viable. It is a natural successor to the TOPILOT, FLEX (both were EU funded international projects in which SCET and Glasgow City Council (GCC) participated as partners) and the Ultralab notSchool.net project. An unusual aspect of that last project was the inclusion of the young people as 'researchers' within the learning process. An evaluation of Glasgow City Council's involvement of some 16 pupils in Notschool.net revealed that, for a variety of reasons, these pupils' learning needs were not being fully met. GCC considered that a more traditional approach to such a provision was worthy of trial, hence the format outlined for SchoolsOutGlasgow.net. The project has been specifically designed to draw on the lessons learned in these past projects in order to provide a more motivating, relevant and enabling learning experience for groupings of pupils in outwith school settings.

The project aimed to provide as supportive an environment for learning as possible, within the resources available to the pilot project. Traditional schooling had not been successful for the majority of the identified group. This was an attempt to apply emerging technology to the issue of out of school learning and to provide a more flexible curriculum that could be tailored to particular needs and interests. It aimed to explore the possibility of establishing an identity and ethos that developed a sense of community, not totally divorced from that of a school but in an on-line environment. In contrast to the notSchool project, the face-to-face element was envisaged as a key element of the project in this regard.

The salient features are

• the continuing use of ICT-based curriculum delivery
• the development of a learning centred community (loosely echoing the notion of the mainstream school community)
• dedicated tutor/teacher staff working on-line and face-to-face with individuals and groups to promote active learning and mutual support.

Learners are expected and supported by tutors to take increasing responsibility for their learning. While pupils engage individually with their ICT equipment and their tutors, there is also an encouragement to engage with their peers through the medium of on-line communication channels. Such novel forms of communication and interaction, much of it under the control of the learners themselves, provide a relatively stress free context for the learners and teachers and the freedom to develop new ways of working. Within this setting the traditional roles of the teacher and learner become blurred.

The target groups

The project's development of structured on-line support was targeted initially at four specific groups of learners:

• young people who are absent from school on a medium/long term basis through ill health
• young people who are alienated/school phobic and who are school refusers
• young people who are looked after by the local authority
• young people excluded from school.

Another group considered for inclusion within the pilot phase was the pregnant schoolgirls/new mothers: one such learner joined in the second session (another joined briefly before leaving). Both GCC and SEED are hopeful that if this pilot phase is seen to be successful then this approach could be developed to provide support to a greater variety of pupils who have significant interruptions to their school based learning, such as Gypsies and Travellers, refugees and asylum seekers.

Whilst the reasons for young people being unable to attend school can be categorised as above, it is important to acknowledge that the young people do not constitute discrete social groups. Nevertheless, the young people from all of these groups have experienced interrupted learning and have had unmet learning needs.

One specific group targeted by the SchoolsOutGlasgow.net project is the young people excluded from school, some of whom are also 'looked after and accommodated' by GCC. Official figures show that the numbers of exclusions from Scottish state schools dropped by 3% to 37, 442 in the period 2001/2002. However, the numbers of those whose names have been 'removed from the school register', in contrast, had risen by 3% to 332 during the same period. These statistics, and others, highlight the salience of developing support for these and other Scottish pupils who cannot, or do not, attend school regularly. Placed within the context of past alternative educational initiatives, this project could be seen to be representative of the commitment of policy makers and providers to meet their statutory duties to provide education for all pupils, which now includes some in settings outwith schools.

The project stakeholders

This evaluation of SchoolsOutGlasgow.net generated data obtained through qualitative interviews with representatives from the following stakeholder groups:

SEED representatives,

• to establish their particular aims and objectives in relation to the project, for example, its roll out potential for use in other local authorities

Glasgow City Council education department's representatives,

• to gain a sense of their particular aims and objectives for supporting pupils from the targeted groups living in a large urban setting (feasibility)
• to determine costs of this approach relative to other forms of support for learning outwith schools (viability)
• to gain descriptions of the processes involved in implementing the practical underpinning that makes possible the day-to-day experiences within the project

Learning and Teaching Scotland's representatives,

• to examine their particular contribution towards this project's development of on-line content and offline resources and support for tutors
• to evaluate curriculum content and pedagogical aspects within the software used and the teaching approaches adopted

Tutors working with pupils,

Pupils and their families,

• to explore their teaching remit, their training in the use of technology and on-line materials and their opportunities for providing feedback on the relevance, appropriateness and quality of the learning materials
• to encourage them to reflect upon their relational experiences with learners and their families to gain a sense of the benefits of and barriers in on-line learning
• to explore their experiences of learning in this alternative setting in order to gain a sense of the project's impact upon their attitudes towards themselves as learners
• to gain their reactions to learning in a safe and potentially intellectually challenging and rewarding new learning environment
• to assess the success of this mode of learning in helping them achieve their educational goals, for example as set out in their IEPs.

ICT technical support staff,

• to gain a sense of the everyday benefits and problems in using ICT with pupils in out of school settings, specifically with computers, printers, web cameras, technical connections between the GCC Intranet, email and the filtered Internet access provided.

Given the literature on gender differences in engaging with ICT-based activities, this aspect, too, was thought to be worthy of investigation. The intention was to discover in what ways the project anticipated and pre-empted any such bias before trying to find if there were any gender differences emerging. However, given the relatively small cohort it was not thought possible to arrive at any definitive conclusions.

A particularly sensitive aspect of the evaluation would be the contact with parents/carers. Yet, their contribution was thought to be significant in trying to contextualise the qualitative data regarding family and community attitudes and their relationships to formal education. Where interviews took place within the home there would also be sensitivities about the intrusion of strangers (the researchers), with no obvious commitment to their children or the project, into the families' personal 'space'.

The following aspects were to be given close investigation:

• the stated roles and remits of the staff, parents and carers
• the degree of clarity of purpose and shared vision within the participants
• the type and quality of interactions between learners and teachers; learners and learners; teachers and teachers; teachers and management
• the role of the learner's family, the relationship between family and tutor and family and management or technical staff
• the notion of a learning centred community (being more than a community of learners)
• the content, relevance and differentiation of the curriculum
• the nature of the behaviours expected and aimed for in the learners.
• the cost of this approach relative to other forms of outwith school learning support.

Time-scale

Due to the imminent leaving date of some of the pupils within the first cohort, the field research with that group and their families had to be completed in June, 2003. The remainder of the practical research work was scheduled to be completed by November, with the analysis and the drafting processes being undertaken in December 2003, January and February, 2004.

Methodology

Guided by the information provided in the tender documents, the research followed the Scottish Executive's arrangements and protocols for access to learners at school. Because of the nature of the project and the situation of the learners, the process of selecting pupils to interview relied heavily upon the sensitive guidance of those directly involved in day-to-day contact with the learners, acting as gatekeepers. In order to ensure that a breadth of learners' views was obtained, the provided contact list of names and addresses of learners was to include some learners from each of the five official groupings and a gender balance. The researchers have to accept that there may have been some unintended bias in the selection but, based on the data retrieved by this method, this would seem to be minimal.

A fair and effective evaluation is judged largely upon the range and sources of the data it selects or produces for analysis. Such an approach reduces the likelihood of being partisan. At the same time it provides a breadth of data and rigour of analysis to allow further debate to focus upon its findings. The main task then was to gather different kinds of data, in a range of ways that would allow for analytical comparisons between what was expected of the project and what it had been possible to achieve.

As stated in the evaluation proposal, "The spirit of ethical researching entails that a researcher takes all possible care to prevent harmful outcomes for the participants, including the researcher, as a direct result of knowingly and freely participating in research activities." The methodological implications of this statement were heightened in the context of a pilot project, which involved learners who could already be described as 'vulnerable'.

Central to the evaluation of the project and the participants' experiences within it, the following ethical principles were adopted to ensure that 'informed consent' was obtained throughout the evaluation process. These key principles, governing the negotiated research relations, were

• to establish rapport with all the participants
• to assure them that what they said was private (in the sense that what they said would not be attributed to them personally)
• to ensure that it was understood that the non-professionals could exercise their right to withdraw from the research
• to ensure individual's confidentiality in the writing up and dissemination of the research findings.

In practice, gaining informed consent involved providing participants with

• a clear description of the purpose of the evaluation
• a number of opportunities to meet in a mutually acceptable place and with sufficient time and space to carry out the interview
• information about the kinds of questions the evaluation needed to address
• a clear understanding that their own accounts of their experiences were important to the quality of the evaluation report and to the possible future of the project
• a clear understanding of what participation would entail, i.e., in the short term in time and energy and, in the longer term, in its impact upon their lives.

The young people, from 14-16 years of age, participating in the SchoolsOutGlasgow.net project, had not been able to attend school for a range of personal reasons. All participant learners had experienced significant interruptions to teaching and learning and in accessing a relevant and appropriate curriculum. In these situations, learners are recognised as likely to have low self-esteem and low expectations of themselves as learners and of achieving formal certification. Sensitivity in approaching them and in probing for information was a priority. It was essential that the evaluation process would not in any way introduce a negative impact on the progress of the project. Likewise the teaching staff too had to be treated with respect as they were very exposed in their current high profile positions, where the success or failure of the project could in some ways be attributed to their contribution.

In practice, the principle of ensuring confidentiality was most problematic for participants employed in some capacity within the project. Being a small number of local authority employees, in a high profile project, individuals could easily be identified. This problem was addressed by making sure that staff had an opportunity to amend or withdraw any statements made in their interview transcripts and to challenge any use made of their statements in drafting the report for the Scottish Executive. A further approach adopted was to use 's/he' throughout the quotes within this report in order to protect the individual's identity.

The research process

The initial approach to the project evaluation was through ICT, echoing the medium to be used within the project. Members from each stakeholder group were invited, through the medium of an introductory short video and/or letters, to participate in the research. We posted the video, prepared in house,

• to tutors and members of the operational management team to familiarise them with us and the evaluation aims
• to certain pupils and their parents/carers to explain what their participation in the evaluation entailed for them personally, and to ask them to discuss their participation with their tutor.

The original plans to email, or conduct video conferencing with the learners did not materialise due to the technological difficulties which constrained the use of ICT throughout the evaluation period (reported below). Tutors arranged a telephone interview with one learner and face-to-face interviews with the remaining learners. Interviews were arranged to suit individual's work schedules and family commitments and always in a place of their choosing.

Interviews with the professionals were arranged through telephone calls and emails. Each was asked questions that specifically related to his/her individual role and remit within the project's stated aims and objectives. Aspects of partnerships, finance, leadership, including staff management and professional development, were also investigated.

Taking into consideration the official aims for the project, several sets of questions were developed, some for written responses and others for use as a guide for face-to-face interviews (See Appendix A). These questions were designed to identify the extent to which the SchoolsOutGlasgow.net project was able to offer and promote the following outcomes:

• dedicated tutor/teacher staff working closely with individual learners
• pupil re-engagement with learning and the curriculum
• individual development through the opportunities afforded by the Internet
• the development and use of an ICT-based curriculum delivery,
• the development of virtual learning groups actively engaged via email, stickies and chat room
• the development of a learning centred community (loosely echoing the model of the mainstream school community
• the formation of face-to-face groups able to offer each other mutual support.
• on-going interest in learning/employment opportunities
• enrichment in access to learning for learners' families.

All interviews were tape recorded and transcribed verbatim, except for one, where notes were taken during the interview. The analyses of these transcripts and official documents, outlining the genesis and development of the SOG.net project, provided the main sources of evidence. Other relevant documentary and web based sources of information were provided by several key respondents - NEDD; L+TS; GCC; Edict.

Interviews with GCC and Edict staff were undertaken at an early stage of the evaluation process in order to gain an overview of the project as it was understood from a managerial and delivery perspective. These were useful in helping to map out the project time-line, the partners and partnerships, salient features of its progress in achieving its aims and the problems encountered, how they were dealt with and who could provide us with which information. These same respondents provided further written information and lengthy individual interviews in relation to their own specific roles and experiences.

Full co-operation was given throughout the evaluation process, with a great deal of apparently honest reporting on the successes and failures experienced throughout the lifetime of the project. When requesting further clarification respondents were always willing to provide further information, despite their often very busy work schedules.

It was clear from the very high levels of co-operation and the support provided to facilitate the research process that all the respondents were indeed stakeholders, with a personal interest in the progress of the project.

Footnote

1 The authors of this report take the stance that this is a form of labelling and should be avoided. Our preferred description is 'pupils with interrupted learning'.

« Previous | Contents | Next »