The Scottish Government

Listen

INSIGHT 14
EVALUATION OF PERSONALISED LAPTOP PROVISION IN SCHOOLS

Mary Simpson, University of Edinburgh
Fran Payne, University of Aberdeen

This document is also available in pdf format (120k)

Welcome to Insight

Insight is a publication of the research group within Information, Analysis and Communication Division, which is responsible for providing analytical services within the Scottish Executive Education Department (SEED). Their work is part of a multidisciplinary unit (consisting of researchers, economists and statistics staff) and the staff undertakes and funds economic analysis and social research in the fields of: school education; children, young people and social work: architecture; and tourism, culture and sport.

The Scottish Executive is committed to the use of sound evidence in the development of policy and practice as well as in the evaluation of policy and its implementation. We therefore want to disseminate the results of research that SEED has undertaken and funded, in a manner that is accessible, interesting and attractive.

Insight aims to present the essence of research projects in a format that will be useful and informative for practitioners, policy makers, parents, academics, and anyone else who has an interest in economic and social research in these areas.

The views expressed in this Insight are those of the author(s) and do not necessarily reflect those of the Scottish Executive or any other organisation(s) by whom the author(s) is or are employed. Copyright © November 2004, Scottish Executive Education Department

ISSN 1478-6788 (Print)
ISSN 1478-6796 (Online)

Insight may be photocopied for use within your own institution.

A limited number of additional copies can be obtained from the Dissemination Officer, Information, Analysis and Communication Division, Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ (telephone 0131-244-0316). Copies of Insight and our other publications can be downloaded from our website: www.scotland.gov.uk/insight/

The Vision of Personal Laptop Provision for Teachers and Pupils

The laptop initiative is very much a shift in thinking, away from devices associated with particular subjects or places to devices being associated with particular learners or groups of learners. It's about moving away from fixed context, moving away from positions of computers determining the pedagogy to the teacher being able to say, "I want you to work in pairs, or groups of three. I want you to go to the library." So it gave back a lot of control to the teacher, and it also, in theory, and I'm sure this will happen over time, will give control to the learner, and promote home/school links, and out-of-school learning, and family learning. And these are all things that we consider to be important. (Authority Project Planner)

The Scottish Executive Education Department (SEED) are currently funding a range of projects which form part of the overall development of the use of ICT for a range of purposes in educational settings. The overall remit of the SEED programme is to support innovation and to enable access, connectivity, training and content to be developed and effectively applied in the delivery of educational services. The Personal Laptop Provision Project initiated in 2000 was one of these projects.

The Personal Laptop Provision in Schools Project was an ambitious and innovative Local Authority enterprise, aiming as it did for radical change in learning and teaching relationships, mediated by technology, including wireless networking. Laptops and digital projectors were provided to give personal ICT access to selected teachers within one primary and one secondary school within a Scottish City Authority, as a pilot for possible extended laptop provision. The laptops provided the teachers with the means to access and use computers in their daily teaching and to be flexible in their presentation of materials to pupils. In each of these two schools, pupils in selected classes were also provided with laptops: 60 pupils (two S1 classes) in the secondary school; 21 pupils, the P5 class, in the primary). These were intended to be used for a range of learning related activities: accessing a wide range of information, communication with other pupils and with teachers, and the extension of access to learning resources to their homes. We consider many of the findings to be generalisable to other primary and secondary settings.

The Aims of the Evaluation Programme

The evaluation research programme (2002-04) had four aims:

• The evaluation of the initial and ongoing planning processes;
• Monitoring of the implementation of the project;
• The evaluation of the impact of the technology in the classrooms;
• The evaluation of the project's impact on the attainment and learning of the pupils.

The Research Methods

• Interviewing of the Authority Planners of the Laptop Project and the Headteachers;
• Scrutiny of key documentation on planning and implementation;
• Attendance at committee and school staff meetings;
• Interviews at an early and late stage in the evaluation of a sample of the school staff involved;
• Classroom observations and discussions with pupils;
• Semi-structured interviews with a sample of pupils involved in the project;
• Informal discussions and interviews with parents;
• A questionnaire to all pupils and parents involved in the project.

Details of the samples and methodology can be found in Simpson and Payne (2004).

The Findings from the Primary School

The primary school was quite advanced in terms of staff competence in ICT skills and commitment to the use of ICT in classrooms. The school had a catchment area with a low socio economic profile, and was regarded as typical of such schools in the City.

The Experiences of the Primary School Staff

Despite the initial technical difficulties encountered with the laptops, the two laptop class teachers (P5/6 and P7) who had good ICT skills prior to the beginning of the Project were overwhelmingly positive about their involvement.

One of the most important factors which contributed to their experience of success was the collaborative efforts of all the staff in the early planning stages, the fact that all teachers in the school were aware of the aims of the Project, and the full integration of the Project into the school development plan. The school had a strongly affirmed child-centred approach to teaching and learning.

There had been a measured approach to the introduction of the laptops and matters had been more straightforward than in the secondary since it only involved one fairly static class of 21 pupils. Considerable effort was taken to ensure that the children had been given time to 'play' with the laptops and so become confident users before they were allowed to take the laptops home.

The Technical Problems

The technical problems that arose with laptops meant that the laptop teacher was often a first line engineer and relied on a knowledgeable family member's expertise to assist her.

A variety of staff development strategies were encountered, but the most effective appeared to be their own in-school model of mutual support and peer tutoring.

The Impact in the Classroom

The main impact in the classroom was identified as increased pupil motivation, increased availability of information to pupils, the teachers' regular use of the laptops, and a projector and interactive whiteboard for whole class work. The machines were used regularly within the normal curriculum, but they allowed extended and creative enterprises - such as the production of an animated video - to take place.

Their final comments in the form of advice to other schools included:

a) Ensuring the suitability of the technology and compatibility with existing systems;
b) Resourcing the production of a bank of ICT linked curricular resources for different areas of the curriculum;
c) Ensuring support for teachers to develop technical skills to deal with problems as they arise in the classroom as well as confidence in key ICT skills;
d) Establishing an ethos in the school that encourages collaborative action and responds to challenges.

The Experiences of the Primary School Pupils

Overall, the pupils' initial excitement and positive expectations were maintained throughout the two years they had been using the laptops. The had enjoyed the use of the laptops and they felt it made their learning more fun, more enjoyable, their work neater and their presentations better.

Although classroom observations identified a range of skills among the laptop pupils, by the end of the two years a high proportion of the laptop users (88%) rated themselves as 'real experts' in the use of computers, in comparison to 67% of the non participant pupils in their year group.

Playing games and communicating (chat and e-mail) were the two most frequent activities primary laptop pupils engaged in while using the internet at home. Researching for school work and 'getting information' generally was next most frequent.

The support given to the pupils in the initial stages of the project included input from the Authority ICT Staff Tutor delegated to support the school, who gave introductory sessions to show the pupils how to use the software. The ongoing support, highly rated by the pupils, was provided largely by the classroom teacher in P5 and P6 who put in a considerable amount of her own time to help solve technical matters although this was not her responsibility.

In P5 the pupils had exclusive use of a laptop. However, in P6 and to a much greater extent in P7 they shared the laptops with pupils in other classes. There was a range of pupil views on the laptops having to be shared with other pupils in P7, but most accepted that it was fair to share them with others when they were not being used. Despite sharing them with other classes, the pupils still retained a sense of ownership, personalising them as well as taking responsibility for them. By P7, six percent of pupils indicated that they used the laptop every day; 77% three or four times per week; and 18% once or twice per week. Pupils reported that the level of use varied from week to week depending on the activity but that frequency use was much lower than when they were in P5 and P6.

When pupils were observed researching information for project work, it appeared that the internet was not always the best resource for pupils to use. With the encouragement of the teacher, pupils came to realise that they could find relevant information more quickly from books which were much more appropriate to their level of understanding.

All the primary pupils said the use of laptops enabled the application of their own imagination. This is a reflection of one of the key differences between the use in primary and that in secondary: primary pupils had far greater opportunities to explore the creative and personal possibilities the laptop had to offer.

[Interviewer: What was the best thing about the laptops?]

We just got to explore when we felt like it. (Pupil P6)

The main problems identified by the pupils, both in the interviews and through an open question in the questionnaire, were technical ones.

The pupils were all enthusiastic about their laptops, and despite the various technical problems they had suffered, the use of the personal technology had been a positive experience from which they felt they had gained enormously. They clearly recognised the value of what they had gained from the experience.

Because we have been on the laptops I know more things to do on my own computer. I know more websites for school stuff and things. (Pupil P7)

The Perceptions of the Primary School Parents

I think the laptop project was excellent and I cannot find any fault. (Parent)

The overall response of parents to the project was very positive for most of the respondents. They perceived it as an excellent initiative, which, although there had been some disappointment with respect to the frequency of home use and technical problems, had greatly benefited their child. These benefits were identified as increased confidence generally as learners, and greater confidence and competence in the use of computers.

The Findings from the Secondary School

Although regarded by some as rather traditional, the secondary school was accepted as suitable for the Project since it was not considered to be significantly different demographically from other schools in the city. The school management saw involvement in the Project as an opportunity to advance the practices and skills of the staff in the use of ICT in classrooms, and reassured parents that their children would not be disadvantaged by their participation.

The Experiences of the Secondary School Staff

I enjoyed being part of the project, I enjoyed being part of the steering group and even though some people might say this project was not a success, I think in a lot of ways this project was a success and yes, we had a lot of teething problems and yes, there were a lot of difficulties but we learn from them. But if we were to do it again we would do it differently. (Teacher 2)

Laptops were given to two S1 classes and their teachers were invited to opt into the Project. The 17 who did so had a range of starting skills, past experience, and a number of ideas on how they might or could use the technology within their subject.

Their initial overall aims included:

• improved motivation of pupils;
• easier pupil access to information on the internet;
• improved pupil computer skills;
• and a range of subject specific applications.

From their responses in the final interviews and other data, it was clear that these aspirations had been realised.

The downside of their involvement was their perception of the unrealistic aims set out for them by the Authority and the frustrations they felt at the many technical problems associated with the wireless connectivity and the pupils' use of the laptops for music and games - thus eating up memory space. They felt that the Project would have been greatly improved if there had been a greater degree of early consultation with the school staff, a much higher level of school based technical support, and mechanisms for giving them curricular support in their developments.

The Impact in the Classrooms

We observed a range of teaching styles and levels of ICT skills and experience at the outset of the project. The introduction of the technology allowed an extension of and an increased flexibility in the teaching and learning experiences of most of the teachers and learners involved in the project. However, our classroom observations indicated that the typical teaching approaches of the individual teachers were not basically changed by the introduction of the technology and the learning experiences of pupils were centred on the acquisition of curriculum content.

We classified the styles of the 16 observed ICT teacher users into four categories:

• The potential innovators: confident ICT users who deployed ICT and teaching methods which could increasingly incorporate giving pupils increasing charge of areas of the curriculum, the level at which they worked, and the extent to which they initiated and self-managed their learning off the school premises (N = 3);
• The traditional users: although confident ICT users, they had deployed the technology within their very traditional teaching styles mainly for transmission, recording and presentation of content (N = 7);
• The curriculum centred users: highly experienced in ICT or computing teaching, these teachers used their laptops to transform their courses onto on-line lock step presentations for whole class use in computer suites (N =2);
• The novice users: a few teachers had little experience with computers and although they advanced their ICT skills, their progress in confident classroom use was slow (N =4).

The Subject Barriers

The typical model of professional development for secondary teachers is based on the autonomous development of those skills which an individual teacher selects as important to be applied in their selected strategies for presenting the subject content material. Several attempts to initiate a communal forum for professional exchange failed, either because of lack of time or the perception on the part of staff that cross subject exchanges about strategies for pedagogical ICT uses were not profitable.

As a consequence of this and the associated teacher and subject differences, the experiences offered to the pupils were wide ranging in quality and effectiveness. There was little cross subject consultation, and no coordination between the ICT skills required by subject teachers in the classrooms, the 5-14 ICT guidelines and the ICT classes for the pupils. This led to duplication of teaching, and some subject teachers having to teach the ICT skills required for their class work.

The Technical Difficulties

The technical problems faced by the staff and pupils were considerable. The laptops had to be withdrawn on two separate occasions for several weeks at a time to be given attention. It was found that a confusing interaction of hardware, software, and inappropriate user actions was underlying the unreliability which had disrupted the effective use of the technology.

The Whole School Context

The management had given staff and parents a reassurance that involvement in the Project would not disrupt the normal progress of the pupils' education. As a consequence, the setting of pupils which normally takes place in key subject areas at the end of S1, was applied, breaking up the two whole classes of pupils and the group of teachers who had laptops. Although some teachers subsequently attempted to get pupils to share their laptops with classmates who were not involved in the project, the nature of the classroom tasks were still designed for individual work and the sharing caused some resentment on the part of the pupil laptop owners. Laptop use declined sharply during S2.

At the end of the project the teachers involved were allowed to keep their laptops, although the pupils' machines were returned and allocated to departments as sets for subject use. Thus overall, the school's aim of extending ICT use in the school had been achieved.

The Experiences of the Secondary School Pupils

[Interviewer: Well, anything final you want to tell me at all, anything that you think is important that I ought to be able to say on behalf of you all?]

Thanks for giving us the chance to do it. (Pupil S2)

There was evidence of a wide range of skills among the pupils at every stage in the project, but overall, a greater proportion of laptop pupils (61%) indicated by the end of the project that they were now 'real experts' in the use of computers in comparison to 34% in the non participants in the same year group.

Pupils indicated that the advantages of the laptop use had been the use of the internet for accessing information and ideas, and the ability to produce neater, faster work.

Gradually, however, the pupils' initial excitement and positive expectations of being allocated laptops began to be dampened by their experiences of the range of technical problems they encountered and the increasing constraints put both upon the classroom uses and their personal uses of the laptops with respect to games and other hobbies. There was a clear tension between the view that this was their own personal laptop, and the constraints the adults thought appropriate:

Yeah, you put your own backgrounds on it, install the stuff you want and stuff and then someone brought in a game and some of the boys, some people installed it and then we were all playing it and we were all happy. And then we weren't allowed to do it and so things got a bit boring - at least we'd had this game. (Pupil S2)

Given the experience which many pupils clearly had from their extensive use of computers at home and in their own rooms, it was understandable that the restrictions put on them by the school were seen as extremely irksome and unnecessary. Some were well aware of the dangers of the open chat rooms and mailing systems, and could describe how to set up systems to avoid the pitfalls.

They had all disliked having to carry the heavy laptops in bags continually around the school all day, every day, even when they knew there was little prospect of using them.

At the start of the year quite a lot of us were hardly ever carrying them around because we knew we weren't using them in classes with little room to put them in. But then the teacher was saying that you had to keep them with you wherever you go so we had to take them with us. (Pupil S2)

The lack of technical support, and the initial overly brief induction into general and specific ICT skills and management meant that the pupils' difficulties presented themselves in the classrooms, but it depended on the competence and patience of the individual teacher whether their problems were satisfactorily engaged with.

The overall judgement has to be that the technical and initial ICT management support given to the pupils was not adequate, and was likely to have contributed to much of the negativity and lack of care which some of the teachers indicated they perceived as a problem with some of the pupils. Some of the pupils got the much needed support from knowledgeable parents or others at home.

Their final comments reflected their mixed feelings about their experiences and the removal of their laptops:

I think kind of overall I'm kind of glad that they're away but what we were saying about the PowerPoint and stuff - it made it easier. But the carrying it around, it was always back breaking. (Pupil S2)

The Perceptions of the Secondary School Parents

The most frequently mentioned benefit to their child's learning was the improvement of their skills in the use of computers. The most frequently mentioned disadvantages were the difficulties associated with the problems of the technology, along with the requirement for the machines to be carried around all day regardless of the low level of use.

The overall verdict of the parents was negative in two thirds of the sample. They perceived it as an excellent opportunity which had ended up with more difficulties and frustrations than they had anticipated.

The most frequently expressed view on how it could have been improved was that more teachers should have been involved or committed to the project, or that the teachers themselves should have been more skilled in the use of the computers.

Lessons Learned and Strategies for the Future

The Authority staff emerged from the project with the view that their visionary focus on ICT as the driver of change in teaching and learning practices had perhaps been over ambitious and misconceived. They felt that given intensive planning of time and resources, with greatly increased technical and curricular support, many of the managerial difficulties, particularly as experienced by the secondary school staff, might have been overcome. But without other conditions being present, the technology would make little impact on changing teaching procedures.

They considered that the reasons the project had worked well in the primary school (and indeed the reasons ICT in general works better in that sector) had been that the curriculum is less 'stuffed and rigid' than in secondary schools, and the primary staff had benefited from a mutually supportive collegiality, with respect to integrating the ICT uses into the curriculum, ICT skills development, and teaching and learning. Most importantly, their existing model of learner centred pedagogy had been supported and enhanced by the pupils' use of the flexible, empowering technology.

The Authority felt that with respect to the teaching approaches and skills in the use of ICT generally, the gap between where teachers are and where the Authority would like them to be is greater in the secondary school sector than in the primary. They attributed this to the fact that primary teachers have, over a longer period of time, had a computer in regular use in their classrooms and have already generated models of how the teacher and pupils might use the technology in classrooms for enhancement of learning, rather than for coverage of subjects.

Their views now were that in order to achieve transformation in learning and teaching:

a) personalised ICT within the schools in the form of laptops was not a means to achieve this end in schools in which learner centred pedagogies had not yet developed;
b) that the introduction of new technologies without adequate prior changes in pedagogical practice would tend to result in the technologies merely being fitted into current practices;
c) the new focus for developments should be understanding and establishing new kinds of relationships between learning and teaching and the technology, the teachers, the pupils and the curriculum. Assessment is for Learning was seen by the Authority as a possible vehicle for taking this forward.

These findings are mirrored in studies outwith Scotland. There is a growing research literature which suggests that although the permeation of ICT throughout education brings the power to introduce significant innovations into schools, neither teaching or learning practices appear to be greatly affected by the introduction of ICT into classrooms. Teachers continue to engage in already established teaching routines, using ICT as a tool to improve current classroom procedures and to streamline their current activities. Learners, initially at least, do become more interested and motivated, but this has yet to show an impact in terms of increased academic attainment or their engagement in more truly independent learning.

The Authority concluded that in the future, their focus for developments to initiate change in classrooms should not be on the introduction of ICT into schools, but on initiatives which were focussed on developments in learning and teaching. They identified that these activities should focus on developing different forms of relationships:

• between learning and teaching and the technology: changes in strategies for teaching and learning should ideally be in train before introducing the technology;
• between the teachers and the technology: staff development has to be at a high and continuing level;
• between the pupils and the technology: the existence of pupil ICT-based cultures had to be acknowledged and catered for, e.g. by provision of areas in the schools' servers for pupils' games etc;
• between the teachers and the curriculum: the secondary curriculum is still too overloaded with content to allow for new pedagogy to be introduced; curricular support to aid teachers with development of quality learning experiences is necessary;
• between teachers and their models of staff development: the communal, co-ordinated staff development activities of the primary schools appear to be more effective in advancing skills and innovative practice than the individualistic, autonomous, subject oriented development of teachers characteristic of secondary schools;
• between the technology and School Development Planning: the full involvement of school management to coordinate ICT policies and initiatives across the school is necessary, and costly initiatives such as the introduction of personalised systems have to be fully integrated into the school development plan.

If you have views on the Insight series or wish to find out more about SEED's research and economics programme, please contact Information, Analysis and Communication Division, Scottish Executive Education Department, Victoria Quay, Edinburgh EH6 6QQ or by e-mail on recs.admin@scotland.gsi.gov.uk