The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER THREE: EDUCATIONAL ATTAINMENT DATA

3.1 Educational attainment data

3.1.1 The aim of this aspect of the MEPESS project was to explore patterns of educational achievement shown by minority ethnic and other pupils in the schools covered by this research. However, the general context concerning the collection and holding of assessment data by schools and authorities in Scotland was instrumental in determining the quality and quantity of data received. In the event, for reasons which are detailed in this section, the statements which can be made about educational attainment are, if anything, more about the context in which the data was collected than they are about the pupils themselves. As a result, this section is mainly about methodological and practical issues surrounding the availability of attainment data with ethnicity as a variable.

3.1.2 For many schools, the practice of routinely collecting data on educational attainment and forwarding the results to the authority for central storage and interrogation is a new one. Also, it is not one which all teachers have embraced enthusiastically since there is some unease about the use to which such data might be put. However, there is growing acceptance by the teaching profession that the practice will become established and that there are strong professional arguments in its favour. The position therefore, is currently changing, not only over time but also over place. Since circumstances vary from one authority to another, they will be considered separately.

3.2 Data made available to MEPESS

3.2.1 Authority A

Authority A is currently developing its practice in this area and in time intends to use a dedicated system to develop a central database but at present the material available is not in a state suitable for research use. A handful of primary schools are putting pupil-level 5-14 data (i.e. A to F levels) on to Phoenix but the data quality is variable at the moment. At secondary level, pupil-level data has been entered into Phoenix for Standard and Higher Grades but these do not have a field for ethnic background. The MidYis tests from Durham University are also used by this authority and these results will be stored along with a field for ethnic background, though it is not compulsory for parents to fill this in.

Towards the end of May 2002, the authority indicated that it did not feel that it would be appropriate to pass on the data to MEPESS. The reasons were: (1) because of the very small numbers of pupils involved any results would be inconclusive and (2) even without providing names directly, any mention of different ethnic groups in the analysis may enable individual identification of pupils. The authority subsequently indicated that it did not wish MEPESS to make direct approaches to schools to seek pupil level data. After a further final approach, Authority A agreed to supply data for the two secondary schools concerned but at the time of submitting this report, no data had been received. Consequently no assessment data is available from Authority A for the current research.

3.2.2 Authority B

Over recent years, Authority B has been participating in a Collaborative Research Programme with a Scottish university. One of the strands of this programme has been the evaluation of the Early Intervention Programme for numeracy and literacy in Primary 1 and also the analysis of reading and mathematics test data collected by the Council in Secondary 1 and Secondary 2.

As one of the independent variables included in these analyses was ethnic background, there already exists a base of research evidence on the effect of ethnicity on both the statics and dynamics of educational attainment (i.e. whether minority ethnic pupils score higher or lower and whether their scores change more or less over time than majority ethnic pupils). As the Authority funded this research, it has ownership of both the findings and the report as well as the original data on which they were based. However, the Authority when approached did not give its permission either for the data to be used, or the findings to be quoted for present purposes. A final approach from SEED to the Director of Education was made but did not lead to any change in this position. Consequently, no assessment data is available from Authority B for the current research.

3.2.3 Authority C

Like Authority A, Authority C is in the process of developing mechanisms for holding a central database of pupil achievement and from session 2002-3, this will be done using a dedicated system. At the moment there is no central database of pupil-level attainment in SQA examinations in secondary schools. At primary level however Authority C was able to supply a spreadsheet of 5-14 levels in reading, writing and mathematics accorded to each of the 832 pupils from P1 to P7 in the three primary schools concerned (Schools A, B and C). In addition, the file contained each pupil's gender, ethnic origin and home language. Consequently 5-14 assessment data from three primary schools was received, but no data from secondary schools is available from Authority C for the current research.

3.2.4 Authority D

Using a dedicated management information system, Authority D has central access to pupil-level 5-14 data (i.e. A to F levels). The Authority was able to supply a spreadsheet of 5-14 levels in reading, writing and mathematics accorded to each of the 1,197 pupils from P1 to P7 in the three primary schools concerned (Schools D, E and F). In addition, the file contained each pupil's gender, ethnic origin and home language.

At secondary level, the authority was able to supply a spreadsheet containing the same fields as for the primary schools for 757 pupils in S1 and S2 in the two secondary schools concerned (Schools G and H). In addition, the file contained records for 268 pupils in S4. Of these records, 124 contained the number of standard grades taken, and the grade achieved in each as well as each pupil's gender, ethnic origin and home language. Consequently, assessment data from three primary and two secondary schools is available from Authority D for the current research.

3.3 Data presentation

This section is confined to a factual presentation of the data received and comments on its nature. Evaluation and discussion of the data is included in section 3.5.

3.3.1 Ethnicity and home language

Table 3a presents a complete listing of all the ethnicities and languages recorded in the six primary schools in the data from Authority C and Authority D.

Table 3a - Ethnicities and languages in the Authority C and Authority D primary schools included in this research

Table 3b presents a complete listing of all the ethnicities and languages recorded in the two secondary schools in the data from Authority D.

Table 3b - Ethnicities and languages in the Authority C and Authority D secondary schools included in this research

3.3.2 5-14 levels

The 5-14 curriculum covers the expected spectrum of pupil attainment from the start of primary education to the end of the second year of secondary education. Achievement is measured by the level which the pupil has reached. There are six levels ranging from A, the lowest level, to F, the highest. There is also a code U which indicates that the pupil has not yet been assessed as having reached level A. Attainment is measured in a number of curriculum areas but for present purposes, only three of these, Reading, Writing and Mathematics, are relevant.

Table 3c conveys the completeness of the data. The first column gives the number of pupils in each of the six primary and two secondary schools for which data was received. The remaining columns give the number of valid fields for each school for Reading, Writing and Mathematics. The figures in brackets give the number of valid fields (i.e. as a percentage of the figure in the first column).

Table 3c - Data completion rates by school

However, these completeness figures are difficult to interpret. In particular, there is (for some of the data at least) a theoretical distinction between code 'U' (meaning 'not yet at level A') and 'missing data' (meaning that the pupil could be at level A or beyond but a result is not available).

No data for 5-14 levels in Reading, Writing or Mathematics were available for P1 or P2 in School A, School B or School C or for P3 in School C and no 5-14 levels for Writing were available in P3 for School A. In the case of P1 pupils, there is unlikely to be any distinction between 'U' and 'missing data' since pupils may well not have been assessed so early in their school careers. Beyond that however the distinction may be important. It is unlikely for instance that no P3 pupil in School C has achieved level A in any area. More likely is that either it is the school's policy not to administer 5-14 tests until P4 or that for some reason the entire batch of assessments had not been forwarded to the authority. Even more marked, one P7 pupil in School A was recorded as being at Level C in Writing and Mathematics but had no recorded Level for Reading. One P7 pupil in School C was recorded as being at Level D in Reading but had no recorded level for Writing or Mathematics. These are very unlikely to be a genuine Unclassified so they were treated as missing values and excluded from the analysis.

Generally speaking, in what follows, missing values were excluded if they covered the attainment in a given area (e.g. Reading) for a whole year group in a particular school or, if they differed by more than two levels from another attainment recorded for the same child.

The following attempt was made to investigate whether missing data rates varied between minority ethnic pupils and others. It was assumed that for pupils in P4 and above, missing values denote unknown level of attainment as opposed to 'known to be not yet at level A'. This will not be true in all cases but is probably true for most cases and it does enable the comparison to be made. Table 3d gives the missing data rates, with numbers of pupils in brackets, for pupils whose home language was given as English and those whose home language was other than English.

Table 3d - Missing data rates by home language (P4 and above)

Clearly the missing data rates are substantially higher for those pupils whose home language is not English. It is possible that part of the explanation for this is that the assumption described above is incorrect and that the difference reflects the fact that more of the non-English group are not yet at Level A. It seems very unlikely however that this explains the whole of what are substantial differences between the groups. It is not possible to comment further at this stage on why this might have arisen but it is worth noting that one implication is to complicate even further the comparison of the different pupil groups since there appears to have been different degrees of data selection amongst them.

In order to facilitate the processing and presentation of the data, the 5-14 levels were numerically coded as follows: U = 0; A = 1; B = 2; C = 3; D = 4; E = 5; F = 6. Table 3e gives the mean value of this variable, taken over all pupils, for each area of attainment, along with its standard deviation (in brackets) and the approximate number of cases on which each mean is based.

Table 3e - Mean 5-14 levels by year and curriculum area

Data about the cultural/ethnic background of pupils was contained in two fields relating respectively to 'Ethnic Origin' and 'Home Language'. For the former, Authority C and Authority D used different systems of categorisation and technical difficulties were encountered in transferring the Authority C data to a common file format carrying the data from both authorities.

It was decided therefore to use home language as the dependent variable as the coding systems used by the two authorities were easily reconcilable. This had the obvious consequence that minority ethnic pupils whose home language is English were not differentiated from majority ethnic pupils. Considerable collapsing of categories was necessary in order to maintain reasonable cell sizes. For present purposes, four categories were used: (1) English; (2) South Asian (including Punjabi, Hindi, Bengali, Gujerati, Urdu, Mirpuri and Memon); (3) Arabic; and (4) any other language or language unknown.

Table 3f gives the mean 5-14 level for each year from P2/3 to S2 of pupils in each of these four categories for Reading, Writing and Mathematics. Only cases where there are at least ten pupils in a cell have been included. The total number of Arabic-speaking children was 35 but no year included as many as ten of them and so the category does not feature in Table 3f.

Table 3f - Mean 5-14 levels by curriculum area, year and language group

In theory, Table 3f should allow the identification of any differences in the rates at which pupils in the three groups progress through the 5-14 levels. In fact, this will only be possible to the extent that the groups in the various years are comparable with each other. This is probably a reasonable proposition for the 'English' language group, partly because of its size and partly because of its comparative homogeneity. It is less reasonable for the 'South Asian' language group, partly because of its much smaller size and partly because some pooling of linguistic and cultural backgrounds has already been necessary. For the same reasons, and even more so, it is not reasonable for the 'Other/unknown' language group. Indeed in this case there is marked evidence of a lack of comparability. In Reading for example, the means for S1 and S2 respectively are 1.52 and 4.30. It is unlikely that the S2 group would have had a mean of 1.52 if they had been measured a year earlier. A much more likely explanation is that the groups are not comparable.

3.3.3 Standard Grade

Regarding the completeness of the data, of the 268 S4 records for Schools G and H, the number of Standard Grades taken was available for 258 or 96.3% while the actual Standard Grade passes obtained, and hence the grade point average (GPA), was available for 124 (46.3%).

In Table 3g, the first row gives the mean number of Standard Grades taken by pupils in three language groups. The second and third rows give the GPA and the number of Standard Grades taken in those cases where the GPA was known.

Table 3g - Standard Grade summary figures by language group

It can be seen that overall the groups whose first language was not English took slightly more Standard Grades than those whose first language was English. The South Asian group attained slightly lower grades (indicated by a higher mean value since grade 1 is the highest level of attainment) but neither they nor the 'Other/unknown' group is statistically significant from the English language group (two-tailed p = 0.41 and 0.83 respectively).

3.4 Discussion

3.4.1 Given the imperfect quality and coverage of the data on which they are based, it is perhaps not wholly surprising that the figures presented above tell us little about the difference between minority ethnic and majority ethnic pupils in the eight schools in Authority D and Authority C to which they refer.

3.4.2 Specifically, there are two reasons for this. The first is that 5-14 levels are not primarily designed for the purpose to which they are being put here. The present research ideally needs standardised assessment measures taken at constant intervals (i.e. all pupils assessed at the same time) to provide the most accurate measure possible of pupil attainment at each testing point. In contrast, 5-14 levels are confirmatory indicators that pupils have reached specified levels. They are not good measures of the time at which the level was reached and they are not measures at all of the extent, if any, by which the level had been exceeded at the time of the testing.

3.4.3 The second reason is that, while they are pupil-level (which is what is required here) as opposed to school-level measures, they are not longitudinal. A table corresponding to Table 3f above could be constructed from longitudinal data by following a cohort of students through the years from P2 to S2 to examine whether differences between minority ethnic and majority ethnic pupils change over time (e.g. whether there was any evidence of a 'catching-up' effect). This would be a much more sensitive test of the existence of any such changes.

3.4.4 As it is, Table 3f reflects not only any such changes which may be taking place but also year-to-year differences in the overall educational attainment of the pupils. Given the small size of the non-English language groups, even after collapsing the linguistic categories, it is likely that the year-to-year differences are substantial and in fact may well be greater than the longitudinal effects which are the focus of interest.

3.4.5 This likelihood is reinforced by the uneven progression of the mean 5-14 level of the non-English language groups in Table 3f. The size of the English language group is sufficient to give it some stability and the mean 5-14 level increases fairly smoothly from P2 to S2. In the case of the minority groups, however, there are several occasions where the 5-14 mean actually falls from one year to the year senior to it. It is highly unlikely that this would happen if the same pupils had been tracked over time.

3.4.6 A third feature of the data reported here which reduces its sensitivity as a measure of the educational effects of minority status is that, even to achieve the cell sizes reported, it has been necessary to pool linguistic categories which conflate cultural differences that might be important for the purposes of this research. The only category that was large enough to produce cell sizes of over ten in each year was the South Asian group and even this consisted of languages from a range of cultural and social traditions which might impact differently on the educational experiences of pupils.

3.4.7 The 'Other/unknown' group pooled languages from French via Turkish to Mandarin. It is not certain that these pupils have much in common socially or culturally other than that their first language is not English. When it is considered further that the available data does not take account of the length of time each pupil has been resident in Britain or the pupil's fluency in English, it will be appreciated that the database of educational attainment is well short of that needed to take account of the multiplicity of social and cultural influences which could be relevant to educational experience.

3.4.8 For the future, it appears to be the case that centralised holding of pupil-level data by authorities is becoming more widespread at the present time and it is likely that over the next two or three years the evidential base relating to pupil attainment will develop in both the proportion of pupils covered and the detail held about each one.

3.4.9 If additionally, the data is held in such a form as to permit the extraction of pupil-level longitudinal statistics, it could have real potential as a source of information about the relative progress of different pupil groups. This potential would be further realised if the use of standardised tests, currently being adopted or at least considered by a number of education authorities in Scotland, becomes accepted practice, at least amongst the four authorities which have featured in the present research. However, such an eventuality depends on a range of professional, political and technical developments, the investigation of which lies outside the scope of the present research.

3.5 Conclusions

3.5.1 One of the aims of this study was to make valid and reliable statements about the educational achievements of minority ethnic pupils. These statements may have been made either absolutely, in relation to 5-14 or other attainment benchmarks, or relatively as compared to their majority ethnic peers. The quantity, quality and representativeness of the data which was made available to the project was not sufficient to enable any reliable statements of any type to be made.

3.5.2 The paucity, in 2001-2, of the required database may reflect the state of development of the procedures and practices of schools and authorities as regards collecting, collating and using pupil-level attainment data. However, the professional analysis and application of such data can bring considerable benefits to both authorities and teachers in terms of monitoring patterns of pupil performance and the effectiveness of the school. The position regarding data availability may well change over the next few years in response to increasing appreciation of these benefits and greater familiarity with the technology of data storage and transfer.

3.5.3 Specifically, the requirements of the RR(A)A2000 may provide a further spur to the building of accurate and comprehensive methods of storing and using pupil-level attainment data. Recognition of the rights and responsibilities which the spirit of the Act places on all parties to the educational process should provide an incentive for teachers and local authority staff to accept ownership of this development and to identify the part they can play in it. It is likely however, that the achievement of this will require further staff development for both schools and authorities.

« Previous | Contents | Next »