|
| In June 1995, The Scottish Office commissioned research to study the Standard Entrance Test (SET) for entrance to the police in Scotland. The five month project was undertaken by a small team from the Scottish Council for Research in Education. The research comprised two phases: an analysis of extant recruitment and selection statistics for the years 1992-94 from the eight police forces; and a more detailed exploration, utilising a case study approach, of practices within two forces. This paper reports the main findings from both phases. The full report presenting evidence, conclusions and recommendations was published in the Central Research Unit Series in October 1996. |
|
| Main findings |
- Both society and policing are changing.
- Recruitment and selection is an important but time consuming activity for the police.
- Each force has developed a slightly different recruitment pathway which affects the role and function of the SET.
- There are significant differences between both the four sections of each test and also the three existing versions of the test.
- The SET should be perceived as one step in a recruitment pathway from civilian recruit to fully trained police officer.
- Candidates experience difficulties particularly with Section 2 (Data Interpretation) and Section 3 (Calculation and Number Work) of the SET.
- Male and female candidates have different prior educational attainment levels and also achieve slightly different scores on the SET.
- Candidates with a higher level of educational qualification have a slightly increased Chance of being appointed.
- Proportionately more women and candidates from ethnic minority groups experience difficulties with the SET.
- Both the insufficient number of candidates from ethnic minority backgrounds and their disproportionately higher failure rate on the SET need to be addressed.
|
|
| The police standard entrance test |
| The Standard Entrance Jest (SET) is the test taken by most candidates for entry to Scottish police forces. (Serving officers of other United Kingdom police forces are exempt). It is a timed 'paper and pencil' type test administered by polite officers at force level. The SET consists of 90 questions each of which carries one mark. The questions were developed to match the level of an Intelligence Quotient (IQ) required for the ordinary grade of the Scottish Certificate in Education. |
| There are three versions of the SET currently in use and each contains the following sections: |
- Listening comprehension (25 marks)
- Date interpretation (17 marks)
- Calculation and number work (15 marks)
- Reading comprehension (33 marks)
|
|
| The SET as a test instrument |
| Visual presentation |
| The printing, presentation and reprographic qualities of the SFT were poor. Not only did these present difficulties for candidates but also created a less than favourable professional image for the police forces. In particular, inexact graphs in the Data Interpretation section could invalidate results. |
|
| Aims of the study |
| The SET was last revised in 1985 and considerable changes, in both society and the police, have taken place since then In particular, the Central Conference of Recruiting and Training Officers (CCRTO), a subcommittee of the Association of Chief Police Officers in Scotland (ACPOS), suggested that the test may now be outdated. This provided the main impetus for the research. |
| The study set out to explore: |
- the function(s) of the SET in the recruitment and selection processes of Scottish police forces
- the effectiveness and fairness of the SET in selecting candidates with the potential to become good police officers
- the functions which Scottish police forces want the SET to perform.
|
|
| Levels of difficulty |
| There are three versions of the SET and police forces may use whichever version they wish Clearly, it is important that the level of difficulty of each is approximately the same. However, when we compared the three versions using a readability test (Flesch method) significance differences emerged. |
| Version 1 of the SET contained 91 words more than version 2 and 141 words more than version 3. Version 3 has considerably fewer lines and sentences than the other two versions. Versions 1 and 3 have reading levels which can be categorised as 'difficult', that is equivalent to that required at the end of secondary schooling; whereas, the reading level demanded by version 2 falls within the 'very difficult category - equivalent to that required for a university degree. We concluded that the three versions of the SET are not comparable in level of reading difficulty. |
| Perhaps of greater significance for most candidates have been the consequences of changes in the Scottish Educational System, both the curriculum and teaching and learning styles, which now impact on the SET Many items were confusing simply because they contained out-of-date information or terms with which candidates may hot be familiar, for example use of both metric and imperial measurements. It is easier for people to solve problems when they are properly contextualised and with familiar aids to problem-solving. We know from theories of situated learning that formal mathematics taught in schools is very different from the maths which is practised in everyday life or used by serving police officers. |
|
| Is the SET effective? |
| The main purpose of the SET is to select those candidates who, with the benefit of training at both force and Scottish Police College levels, have the abilities to become good police officers. Attrition rates from the police forces, through drop out or failure to complete probation, are low. However, our main concern here is that the SET may eliminate candidates who have the very qualities now sought by police forces but are unable to pass the SET in its current format |
| Approximately one in five candidates in our random sample of applicants to all forces between 1992-94 had a university degree or other academic qualification. However, 50% of the total sample had only achieved SCE ordinary/standard grade examinations. Therefore, one test is attempting to differentiate between candidates with a very wide range of educational attainment. On average, female candidates were better qualified than their male counterparts: 60% of women, compared with 48% of men had achieved more than SCE standard grade (see Figure 1). |
|
| However, the position was reversed when we considered pass rates: 41% of women failed the test, compared with 36% of men as shown in Figure 2. |
|
| Additionally, an examination of the test scores for each section of the SET revealed that the distribution of scores on each section was very different. The most pronounced was for Section 3 (Calculation and Number Work) but also Section 2 (Data Interpretation) which indicated that more candidates have difficulties with those two sections. For these reasons, we suggest that the current SET is unreliable. |
|
| Is the SET fair? |
| Obviously, the purpose of the SET is not to discriminate against certain groups of candidates, and Scottish Office guidance (Scottish Office, 1992) and an HMIC report (HMIC, 1993) have specifically addressed issues of equality in the police service. Forces, themselves, are very aware of these issues and have demonstrated their concerns in the recruitment targets they have set. A number of our respondents expressed the view that the police should better represent the communities which they serve. However, bias may have inadvertently been built into the SET's original design which included the piloting of the test on recruits at the Scottish Police College in 1985 - a very different group from contemporary Scottish society which the police now wish to represent. |
| Statistical evidence supports our concerns regarding the internal validity of the SET. For example, an examination of a sample of recruitment statistics over the three year period, 1992-94 in one of our case study forces, generated information on 17,513 applicants. The breakdown for the total sample was 77% male and 23% female candidates. Women are, therefore, under-represented as applicants compared with their proportion in the population as a whole and proportionately more of them fail the test. (Of all who fail the SET 55% are men and 45% women). Additionally, approximately three times as many ethnic minority candidates failed the SET as would be expected from their representation in the total number of applicants. These findings are shown in Figure 3. |
|
| The demands of policing |
| Effective recruitment and selection procedures should provide a ready supply of good recruits who match the requirements of the job. To identify specific competences required of police in Scotland, we utilised existing functional maps of policing and compared these with the work practices as described to us by officers in two case study forces. There were similarities in a number of studies. We identified that probably the most important was situational understanding. This was demonstrated by a number of our respondents who described situations in which they had to: |
- process information very quickly under difficult or dangerous circumstances
- solve problems quickly by taking decisive action
- stall or 'fish' for information upon which to base their decision.
|
| Ability to work in teams; understand the culture of the police and the danger they face; display 'common sense'; utilise discretion; cope with stress and work flexibly were all identified as important areas. All of these are extremely difficult, if not impossible, to assess by means of a 'paper and pencil' test such as the SET. |
|
| The recruitment pathway |
| We discovered that the path from civilian to regular police officer varies greatly across the eight forces. Each force has developed a slightly different recruitment pathway which affects the role and function of the SET. Discrete steps included: application forms; the SET; background reports; medical examinations and interviews. Some forces have extended the range of entrance requirements and included other pre-selection methods. Others include assessment centre type activities in the recruitment process. Both the variety and sequencing of the steps in each force's recruitment pathway varies. |
| Additionally, there is increasing evidence that the face validity of the SET (viz. the extent to which the test actually measures what it intends to measure) has declined over time. Some forces have already recognised these problems and tailored the recruitment pathway to suit their own particular circumstances. viz. size, local circumstances and numbers of recruits processed. |
|
| How can the SET be improved? |
| Most of our respondents, while holding fast to the concept of a SET, generally indicated support for its revision. We would recommend that the following be considered to improve both the Test and some of the wider recruitment and selection issues identified during the research: |
- develop a criterion-referenced test ensuring that it reflects changes in both education and society
- pilot the test on a representative sample of the population to ensure that no one group is disadvantaged
- base the test on the abilities and knowledge demanded by modern police work
- agree a standardised recruitment pathway for Scottish forces
- develop methods to encourage more informed prior self-selection by candidates and perhaps more contentiously;
- consider developing a system for accreditation of prior learning and relevant past experiences as an alternative to the SET in order to reduce the overall number of candidates tested.
|
|
| This Research Findings paper is based upon research carried out by Valerie Wilson, Peter Glissov and Bridget Somekh, of the Scottish Council for Research in Education, between June and November, 1995. Copies of the full report, Examining the Test: An Evaluation of the Police Standard Entrance Test, are available from The Stationery Office at a cost of £5.00 each. |
|
Cheques should be made payable to The Stationery Office and addressed to:
The Stationery Office Ltd, Mail Order Department, 71 Lothian Road, Edinburgh, EH3 9AZ. |
| Telephone: 0131 228 4181 Fax: 0131 622 7017 |
| The report can also be ordered online from:www.thestationeryoffice.co.uk |
| Further copies of the Research Findings Paper, or information about the Central Research Unit Programme, can be obtained by contacting the:The Scottish Office Home Department |
Central Research Unit
J1-0; |
Saughton House
Broomhouse Drive
Edinburgh EH11 3XA
Tel: 0131 244 2114 |
|
| List of Previous Research Findings |
| 1. The Scottish Crime Survey 1993: First Results |
| 2. A Fine on Time: The Monitoring and Evaluation of the Pilot Supervised Attendance Order Schemes |
| 3. Use of Controlled Drugs in Scotland: Findings from the 1993 Scottish Crime Survey |
| 4. Live Television Link: An Evaluation of its use by Child Witnesses in Scottish Criminal Trials |
| 5. Information Needs of Victims |
| 6. Public Interest and Private Grief: A Study of Fatal Accident Inquiries in Scotland |
| 7. An Evaluation of The Scottish Office Domestic Violence Media Campaign |
| 8. Does CCTV Prevent Crime? An Evaluation of the use of CCTV Surveillance Cameras in Airdrie Town Centre |
| 9. Making Our Cities Safe: Evaluating the Safer Cities Programmes in Scotland |
| 10. Proactive Policing |
| 11. Foreign Language Interpreters |
| 12. Grounds of Appeal in Criminal Cases |
| 13. Listening to Victims of Crime: Victimisation Episodes and the Criminal Justice System in Scotland: An examination of white and ethnic minority crime victim experience |
| 14. The Use of Compensation Orders in Scotland |