Chapter Four Measures to assess enterprising skills and attitudes
We now consider the nature of some of the approaches currently in use to measure these skills and attitudes, and comment on their usefulness for this study.
The measures examined are categorized as follows:
- self-report only
- observation and triangulation in addition to self-reporting
- psychometric evidence
The majority of existing measures are either wholly or partially self-report. The preponderance of self-report measures, despite their limitations, reflects the difficulty of designing and using observation/ triangulation approaches and psychometric measures.
We have added a comment in italics giving our assessment of the possible use of each of the measures to our research. With respect to the focused study element of the research, this is an initial assessment. More detailed consideration needs to be given to other aspects, for example, how the validity of using only parts of existing measures.
The Changemaker materials in England (1999) include, for each of the 14 enterprise skills, four exercises; identifying the skills, developing the skills, the transferability of the skills and reviewing the learning. They are designed to be used in any real projects which have been chosen by and are led by the learners. Facilitators help the young people to consider their own development in relation to each of the skills.
Elements of the wording of the skills are useful, but the exercises are more appropriate when attached to an educational activity.
The Evaluation of new Approaches to Work-Related learning at Key Stage Four in England and Wales (2002) used questionnaire materials for pupils, asking them to rate themselves on a 5 point scale (excellent, good, ok, poor, very poor) against various enterprise skills and abilities. This was completed following an experience of work-related learning. A similar approach was used in the evaluation of the Earn and Learn programme at Bathgate Academy and of the Motivator programme at Inveralmond Community High School in Livingston. Young people who had taken part in the programme were asked to identify how it had helped to: prepare them to go out into the adult world; make them more aware of their potential; make them more employable. They were presented with a list of 'enterprising' skills and asked to rate themselves against each of these on a 4 point scale.
Elements of this have been used in questionnaire items.
The Learning Gains Research (2002, Semple et al) used open questions on a self-report questionnaire to identify skills and attitudes towards work and 'enterprise' broadly defined.
These open questions could not be used in our pupil questionnaire because of the cost that would have been involved in coding and analyzing them given the large sample size.
The Employability Skills Toolkit, developed by the Conference Board of Canada (2001) is a resource which has been devised jointly by business and education, due to concerns about the gap between employers' needs and pupils' skills. It was designed with a view to helping young people begin to manage the process of knowing themselves, recognizing their strengths and weaknesses, seeking feedback and planning further development.
It outlines a range of personal skills and qualities and provides a framework for self-assessment which gives suggestions as to what relevant academic, personal management and teamwork skills might 'look like' and how they might be demonstrated at home, at school, at work or in the community. This paper-based aid is used in various ways;
- one to one coaching for disaffected pupils in school
- a personal development tool for careers guidance
- in a process of work based learning
- one of the key strengths of the toolkit is that it helps pupils to see the connections between competences, skills and behaviours as they apply in different situations.
While this tool refers to some enterprising skills and attitudes, it has a much greater focus on employability and is therefore less useful for this research.
Observation and triangulation in addition to self-reporting
The ongoing study being undertaken by staff at Strathclyde University, looking at the economic and educational benefits of enterprise in education includes examination of teacher and pupil perceptions of what 'being enterprising' means in a representative sample of schools. The next phase of data collection will include specific questions for teachers about what they consider important to assess during enterprise activity and how they do this. This will provide an indication of both the extent of the consensus on the definition of being 'enterprising' and the measures employed for a variety of groups.
These measures are still under development, and have been designed for a younger age group than the present study.
Resource material for Careers Scotland, created by the National Centre: Education for Work and Enterprise at Strathclyde University, with the aim of providing support for enterprise activity with young people from the ages of 5-25, includes various tools for assessment, including self-assessment, peer assessment, group assessment, teacher assessment/ observation and observation by adults other than teachers. These tools range from observation schedules, with an expectation that evidence of particular types of behaviour should be identified along with dates on which they were observed, to ratings against various enterprise skills and attitudes. The aim of this range of tools is to try to ensure that by including the views of a variety of people, a more complete picture of pupils' development can be developed.
Elements of the self-assessment tools have been incorporated into the pupils' questionnaire.
In Opening Minds, (1999) the set of 10 core competences defined by Halifax plc are detailed. These competences are relevant to all grades within the organization, but also give an indication of the desired qualities of potential employees. The framework defines key 'attributes, characteristics, behaviours and knowledge exhibited by successful performers'. There are three categories of competence: people, personal and process.
People: direction setting, developing self an others, communication and working with others
Personal: achievement orientation, customer orientation and change orientation
Process; forward thinking, judgment and quality process.
There is a structured assessment process, with each of these competences having 5 progressive levels, becoming increasingly complex and demanding, each describing a different type of behaviour. The system is designed to help participants set goals for the future, thereby increasing motivation. This may be a useful basis for elements of focused study element of the research work.
In Scotland, knowledge, skills, attitudes and understanding in relation to enterprise may be measured and accredited through the formal examination system. This can take place at secondary level, in options within Standard grade Social and Vocational Skills and Business Management Intermediate 1 and 2.
There may be elements of this which could be used in focused studies.
In addition, there are a number of units within the National Qualifications catalogue: Enterprise though Craft; Enterprise Activity; Considerations for Self-employment; and Identifying Opportunities: Recognising Entrepreneurial Potential. These units are still fairly new and to date only a few pupils have taken these. Within Higher Still and National Qualifications, core skills are embedded in the subject. Discrete core skill national units can be taken at certain levels. The Scottish Qualifications Authority is currently (December 2003) consulting on the development of a Scottish Progression Award in Enterprise.
There may be elements which might be used in focused studies.
Young Enterprise Scotland has an exam and award scheme, although not linked to the SQA framework. Some of this is workshop based, much web-enabled and all linked to Core Skills.
This is likely to require more time to use than will be available in the focused studies.
The Pathways to Enterprise programme in Canada uses observation records and ratings sheets for each pupil to build up an overall picture of some enterprising skills and attitudes, the main ones being self-confidence, showing initiative, taking responsibility and showing perseverance. The key participants in the assessment and evaluation process are identified as being pupils, teachers, parents and other community members. The role of each of these participants is described, along with the specific instruments which are regarded as appropriate for their involvement.
Key enterprise skills and qualities are rated during and/or after an enterprise project, while individual personal and team qualities can be graded at any point. Evidence of such skills and qualities are rated on a 5 point scale (1=never, 3 = sometimes, 5 = always).
Elements of this may be appropriate for focused studies.
Psychometric measures tend to be expensive to use and analyse - both in time and money resources. They also normally require an extensive set of questions and to be administered under particular conditions. There are some examples of psychometric measures which focus on either enterprise or entrepreneurial skills.
Research currently being undertaken at Kingston University, aimed at measuring young people's attitudes to enterprise, is piloting the use of the newly designed ATE test (Athayde 2003). This measure considers the expression of enterprising attitudes, skills and behaviours in teaching and learning. This is done through an extensive questionnaire which covers varied aspects of a young person's life, with considerable focus on the school experience.
If available within an appropriate timescale, this measure, once piloted, may be useful for focused study work with pupils since it recognizes and uses the school context. It could not be used to compare responses with other groups such as parents. The measure is still under development. The research team is maintaining a link to the work.
The Entrepreneurial Spirit programme being piloted at present in Scotland uses a psychometric measure normed against adult entrepreneurs. It includes two elements. The first is DISC which covers Dominance, Influence, Steadiness and Compliance. The second is PIAV which covers values and attitudes reported through a six element profile analysed under the headings of Theoretical, Aesthetic, Traditional, Social, Utilitarian, and Individualistic. The entrepreneurial individual is seen as individualistic and utilitarian in approach. These headings have proved difficult to add to the matrix but the best approximation has been made.
This measure is perhaps focused more narrowly on 'entrepreneurship' rather than the broader 'enterprise'. It requires formal training and accreditation for its use and would therefore not be appropriate for this study.