« Previous | Contents | Next »
Listen
CHAPTER TWO: METHODOLOGY: THE LEARNER SAMPLE
2.1 Obtaining the views of literacy and numeracy learners is quite a methodological challenge as by definition they are unlikely to have the reading and writing skills necessary to complete questionnaires, or the other usual instruments employed by researchers seeking to ascertain the views of a large number of respondents. Moreover, this group of learners are potentially vulnerable and may not respond to people that are strangers. For both these reasons, face-to-face interviews were conducted with learners in their place of tuition by a team of interviewers, most of whom were based in the same Partnership area. This approach meant that learners were more likely to be responsive because they were interviewed by a person that they had been introduced to by their tutor, and the interview took place in familiar settings. The impact on learning was measured using a repeated measures design to ascertain respondents' views of ALN provision and to assess how these views had changed over time.
First Survey of the Learners
The Sample
2.2 The sample of learners was selected from 9 ALN partnership areas that represented the geographical diversity of Scotland (large conurbations, small village communities, rural, mixed, urban, North, South, East and West) and learner population sizes. ALN Partnerships were asked to identify providers from community, further education, voluntary sector and the private sector, and a variety of locations including workplaces, where learners were engaged in integrated and dedicated provision, in groups, 1:1 and open learning provision. A target number of learners was set representing roughly 10% of the learner population that each Partnership had identified as receiving tuition in the preceding year, and each Partnership agreed that identifying this number of learners was feasible. There were, however, considerable difficulties in obtaining accurate information on the provision, location and number of learners receiving tuition from Partnerships. This was partly due to the devolved nature of administrative responsibility that meant, in most of the Partnerships, that no one had overall responsibility for providing accurate information on learners and so identifying and selecting the sample was problematic. This meant that the research team had to contact places of tuition directly using information supplied by Partnerships rather than contacting tutors as had been the original method of obtaining the sample. It was subsequently agreed, in the light of the difficulties in obtaining the target number of learners from the Partnerships, that the sample would be reduced from the original 1000. Two thousand and four learners were contacted based in 114 different providers and of these 613 were interviewed. Learners' informed consent to participate was obtained through tutors at the identified study sites, who read learners an account of the research and then asked them to sign an informed consent form. On average 22% of the learners that were approached agreed to participate in the research and this low consent rate partly reflects the particular care that was taken to ensure that this potentially vulnerable group of learners were committed and willing participants. It also partly reflects the views of some tutors who were unwilling to allow their learners to be interviewed. Other research (for example, Evans, 2005) has identified the role of such 'gate-keepers' in preventing access to vulnerable subjects.
The First Interview
2.3 Face-to-face interviews, based on a questionnaire that had both closed and open questions, took place between September 2003 and April 2004 and lasted around one hour. The questions were designed to ascertain demographic information; the 'social capital' of the learner; an assessment of confidence; learners' pathways into ALN; any perceived barriers to entry; their experience of learning, teaching and the curriculum; the guidance learners had received at entry, during and at the end of the programme; the effect of participation on their personal, family, public, education and working lives; degree of satisfaction with the quality of the learning programme; views of how provision can be improved; plans for future action. A mixture of closed and open questions was used to enable the views of learners to be captured. Open questions were asked about all these areas except for the sections on social capital and confidence, and these qualitative data were analysed for 200 (just over 50%) of the learner sample. This analysis was carried out in order to explore in-depth processes and events that influenced changes in learners' lives, and enable emerging themes to be identified and explored. Cases were selected using a stratified random procedure to ensure that the sample included learners from all partnerships involved in the research. Not surprisingly, given the proportion of the total sample included in the qualitative analysis, the demographic profile of the qualitative sample closely matched that of the total sample. For example, there were no significant variations between the two samples when examined by sex, age and type of provider.
2.4 The questions were derived from a variety of sources. The questions on 'Social Capital', 'the processes between people that establish networks, norms and social trust that facilitate co-ordination and co-operation for mutual benefit leading to reciprocity', were derived from the literature based on quantifiable measures used in the USA, Australia and the UK. The confidence scales involved identifying scenarios that comprised typical situations for learners in their everyday lives and asked them to identify how confident they were. These questions and those related to social capital were designed to measure change over time. The questions on the quality of provision were based on the 'Literacies in the Community; Resources for Practitioners and Managers' (2000) evaluation framework in order that the benchmarks identified in the framework could be evaluated by the learners.
Second Survey of the Learners
The Sample
2.5 Learners were sent greetings cards by the project at regular intervals and then re-contacted by interviewers after approximately one year by telephone or letter to arrange another interview. They were re-interviewed in a variety of locations ranging from their place of tuition to their homes and including community centres, further education colleges, prisons, YMCA, family learning centres, business centres, libraries and schools. Those that agreed to an interview received a £10 voucher as an acknowledgement of the time involved. Sixty-six percent of the interviews were face to face and the remainder were by telephone at the request of the learners. Interviews took place between September 2004 and April 2005. Three hundred and ninety three learners were interviewed a second time representing 64% of the original sample. This is a high re-contact rate compared with other projects that have tried to track this type of learner group. For example, a conference that focused on longitudinal literacies projects found high attrition levels of around fifty percent of their respondents ( e.g., Bingham and Merrifield, 2005). In this project and in others the main reason given for being unable to re-interview respondents was that they could not be found because they had changed their addresses and telephone numbers (75%). Of the remainder, ten percent were unavailable in the time period designated for the interview, and 15% did not wish to participate in another interview. The comparison between the two samples is therefore based on the 393 respondents who were interviewed twice. This comparison is used to assess the 'distance travelled' between the two interviews.
The Second Interview
2.6 The focus of this interview was on changes since the first interview so generally the same questions were asked. This enabled changes in learners' social capital and self-confidence between the first and second interviews to be assessed. In addition their views on the barriers that might put people off joining a programme and their experiences of learning, teaching and the curriculum; experience of guidance during the course and on exit; the effect of participation on their personal, family, public, education and working lives were explored. A new question was a general reflection from learners on all aspects of the ALN experience. As in the first interview, the questions were both open and closed and the sample of 200 learners whose data is reported on were people that had taken part in both rounds of interview.
Case Studies of Learners
2.7 After the results of the first interview had been analysed, it was clear that learners were overwhelmingly positive about their experiences and it appeared important to find additional means of checking that these findings were sound. Interviewers were therefore asked to provide purposive case studies of all learners that reported negative views of their experience of teaching and learning and five were reported in total. In addition, six interviewers provided case studies of learners that they had found particularly interesting and two of these are used to illuminate the section on self-confidence. These case studies are a result of purposive sampling so can only be seen as illuminating other data.
Data Analysis
2.8 So that the research could measure broad trends and at the same time analyse in depth testimonies from a sub-sample of the population studied, data derived from the questionnaires were both quantitative and qualitative. The quantitative analysis was undertaken for all learners and mapped a range of characteristics of sub-groups of adult learners and indicators of the nature and efficacy of the learning journey. The closed questionnaire data was coded and analysed using SPSS( TM). Key variables for the analysis were learners' background characteristics: personal, family and community characteristics and type of education provision being utilised. Other key variables were outcome measures: social capital development, self-esteem, motivation, and perspectives on the learning programmes. Bivariate analyses were conducted, however the low numbers in some variable categories precluded any systematic multivariate analysis of the data.
2.9 Unless otherwise stated in the text statistical significance is at the 1% level. This means that the likelihood of such an observed difference happening by chance is less than one in a hundred. Establishing significance at the 5% level (the alternative level used in the text) means that the likelihood of such an occurrence happening by chance is less than 1 in 20. Where statistical significance is quoted in the text this was established in one of two ways and was determined by the type of analysis being conducted. For analysis involving the cross-tabulation of responses from the first to the second round of interviews significance was tested for using the chi-squared test. For analysis involving the comparison of responses from individuals who had taken part in both the first and second round of interviews (to establish whether change had taken place in the intervening period) a matched pairs procedure coupled with a sign test was utilised. This involved comparing the responses from individuals to the same question asked in both rounds of interviews. The sign test examines any overall change in responses (for example, more or less positive or much the same) to ascertain whether differences tend in one direction or another and whether this is statistically significant.
2.10 The qualitative data were analysed for 200 of the learner sample in order to explore in depth processes and events that have influenced change in their lives, and enable emerging themes to be identified and explored. These 200 were selected to reflect the range of provision in the partnerships. By ensuring that all interviewees were asked the full range of questions in the first interview it became possible to sample the widest range of learners in the subsequent interview, as when some of those chosen could not be contacted for the subsequent interviews then the data from other (matched) learners was included in the analysis.
2.11 The questionnaire data was analysed using the database FileMaker( TM). The answers were copied into the database, codified, and then examined under a number of thematic headings. This method provides for analysis according to a priori systematic conceptual frame as well as allowing a flexible grounded approach for probing emergent issues. It also enabled the extraction of quotations that happen aptly to capture the essence of some relevant concern.
2.12 Interviews were recorded both by notes by the interviewers on the questionnaires and on mini-disc and a sample of interviewers were checked for consistency by the research team and an independent evaluator from Glasgow University. Notes from the open questions were copied into FileMaker, as with the qualitative interview data and analysed similarly.
« Previous | Contents | Next »