« Previous | Contents | Next »
Listen
CHAPTER TWO: EVALUATION AIMS AND METHODOLOGY
2.1 Within the overarching policy context, the aim of this evaluation of the One Plus and Next Steps employment retention projects was to explore the nature, progress and effectiveness of the initiatives from the perspectives of the staff and clients. The specific objectives of the research included to:
- Explore the nature, progress and effectiveness of Scottish Executive sustained employment support initiatives from both a staff and client perspective;
- Describe the work, what the projects offered, and examine processes;
- Explore the perceptions and experiences of both clients and staff who have participated in the initiatives and the most effective methods of support;
- Explore client experiences of support, training and employment;
- Assess the impacts perceived by clients and staff of client participation regarding job sustainability, career progression and measurable increases in self-confidence and;
- Identify and consider any possible improvements or changes to the two projects, gaps in delivery and potential future developments.
Methodology
2.2 The methodology for the evaluation, which was refined at the initial inception meeting, incorporated:
- A small-scale literature review, using the Rapid Evidence Assessment approach ( see Annex A).
- Focused qualitative work including face-to-face depth interviews with managers and caseworkers from each organisation and face-to-face and telephone interviews with 20 clients from each organisation.
- Depth face-to-face interviews with partner organisations.
- Analysis of organisational monitoring data.
Sampling
2.3 For the client sample, each organisation provided BMG with an anonymised database of their clients. Based on the profile of the client database (such as age, gender, family type, number of children and labour market barriers) 40 potential respondents were purposively selected and returned to the providers to enable them to match up the files with named clients and to approach them to seek consent to participate in the research. In the event, some of the sample had to be replaced by the provider as the clients were unwilling or unable to participate in the research, or because they were unavailable on the interview days. Where this was the case, providers attempted to replace them with individuals with similar characteristics.
Analysis
2.4 The analysis was conducted using a data-mapping matrix approach. This comprises an analysis grid for the classification and interpretation of qualitative data. The key themes and topics were identified through the interview schedule (or 'topic guide') and through an initial review of the transcripts. Each theme was then translated to a column heading in a matrix chart (in Excel), with each row within the chart represents a case (that is, an individual). Each transcript was then read, and relevant data extracted and summarised for input into a specific cell within the matrix (also including the relevant paragraph number or page numbers of the transcript). The result was a series of tables representing all of the interviews. This enabled the researchers to read across to get a full account of each individual's views and experiences and to read vertically for the themes. This process allows the full range of experiences and views to be documented, as well as capturing possible explanatory variables. It also ensures that the process of qualitative analysis is both transparent and replicable.
« Previous | Contents | Next »