« Previous | Contents | Next »
Listen
CHAPTER 3: APPROACH TO ANALYSIS
In addition to deploying an organised and robust framework for identifying and collating relevant comments from respondents, the exercise also required a number of ground-rules to be set to ensure responses were prepared for analysis in a consistent and sensible fashion.
3.1 Analytical Framework
Two analytical frameworks were used: a statistical programme, SNAP to enable quantitative analysis, and an electronic ACCESS database specifically written for this consultation to enable qualitative analysis. The use of two frameworks was required to enable different types of responses to be managed and analysed in the most effective way.
The responses that followed the structured format presented on the consultation document were mainly analysed using SNAP, a standard statistical package, although some that were particularly lengthy were analysed in the database.
The electronic ACCESS database was used to store and assist analysis of the lengthier or free flowing responses which did not follow the structured questionnaire format. This database enabled the storage of free text in a systematic manner. The method of data entry could also be controlled via careful design of data entry forms to minimise the likelihood of any erroneous entries.
The fields used to record the material in the Access database were based on the questions set out in the consultation document. The result was a comprehensive list of fields that formed the headings for the consultation database of responses. A preliminary analysis of responses was conducted by the database consultant in consultation with the project manager, to fit responses into each of the fields on the database. Lengthy responses were summarised to enable them to be stored in the fields on ACCESS.
Once formatted into a summarised response corresponding to separate consultation questions, the database responses were then coded using the same codes that were applied to the responses entered in SNAP. This enabled all the data to be entered into SNAP for the final analysis and analysed quantitatively. This ensured inclusion of all responses regardless of the format in which they had originally been submitted.
Copies of computer tabulations and the database have been provided to the Scottish Executive under separate cover.
3.2 Publication of written responses
Where respondents have agreed to publication, these responses are available in the Scottish Executive library. After discussion with the consultation team, the convention adopted for this consultation has been to preserve anonymity of individual respondents and organisations, but to attribute their comments and quotes to the grouped respondent category to which they fit. In this way, a further depth is added to the analysis by providing some contextual information about the respondent type.
3.3 Ground-Rules
Separate Responses from the Same Individual/Organisation
On occasions, one respondent may send in more than one response. The consultation team at the Scottish Executive were primarily responsible for identifying and removing duplicates. The research team at George Street Research was also alert to the possibilities of such double entries to ensure that any identical or duplicate responses that had been missed were picked up and removed from the exercise. This was done by hand searching or electronic screening.
A total of 20 petitions and multiple responses were excluded from the analysis because they did not follow the format of the consultation questionnaire and details of these are provided in Appendix 3.
Quality control
In order to minimise any inconsistencies in approach, the research team at George Street Research was kept to a minimum size and all worked to a well tested set of rules for data examination and entry. In accordance with our standard practice, members of the research team verified 10% of coding and data entry and highlighted areas where a second opinion was required.
Quantitative Material
The following procedure for the coding of responses was used to ensure rigorous quality control throughout the analysis process.
- For each question, responses were listed and code frames developed (a code frame is a summarised list of all points made by respondents)
- Questionnaires were checked and coded manually by our experienced team of coders
- Data was entered into the statistical analytical package
- A computer edit of the data was carried out prior to analysis, involving both range and inter-field checks
Given the self-selecting nature of any consultation exercise, it should be noted that any statistics quoted here cannot be extrapolated to a wider population outwith the consultation population.
Factual Accuracy
The views presented in this analysis have not been vetted in any way for factual accuracy. The opinions and comments submitted to the consultation may be based on fact or may, indeed, be based on what respondents perceive to be accurate, but which others may interpret differently. It is important for the analysis to represent views from all perspectives. The report may, therefore, contain analysis of responses which may be factually inaccurate, or based on misunderstanding or misinformation on the actual proposals but nevertheless reflect strongly held views. In some instances, such inaccuracies and misunderstandings will be relevant findings in themselves.
Interpretation of findings
Those participating in this exercise were self selecting and each had their particular motivation to take part. The exercise was not intended to gain views that were representative of the Scottish population, but was intended to give all those who wished to comment an opportunity to do so. This has to be borne in mind in interpreting the findings presented here in this report.
3.4 Reporting
The following chapters document the substance of the analysis, presenting the main issues, arguments and views expressed in the responses. These follow broadly the ordering of issues raised in the consultation document. For each question, quantitative data obtained from the SNAP statistical analysis is presented with charts to illustrate the key findings. These are then overlaid with qualitative analysis, drawing examples from the responses contained in the database. Verbatim quotations have been taken from the database for illustrative purposes. The themes and issues that emerged during the Scottish Civic Forum events have been incorporated into the relevant chapters of the analysis.
Summary Codes
Due to the wide range of comments emerging across the consultation, throughout the quantitative analysis summary codes were utilised by grouping together the coded responses that were essentially making the same or broadly similar point. These summary codes have formed the basis of the charts and the initial discussion. Responses subsumed within each summary code were then identified to illustrate specific issues or examples of particular interest or relevance, reflecting the depth and breadth of responses provided.
Cross Breaks
The quantitative analysis was broken down according to respondent type to identify the number and types of respondent responding to each question. In order to get a feel for the overall numbers in favour or against the new proposals, an additional cross break was included in the final analysis which was based on the responses to question 4: Do you agree that the current system of parental representation should be replaced by a more flexible system of parent forum as decided locally by parents in a way which suits them and their school best?
The code frame was created by summarising the range of answers of relevance into three broad codes which enabled respondents to be grouped into three broad categories, according to whether they were:
- in favour of making changes to existing practices
- in favour of the new proposals
- not in favour of the new proposals
Contradictions in the data
It should be noted that throughout the analysis there may appear to be inherent contradictions in the responses. For example, when examining a chart, readers will notice that the total number of respondents commenting on a given proposal can add up to more than 100%. This is because there was no limit to the number of comments that a respondent could give to each question and every separate comment was coded and included in our analysis. Moreover, in some cases, a single respondent gave contradictory comments, for example comments expressing both support and opposition to the proposals. Also, not every respondent answered every question.
Report structure
As the majority of respondents addressed their responses to each of the questions posed in the consultation report, the findings are presented for each of these questions in chapters 4 to 8. Although most respondents provided answers to each of the questions, their responses often cited the same or similar reasons for their view separately for each question. Therefore there was some repetition in the themes being cited by respondents.
« Previous | Contents | Next »