« Previous | Contents | Next »
Listen
CHAPTER ONE: INTRODUCTION
1.1 In July 2006, the Scottish Executive's Efficient Government Delivery Group's e-Government and Take-up team, in conjunction with the Local Government and Public Service Reform Research Branch of the Finance and Central Services Department Analytical Services Division ( FCSD: ASD) commissioned ECOTEC Research and Consulting to evaluate the community e-democracy pilot project 'Renewing Local Democracy'. The overall aim of this evaluation was to assess the effectiveness of the project and make recommendations regarding its potential rollout more widely throughout Scotland. This report presents the findings of ECOTEC's evaluation.
Aims and objectives
1.2 The main aims of the evaluation were to assess the effectiveness of the pilot project in terms of:
- its impact on community engagement;
- its contribution to the functioning and role of the Community Councils ( CCs) involved; and
- the potential for wider rollout to other CCs and groups throughout Scotland.
1.3 The specific objectives were to:
- Identify what IT skills and infrastructure issues may affect take-up of the tools and require training or awareness support.
- Ascertain the views of participating councillors on the take-up of the tools and the impact of using the tools on Community Council activities.
- Identify whether use of the e-community toolkit led to any increase in public participation.
- Identify the extent to which the e-community toolkit facilitated the participation of disadvantaged/excluded groups.
- Analyse the views of participants in the context of how the toolkit can be sustained at local level.
- Based on the information available from this pilot, assess the risks, costs and impact that might be associated with further rollout of this toolkit.
- Locate these findings in the context of existing knowledge of, and research about, renewing local democracy through the use of IT in the UK, particularly other Community Council projects offering similar functions or public participation forums.
Evaluation framework
1.4 Our evaluation was underpinned by a framework which was developed at the outset of the study. The evaluation framework was developed following an initial review of project documentation and discussions at the inception meeting with the Scottish Executive. Our initial review of the pilot evaluation report by the International Teledemocracy Centre ( ITC) at Napier University (Whyte et al, 2006) confirmed that significant analysis of the pilot had already been conducted. The purpose of the evaluation framework was therefore to ensure that this evaluation added value and deepened the understanding of the contribution of the pilot to renewing local democratic procedures and engagement, thus allowing the Scottish Executive to make fully informed decisions about potential rollout.
1.5 The evaluation framework took into account several key issues which are noted in the wider literature, and was used as the basis for the development of research tools and analysis of primary research. The evaluation framework is presented in Annex One.
Methodology
1.6 The evaluation methodology involved the stages of research outlined below. Full details of our methodological approach are outlined in Annex Two.
A desk review of pilot project information including secondary data, reports and existing evaluations
1.7 We reviewed a set of project documentation provided by the Scottish Executive including project plans, financial statements and progress and final reports. A full list of materials reviewed is included in the reference list following chapter eight. This aimed to explore the project management aspects of the pilot project and assess whether the project achieved its aims and objectives and is presented in chapter two.
A review of web traffic / usage statistics and content analysis
1.8 We conducted a review of web statistics provided by ITC for the three months that the pilot project was functioning to the date of the evaluation (Sept 2005 - Aug 2006) as well as content-analysing a sample of messages selected from the pilot website. The purpose was to explore the usage of the website and, where possible, compare this with relevant comparator websites. Messages were analysed to assess the extent to which the website was improving community participation and enhancing local democratic processes. Findings are presented in chapters three and four.
A desk review of external literatures and websites relating to other e-community engagement initiatives
1.9 We reviewed a selection of community participation websites from across Scotland and the rest of the UK in order to draw comparisons with the pilot website. A full list of these websites is provided in Annex Three. This aimed to place the pilot project within the wider context of e-democracy initiatives helping to draw conclusions about the possibility of rolling out the pilot project. An inventory of websites was drawn up and then a sample of websites was contacted for further information. Findings are presented in chapter five.
A cost-benefit analysis of the pilot toolkit
1.10 We reviewed financial information for the pilot project, provided by the Scottish Executive, to assess the value for money delivered by the project. More details of our approach to this are presented in Annex Four. We compared the pilot project with other models available for similar websites with other forms of e-engagement. Findings are presented in chapter six.
In-depth interviews with pilot partners and stakeholders
1.11 The purpose of the in-depth interviews was to explore stakeholders' perceptions of the pilot project and provide detail on each of the key issues in the evaluation framework. We conducted a series of eleven in-depth interviews with pilot partners and stakeholders. Interviews were conducted with representatives from Napier University's International Teledemocracy Centre ( ITC), the members of the six CCs (Bannockburn, Cambusbarron, Strathfillan, Thornhill & Blairdrummond, Drymen and Torbrex), the Stirling Assembly, the Association of Community Councils for the Loch Lomond and Trossachs National Park area and the Association of Scottish Community Councils. We also interviewed a representative of the Scottish Executive in order to place the pilot project within the wider strategic context.
Interviews with the local community
1.12 We conducted 15 semi-structured interviews with members of the local community who had used the pilot website, as well those who had not, in order to build an understanding of how effective the pilot was in engaging with the local community. Interviews were conducted by telephone and are presented in chapter seven.
Structure of this report
1.13 The remainder of this report is structured as follows:
Chapter 2 provides an overview of the background and context of the pilot project;
Chapter 3 presents our findings from our technical review of the pilot website including web traffic and usage statistics
Chapter 4 presents the findings from the content analysis of messages;
Chapter 5 looks at comparator websites to locate the evaluation findings in the context of existing knowledge of, and research about, renewing local democracy through the use of IT in the UK;
Chapter 6 looks at the cost-benefit analysis of the pilot project drawing on comparative financial models;
Chapter 7 presents the community's views of the pilot website; and
Chapter 8 presents the conclusions and recommendations from the findings.
1.14 A reference list is presented following chapter eight. Annex one presents the evaluation framework and annex two presents the full details of our methodology. Annex three contains a full list of the external website we reviewed and annex four gives details of our approach to the cost benefit analysis. Annex five provides a glossary and list of abbreviations.
« Previous | Contents | Next »