« Previous | Contents | Next »
Listen
4. Evaluation Framework
Leadership
Lessons from Phase 1 indicate that clearer leadership on
evaluation and greater clarity of evaluation and monitoring
roles are essential. These issues have been addressed
within the new staffing and reporting structures. Building
on the proposal set out in Annex 4, progress has been made
in the development of the evaluation approach for Phase 2.
The Scottish Executive Health Department and
NHS Health Scotland have shown further
commitment by ensuring that
HaHP has support with evaluation. The
specific deliverables required of
RUHBC (Research Unit in Health,
Behaviour and Change) and
NHS Health Scotland have been defined as
follows:
- Senior academic support in the design and
implementation of the Phase 2 evaluation framework,
taking account of existing preliminary progress
- Evaluation leadership within the project which
would include:
- performance management of the
evaluation plan using a suitable monitoring system and
linking with project leads
- capacity building within the project
by means of supporting project leads in developing
appropriate monitoring and evaluation plans and by
developing report writing expertise across the
project
- lead on a particular research aspect
within Phase 2
- lead on commissioning any external
evaluations
- a research and evaluation networking
role with others, both local and national.
The evaluation framework will be firmed up following
discussions with the new Head of Evaluation at
RUHBC and the Head of Evaluation at
NHS Health Scotland in May 2005.
It is proposed that the evaluation in Phase 2 will focus
on structure (the attributes of the setting in which the
intervention occurred), process (how the intervention was
organised, delivered and used) and outcome (the impact of
the intervention) (Donabedian 1980) (see Annex 5). The
outcome measures represent the evaluation endpoints but
qualitative and quantitative descriptive data on structure
and process will also be necessary to explore whether, and
how, the outcome was caused by the intervention itself,
and/or by variations in structure, or the way it was
organised or delivered (process). Process measures refer to
accessibility of the interventions and the quantity and
quality (numbers, type and suitability) of intervention
outputs. Both user and health professional perceptions of
satisfaction will be sought to explore quality.
Evaluation of structure will involve description of the
quantity and quality (numbers, type and suitability) of the
inputs, or resources, required to deliver the intervention.
Inputs might include the distribution of staff, their mix
in relation to level of training, grade and skill,
availability, siting and type of buildings, facilities and
equipment, numbers and types of services, consumables used
and other types of capital and financial resources.
Information about these aspects of the intervention is
particularly important for learning to be successfully
shared in other areas.
Monitoring
Performance frameworks will be established to monitor
progress towards achievement of the evaluation endpoints,
i.e. outcome measures. This will enable the
project to identify and respond to performance issues
within its life course thereby optimising capacity to
deliver on outcome targets.
Performance management
A variety of methods have been used that enable linkage
of planned activities and interventions to high-level
outcomes. A grid is in development that will form the basis
of an integrated method of performance management where key
performance indicators are clearly linked to individual
activities and interventions and a performance audit trail
created. In this way changes in high-level outcomes can be
tracked back to specific initiatives and interventions.
« Previous | Contents | Next »