The use of measurement indicators as a means for assessing the performance of implemented policies is well established. Indicators are also widely used as assessment criteria to allow decisions to be made between alternative investment options. The key to the successful use of measurement indicators is to ensure that:
The second point allows us to assess how practical it is to collect, examine and analyse the relevant data. It also enables us to assess the gaps between what should be measured and the practicality of what can be measured.
To ensure clarity on what needs to be measured, each objective has first been divided into a number of components each with its own detailed meaning. A measurement indicator has been attributed to each one of these detailed meanings.
The need to prioritise investment funding may require assessment of performance against the objectives as the basis on which to make decisions between alternative proposals. In this case, indicator scores will either need to be aggregated to provide an overall assessment of performance, perhaps using a system of weighting, or individual indicator scores will need to be presented so as to enable decision makers to form judgements on the relative performance of each proposal. There are various techniques by which this could be done. The former is the simpler approach but the granularity of the indicators is reduced and its usefulness depends on how the weightings are agreed between stakeholders. The latter will require development of a more complex approach using a technique based on multi-criteria analysis, sustainability appraisal and/or social cost-benefit analysis.
DETAILED MEANING | DRAFT MEASUREMENT INDICATOR
(qualifying comments in italics) | STATUS | NATIONAL SCALE MEASURE (qualifying comments in italics) |
|---|
Objective 1: Overall - Meet needs for flood resilience |
Reduce the total sum of flooding impacts over time, to an agreed level. Specifically: | Net sum of indicators (i), (ii), (iii) and (iv) Aggregate measure requires analysis of the distribution of reduced impacts (benefits) accrued across all future flood probabilities. | | Total unmet need quoted as number of properties, and % of population, remaining at risk at target level. Where there are differences of definition between the types of flooding, each could be quoted separately. It may be appropriate to gradually move to a common definition. Summing the components of the draft measurement indicator in £ appears to be the only way to aggregate the different kinds of flood impact to the national scale. |
(i) Personal social impact | (i) People at risk x personal social impact/person Impact costs for death and injury are used in Highways Agency roads assessment and social impact is being considered by Defra for inclusion in their appraisals. | Potentially measurable subject to research SE has commissioned a study on the social impacts of flooding |
(ii) Potential damage to personal, commercial or public property | (ii) (Personal, Commercial and/or Public) Property at risk x damage potential/property | Already practical standard practice; embedded in Defra's methodology |
(iii) Travel time losses | (iii) People at risk x lost travel time/person | Already practical standard practice; embedded in Defra's methodology |
(iv) Commercial and industrial losses including those due to disruptions to transport or other infrastructure | (iv) Jobs at risk x lost time/job + lost production & sales | Already practical standard practice; embedded in Defra's methodology |
Objective 2: Social - Enhance community benefit with appropriate access for everyone |
Enhanced community benefit is expressed in terms of: | … and measured in terms of: | | |
(i) The impact on local jobs | (i) No. of local jobs protected Needs to take account of changes in labour market activity that do not add to the total number of local jobs. | Practical | Cumulative national results of jobs protected, and total £ spent in communities. The draft measurement indicator can be aggregated across the different types of flooding to create a national measure. |
(ii) Wealth generated for local community. | (ii) £ (and percentage of proposal costs) spent in the local community | Potentially measurable |
(iii) All those affected having fair access to the benefits of sustainable flood management ( SFM) | (iii) Clear statement of who has access to which benefit | Facts | By categorising access for indicator (iii) and weighting by population, the draft measurement indicator can be aggregated across the different types of flooding to create a national measure. |
Objective 3: Environmental - Protect and work with the environment with respect for all species, habitats, landscapes, and built heritage |
The individual elements are to: | Measurement is based on two indicators of the water environment, (i) and (ii), and four indicators for the non-water environment (iii), (iv), (v) and (vi). It may be possible to combine the individual measures (at local level) into an 'Environmental footprint' | | |
(i) Prevent deterioration of, protect and enhance the water environment (quality, quantity, physical habitat and natural processes) | (i) Use indicators developed by SEPA for measuring the water quality, water quantity, and hydromorphological state of water bodies in terms of WFD objectives. | Measurable subject to the results of research | Cumulative national results of all implemented proposals, expressed in terms of percentage changes in the various WFD indicators. |
(ii) Protect and/or improve floodplain ecological habitats and landscapes | (ii) Statement of specific areas ( SSSIs, 'corridors', wetlands, landscape features, etc.) affected in terms of hectares (as per Defra's existing appraisal guidance) by percentage of: - Urban/agricultural land converted to catchment floodplain
- Human activities reclaimed by catchment wetlands
- Flow (or catchment rainfall) stored in impoundments with flow regulation functions
| Potentially measurable | Cumulative national results of all implemented proposals, expressed in terms either of areas or of percentages determined at local level . |
(iii) Deliver Biodiversity Action Plan targets, etc. | (iii) Percentage of targets achieved (The Scottish Biodiversity Forum is developing indicators to measure progress in implementing the Scottish Biodiversity Strategy) | Potentially measurable | Either aggregate the individual indicators, across all implemented proposals, to obtain 3 separate national measures, or use 'Environmental Footprinting' to create a single national measure expressed per person or property. A national indicator for GHG emissions should allow measurement in terms of the targeted reductions under the UK Emissions Trading Scheme. |
(iv) Minimise emissions of greenhouse gases | (iv) Million tonnes of all GHG emissions weighted by Global Warming Potential ( GWP) It is considered that materials use, waste production, and energy use could be encompassed within an overall measure of GHG emissions. | Potentially measurable |
(v) Deliver other targets associated with relevant non-water plans and programmes | (v) Use indicators specified in other plans/programmes consistent with the Schedule 2 topics and methodologies adopted in SEA and sustainability appraisal. | Potentially measurable, otherwise provide facts |
(vi) Provide other benefits through physical improvements to the urban and rural environment | (vi) Shadow price valuation of benefits from specific improvements in amenities and aesthetic appearance (open spaces, views, wetlands, landscape, etc). Willingness-to-pay approaches may enable benefits to be valued. The challenge is establish an indicator that can effectively capture the diversity of 'social capital' in perceived benefits and individual responses. | Potentially measurable
| Cumulative valuation of benefits, across all implemented proposals expressed per person or per property |
Objective 4: Economic - Deliver resilience at affordable cost with fair economic outcomes |
Achievement of this objective is expressed in terms of: | … and measured in terms of: | | |
(i) Delivering SFM for lowest whole life cost of alleviation (capital costs, operation & maintenance, replacement, individual response funding and compensation), plus awareness, avoidance and assistance costs. | (i) £ (Whole Life Cost) | Already practical (and included in C:B ratio calculations) | Cumulative national costs and number of properties affected for all implemented proposals Aggregation across the different types of flooding will create a national measure. |
(ii) A comparison and judgement on 'fairness' in terms of the causes of flooding and those who pay the cost | (ii)(a) A statement of the main causes of flood impacts which are to be reduced | Facts only required | Cumulative 'people affected' weighted average of 'fair', 'partly fair', or 'unfair'. By categorising the 'fairness' and weighting by population, an aggregate national measure can be created across the different types of flooding. |
(ii)(b) A broad assessment of the distribution of who pays the costs | Facts only required |
(ii)(c) A comparison of facts to provide an objective judgement of fairness ('fair', 'partly fair', 'unfair') or even-handedness as developed for WFD cost-effectiveness analysis. | Facts only required |
Objective 5: Future Generations - Allow for future adaptability with a fair balance between meeting present needs and those of future generations |
Achievement of this objective is expressed in terms of: | … and measured in terms of: | | |
(i) Allowance for increased precipitation, storms or sea level rise (as a result of climate change) | (i) Composite indicator (expressed as percentage of current conditions), which includes changes in precipitation, sea level rise and storminess, and is a proxy for more detailed analysis of predicted changes to flood hydrology. | Potentially measurable subject to the results of research | Cumulative net headroom as percentage of design total, weighted by catchment area. If net headroom, for each flooding type, can be expressed as a percentage then it should be possible to aggregate a national measure, by using an appropriate weighting. |
(ii) Allowance for increased or decreased 'catchment' response Defined around man-made causes of river flooding - the word 'catchment' needs to be re-interpreted for other types of flooding. | (ii) Change in average runoff coefficient (expressed as percentage of current) Accompanied by qualitative statements concerning changes in land use. | Potentially measurable subject to the results of research |
(iii) Headroom, as the net result of (i) and (ii) | (iii) Net headroom expressed as percentage of design total Accompanied by a statement of what is involved in securing the headroom allowance. | Potentially measurable subject to the results of research |
(iv) Maximising natural capacity for flood response Although defined around river flooding, it can be re-interpreted for other types of flooding. | (iv) Flood plain storage (m 3 and %age of 'natural' floodplain retained), and floodplain channel capacity (m3/s and %age of 'natural' channel retained) Also incorporate a measure to take account of changes in the flow regime. | Potentially measurable subject to the results of research | Percent of 'natural capacity' retained, for each type of flooding, weighted by area impacted by flooding. |
(v) Future affordability of continuing resilience | (v) Composite indicator taking into account design life, discount rate and predicted change in local income/wealth over time Although Treasury Green Book requires use of 3.5% discount rate, this may not properly reflect this objective. | Potentially measurable subject to the results of research | Cost-weighted average |
(vi) Fail safe provision for design flood exceedence | (vi) Statement of the expectation, and estimate of the relative effect of exceedence - catastrophic, severe, minor. | Facts? | Aggregated sum of exceedence risks weighted by population served by each proposal |