The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER TWO METHODOLOGY

Introduction

2.1 The conceptual framework we have applied in our approach to this project and to assessing capability may be summed up as being based on the recognition that the real issue for policy makers, planners and managers is not disaster management but the management of disaster. Effective planning for and managing disasters relies principally upon generic management skills that will be found in any effective organisation. Competent and effective managers and personnel will usually be able to apply their existing skills, competencies and knowledge to emergencies and disasters, and do this effectively, if they know their goals, their responsibilities, the agencies with whom they have to work cooperatively and the context in which they are working.

2.2 We also subscribe to the following principles:

• Disaster management arrangements need to encompass all hazards, whatever their origin or cause. Many disaster management functions are relevant to many different types of disasters. We have recently heard the rubric 'not all hazards, but any hazard' and we agree with this position.
• Disaster management must be comprehensive, and address mitigation, prevention, preparedness, response and recovery activities. To this we could add risk assessment.
• Disaster management relies on an all-agencies approach, ensuring that all agencies with a role to play, a function to carry out or a need to meet are engaged in planning and operational management as appropriate.
• The informed and aware community is the fourth pillar and is demonstrably essential to effective disaster management. Local people need to aware of, and contribute to local risk assessments. They need to be engaged in planning, to contribute where possible to response activities and to be closely engaged in planning and managing recovery programmes.

2.3 This approach of relying on generic management skills recognises that most agencies involved in disaster management have this role as only a small and infrequent part of their responsibilities. Specific skill enhancement and sustenance is an ideal. It is often difficult in practice to achieve an ideal situation where managers and staff have day- to-day responsibilities to conduct and which may be based on statutory or contractual commitments.

Research ethics

2.4 As required in the Standard Conditions document we agreed to abide by the conditions relating to confidentiality and anonymity. We also agreed to abide by the ethical standards established by the Research Ethics Committee of Coventry University at all times.

Stage one methodology

2.5 In accordance with section 3.2 of the Research Specification, the following tasks were carried out during Stage one of this research:

• The preliminary analysis previously conducted by the SE, ( see Appendix 3 for the questionnaire) was reviewed and the existing data set was transferred from XL to SPSS. This enabled a more in-depth analysis of the coping capacities of the 8 Strategic Coordinating Groups, the SAS and MCA.
• A critical review of the literature including relevant reports and policy document was conducted along with a search of journal articles to identify and evaluate the risks and hazards faced by Scotland and the SCG areas in particular.
• An interim report was then submitted which included the critical analysis of the original data set and a matrix was later developed to provide an overview of these findings. They are provided in this report in summary.

Stage two methodology

2.6 As required in the Research Specification (section 3.5) for stage 2, the following research strategies were adopted:

• Continuation of literature search and review.
• It was intended to develop and administer a revised questionnaire to the SCGs, the SAS and MCA following feedback from the stage one report. Circumstances of time made this task impossible to complete. However a questionnaire focussed on origins, qualifications and experience was developed and distributed to participants at the annual conference of the Emergency Planning Society ( EPS) Scottish Branch.
• Contact with each of the Strategic Coordinating Groups, was established to
  • gather necessary data (through focus groups, observation and semi-structured interviews) covering areas identified as being 'resilience/capability gaps' in stage one of the project; and
  • clarify and further investigate aspects of each of the targeted groups' resilience capabilities.
• Scenarios were then developed and conducted in each of the SCG areas in order to test the issues outlined in 3.6 of the Specification pertaining, for example, to funding, exercise and training needs, and approaches to future reporting.
• Recommendations as to the most appropriate method for collecting and updating the data in the future at both Strategic Group and central ( SE) level, will be presented at the end of this report.

2.7 The following methods were used to enable both the researchers and the respondents to develop a greater understanding of the concepts involved, the players in the process, and ultimately of where the gaps may be and how these gaps could be closed:

• Informal interviews with key individuals from each of the SCGs.
• Case study assessment in the 8 police areas through the use of focus groups working on the chosen scenario. The scenario was based on the scoping of potential risks and impacts requiring scenario based discussion amongst key personnel from the 8 SCG areas.
• Site or SCG area visits to coincide with the focus groups for observation studies.

Designing the framework for assessment

2.8 In order to maximise the collection of information during the primary research process, a framework for assessment was needed to ensure that all the necessary areas were covered. The starting point for a framework was based on the Scottish Capabilities Programme workstreams. This Programme has been reviewed and a revised framework developed with specific reference to the workstreams identified in the research specification ( See Appendix 2 for Framework). This section illustrates how the framework for assessment has been developed and analyses how it has been used in the research.

Assessing capability

2.9 While internal capability assessments are likely to be undertaken at a tactical level within responder organisations, national level strategic assessments are not common. In 1997, the United States Federal Emergency Management Agency ( FEMA) developed and distributed its Capabilities Assessment of Readiness ( CAR) programme in order to evaluate capability at the state level and so ultimately construct an overall picture of the national forces and resources available to respond to (and recover from) civil emergencies and disasters. The CAR identifies thirteen emergency management functions ( EMFs), which are considered to be essential aspects of civil contingencies capability and each of these EMFs is further broken down into attributes that can be studied in more depth.

2.10 These EMFs were adopted as the context in which these research findings were analysed and the workstreams were then integrated with them. The EMFs utilised in FEMA's CAR are as follows:

• Laws & Authorities
• Hazard Identification & Risk Assessment
• Hazard Management
• Resource Management
• Planning
• Direction, Control & Coordination
• Communications & Warning
• Operations & Procedures
• Logistics & Facilities
• Training (& Experience)
• Exercises
• Public Education & Information
• Finance & Administration

The capability workstreams

2.11 Originally, the SE's research specification identified 7 workstreams intended to provide both structure and focus to the research. Following discussions with the SE the research aims were later extended to cover all of the workstreams. The workstreams identified in the original specification were:

• Mass Evacuation
• CBRN
• Animal Disease
• Human Disease
• Mass Casualties
• Mass Fatalities
• Warning, Alerting & Informing

2.12 These are referred to as functional in the Capabilities Programme. It is certainly clear that CBRN, Animal Disease and Human Disease are all hazard based workstreams, while Mass Evacuation, Mass Casualties and Mass fatalities are all consequence management workstreams. Notably, Warning, Informing & Alerting is an aspect that needs consideration with respect to each of the other workstreams and for this reason should be considered more in terms of a set of EMFs. By cross-referencing the EMFs against the remaining workstreams, an assessment matrix was generated within which capability attributes specific to the given situation have been identified.

Capability attributes

2.13 The capability attributes have been compiled from various documents relating to civil contingencies, emergency planning and disaster theory. While every effort has been taken to include the most salient strategic issues in the framework for assessment, it is inevitable that the matrix will continue to evolve as new literature is published and digested.

Applying the framework for assessment

2.14 The framework for assessment was used to inspire the methods of data collection during the scenarios/interviews and to assist in the analysis of the collated information.

Experience questionnaires

2.15 Qualifications, experience and training are personal attributes that need to be assessed individually before the data can be combined to provide a picture of the overall situation for Scotland and the SCG areas. A questionnaire was adopted to complete this task utilising a number of open and closed questions.

2.16 This section of the research was opportunistic and relied on the congregation of the EPS Scottish Branch at their annual conference for research subjects. The questionnaire itself ( see Appendix 5) was adapted from an assessment tool utilised for a previous research paper (Coles & Smith, 1997) as it was found that there was a parallel in the research objectives and proposed research population.

2.17 Initial contact was made with a representative of the EPS Scottish Branch to request their collaboration. A covering letter provided the details of the overall research project and the objectives of this specific research questionnaire. The covering letter and a copy of the questionnaire were then provided to each of the delegates who were requested to submit their completed forms during the conference or subsequently by mail. While it was felt that the conference provided an ideal opportunity to engage with a wide range of emergency management professionals, it was noteworthy that the sample only included those EPS members able to attend the conference.

2.18 After removal of personal information and coding/entry into the SPSS statistical package, the data was analysed and descriptive statistics generated to illustrate the current Qualifications & Experience situation.

Scenarios & participant observation

2.19 Three scenarios (later reduced to 2 - see Appendix 7) centred on Avian flu leading to a flu pandemic were proposed originally to progressively test resilience and capacity. The method employed relied on a 'hypothetical' (a directed focus group) approach where a basic scenario was given to the participants. They then discussed the scenario under the general guidance of 2 moderators. The moderators prompted and channelled participants where necessary, recorded key points and issues and maintained the schedule.

2.20 The participants drove the scenario, allowing it to unfold and evolve over the nominal time period. Interaction between participants was encouraged. Research participants included

• senior representatives of agencies represented on each SCG; and
• senior representatives of Local Government and major Voluntary Agencies, including the Red Cross WRVS and Churches (only the Salvation Army attended).

2.21 Eight workshops encompassing these scenarios took place, one in each of the SCG areas of Scotland and ran for approximately 3.5 hours each. It was planned that senior representatives from the various emergency services (category 1 and 2 responders where possible and they were in all cases) would be present along with local community representatives drawn from the voluntary sector including Red Cross and other local, major participants. However, no voluntary groups were represented in 3 of the case studies.

2.22 These sessions were facilitated by members of the research team (Graham Marsh and Philip Buckle) from Coventry University. Each scenario was hosted by the regional SCG. Confidentiality was maintained at all times.

2.23 The strengths of this approach were:

• the expert and local knowledge of participants was utilised;
• moderator bias was minimised;
• interactions between participants allowed testing of complex, systemic and system-wide issues;
• participant interaction became self-generating and self-sustaining and identified gaps, areas of agreement and system level issues where more direct approaches and those with only a few agencies as participants may not have generated the same insights;
• scenarios could be tailored in situ to particular workstreams; and
• knowledge gained from earlier scenarios could be incorporated into later scenarios hence allowing for comparisons to be made for a cross Scotland picture of capability to be developed.

2.24 The advantages of the use of this scenario method were that we could test an area where all services would be impacted by a flu pandemic with resources being stretched to the limit. However, this did mean that the scenarios were then limited by a focus on health matters. Some of the findings therefore emphasised a lack of medical staff for example, which may not be the case under normal emergencies unless they are cross boundary events.

2.25 As a result, the ability to deal with emergencies generated by natural disasters such as a flood was only tested to a lesser extent. In order to overcome this limitation, the sessions ended with a discussion on the strengths and weaknesses of service provision of the everyday delivery of their emergency services and what were the gaps, if any. The scenarios covered the implications for service delivery during an emergency/disaster if large numbers of staff were absent due to such a pandemic. This enabled not only capacity within and across the services to be tested but also business continuity. The topic of a flu pandemic was deemed to be appropriate because it was topical and all SCGs were aware of the need to plan for what was perceived to be a very likely occurrence. Planning was well underway as to how to deal with the most likely impacts.

Methodology

2.26 With scenarios as the chosen method for stage 2 of the research, the findings are based on the participants' interpretations of how they would respond to the particular event. No quantitative data was elicited and terms were not defined although we did seek clarification as to what was meant by particular terms. For example, no actual definition of 'cope' was provided by the participants but the interpretation we derived from the statement was that essential services would be delivered as usual, while less essential services, on the other hand, would be delivered at a reduced level.

2.27 All respondents were at pains to state that all available resources would be called up and less essential services suspended during a pandemic. From these responses it was assumed then that they would deal effectively with the range of situations with which they would be confronted, despite the high number of colleagues being absent. When they conveyed that they would be 'unable to cope', it would be when there were multiple events or fatalities/casualties beyond specified numbers (the typical response was "we could cope with hundreds but not thousands"); their resources were impacted; or when there was a cross boundary event and external resources were less readily available. 'Coping' then also assumed that resources could be called upon from other SCG areas or from central government - which included both the SE and Whitehall, depending on the circumstances - when necessary.

Interviews

2.28 It was intended that the scenarios would be followed up by semi structured interviews with selected participants to explore the issues raised at greater depth and for clarification where needed so allowing for a fuller assessment of capability and resilience. While the pressure of time on the participants on the day made this impossible to achieve, informal discussions were held with as many participants as possible before and after the sessions as well as during the breaks. Clarification of points raised during the scenarios was obtained in this manner and further discussion on the SCG area's capability was obtained.

Summary

This chapter has discussed the methodologies used in conducting stage one and stage 2 of the research. Key points are:

• Stage one findings are available in the Interim Report and included in this report in summary
• Stage 2 of the research adopted the FEMA's Capabilities Assessment of Readiness ( CAR) and it's EMFs as a framework for assessing Scotland's resilience
• Methodologies included an 'experience' questionnaire
• 'Hypothetical' scenarios were used as a method of collecting data

« Previous | Contents | Next »