The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 2: METHODS AND ANALYSIS

2.1 A number of methods were employed in the study including background interviews, questionnaires and telephone interviews with relevant staff.

ETHICAL REVIEW

2.2 The study was reviewed and approved by the University of Glasgow Faculty of Medicine Research Ethics Committee, a sub-committee of the University of Glasgow Ethics Committee for Non-clinical Research on Human Subjects.

BACKGROUND INTERVIEWS

2.3 Preliminary discussions were carried out with a range of groups/individuals, to establish themes and understanding of specific responsibilities. It was important to speak to people with knowledge of, or insight into, the implementation of the MHCT Act. The interviews were arranged either by the Scottish Executive project manager or the research team. They comprised three specific groups:

• Individual organisations (Mental Health Tribunal for Scotland, Royal College of Psychiatrists, Scottish Legal Aid Board and the State Hospital).
• JLIP co-ordinators
• Individuals who were either suggested by people interviewed in the previous two groups or who approached a member of the research team directly (including psychiatrists, GPs and MHOs).

2.4 The geographical areas from which people responded either as JLIP co-ordinators or individuals were: Aberdeenshire Council, Aberdeen City Council, Edinburgh City Council, Highland Council, North Lanarkshire Council, NHS Ayrshire and Arran, NHS Fife, NHS Dumfries and Galloway, NHS Greater Glasgow & Clyde, NHS Lothian, Orkney Council and, Scottish Borders Council.

2.5 The main questions addressed in these discussions were:

• What were the main issues arising from the current operation of the MHCT Act that should be addressed?
• What was the effect of the MHCT Act on staff time/costs?
• What was the effect of the MHCT Act on working with voluntary patients?
• Which staff group(s), if any, had the most additional work since the implementation of the MHCT Act?
• What information is recorded locally (re: compulsory powers) and is available?
• Were any other work activities being affected by MHCT Act work?

QUESTIONNAIRES

Questionnaire development

2.6 Questionnaires were developed to cover the tasks carried out by psychiatrists, MHOs (and GPs working on emergency certificates) for each of the orders. A further questionnaire was developed for medical records staff.

2.7 A mapping exercise was carried out which involved reviewing the MHCT Act and the accompanying Code of Practice to chart key tasks and those responsible for carrying them out. There was also an attempt to differentiate between what was a requirement to fulfill the conditions of the MHCT Act in relation to detention and compulsory treatment and which activities might be considered good practice. Also, duties and responsibilities under the MHCT Act were different (and possibly additional) but not 'new' in the sense that the 1984 Act also put duties and responsibilities on staff. There was an attempt to get staff to quantify duties and time taken under the 1984 Act.

2.8 Draft questionnaires were developed based on issues raised in background discussions and circulated to staff who had been involved in the implementation of the MHCT Act. Discussions then took place with staff groups to ascertain whether or not the tasks identified reflected their experience. In addition further deliberations took place over the nature of tasks ( e.g. were they good practice or were they routine practice which would happen regardless of the MHCT Act). Questionnaires were further reviewed through the Scottish Executive policy staff within the Mental Health Division.

Participants

2.9 Although some staff may have had some experience of MHCT Act work this would not always be the case. It was not appropriate, therefore, to send out questionnaires to a random sample. People were targeted who had been involved in a specific order previous to the questionnaire being sent.

2.10 Contact details for staff involved in each of the orders were obtained from the Mental Welfare Commission for Scotland ( MWC) (emergency detention certificates), the Mental Health Tribunal for Scotland ( MHTS) (short-term certificates, CTOs, medical records staff) and the Scottish Executive (compulsion/restriction orders and transfer for treatment orders). A breakdown of returned questionnaires can be found in chapter 4 (see Table 4.1).

2.11 Originally it was proposed to include questionnaires relating to appeals against excessive security but this was not possible within this study. The patient is the applicant in this situation (rather than a member of staff). From the outset it was not expected that this study would involve patients. A further complicating factor was that this part of the MHCT Act did not come into operation until May 2006. At the time of the study it was not clear that any appeals would have been completed for the workload to be assessed.

Procedure

2.12 Emergency certificate, short-term certificate and CTO questionnaires were sent to 2 psychiatrists (or GPs in the case of emergency certificates) and 2 MHOs in each local authority) while thirty psychiatrists were sent copies of the CORO and TTD questionnaires. MHO details were not held in these instances so psychiatrists were asked to pass on the name of the MHO they had worked with during an order to the researchers, so that a questionnaire could be sent to them. Twenty-four questionnaires were sent to medical records staff.

2.13 Postal questionnaires were sent between July and September 2006 . Questionnaires on different orders were sent out at separate periods (based on when access to staff contact details were available). A return date of 3 weeks was given in each instance. A stamped addressed envelope was supplied to return questionnaires. In addition people were invited to take part in a telephone interview. For an interview to take place a consent form and request for contact telephone details was sent with the questionnaire.

2.14 Due to a low response rate the project was extended. Reminder questionnaires were sent out in January and February 2007 to those involved in emergency certificates, short-term certificates and CTOs. It was decided not to repeat the questionnaire study for CORO and TTDs because of the low number of orders at the time the research was being carried out and very few people had experience of carrying one through.

TELEPHONE INTERVIEWS

Interview development

2.15 Interview schedules were designed to expand on the questionnaire data collected as well as to ask more specific questions about the impact of the MHCT Act on individual workloads. The main issues discussed were:

• To what extent did the planning for implementation of the MHCT Act locally meet needs?
• Were sufficient resources identified prior to implementation to meet any projected needs?
• What additional resources, if any, should have been considered/provided?
• What specific tasks make the process of application take longer?
• What has been the impact on the use of your time as well as other people's time? (who is taking over the work that you are no longer able to carry out? do you think there will be an impact on services from this type of practice?)
• Have you been using your own time (unpaid) to cover workloads?
• Has there been an impact on other groups of patients because of the increased workload associated with the MHCT Act? ( e.g. canceling clinics, increased waiting lists)
• How accessible have you found the application forms?
• Do you have a specific place where details of named persons and advocates are recorded? How easy is it for you to access these details?
• How accessible are these details to people working in other disciplines ( e.g. health and /or social work and to out of hours services?)
• Has there been a role in assisting and advising colleagues on the specifics of the Act (has this been your experience?)
• To what extent do you feel that the principles of the MHCT Act have been put into practice?
• Have the processes and outcomes under the MHCT Act been better for service users in general?
• To what extent has the implementation of the MHCT Act impacted on the overall psychiatric/ MHO/medical records workforces?
• Do you feel that your priorities have changed in relation to your work because of the MHCT Act? (in what ways?)

Interview participants

2.16 The majority of interviews were carried out between September and October 2006 All were carried out over the telephone and notes were taken and transcribed.

2.17 A total of 42 interviews were carried out with consultant psychiatrists, MHOs and GPs across Scotland. Details of the professions and areas of the interviewees are given in Table 2.1. These interviews were with those people who had returned a consent form with a completed questionnaire as well as a small number who contacted the research team via telephone (having missed the deadline to complete the questionnaire) who subsequently consented to be interviewed.

Table 2.1: Breakdown of interview participants

* This group consists of a range of posts with an MHO function including CMHTs and specialist social work mental health posts. In all of them the MHO function is part of the workload, but they also have a range of other responsibilities including care management.

2.18 As can been seen from Table 2.1 above, interviews were undertaken with MHOs and psychiatrists and a small number of GPs. The majority of partnership areas were represented in the sample and there was a mix of rural and urban areas covered. Five of the 6 compulsory orders were represented in the sample (not appeals against excessive security, as previously mentioned in paragraph 2.11).

2.19 Notes taken from the background interviews and those following the questionnaire were coded and analysed for themes using content analysis. Results were combined as similar themes came out of both sets of interviews.

TIME AND COST ANALYSIS

Questionnaires

2.20 A time and cost analysis was carried out on the questionnaire data collected. Each returned questionnaire was entered into and analysed using SPSS.

Time commitment

2.21 Each type of compulsory order was considered in isolation. In addition analysis was undertaken separately for MHOs, medical professionals and medical records staff. This was because of the specific duties each has under the MHCT Act. It does not imply the time commitment or opportunity cost of their time should be compared. The data was manipulated to provide an estimate of the average (and maximum and minimum) time commitments for each specified task. Individual tasks time commitments were then summed and the average (and range) of each stage of an order was estimated (e.g revoking orders, extending orders, suspending orders). The variation in time commitments between the MHCT and the 1984 Acts were compared (where possible) using a paired samples t-test. It should be noted that it was not always possible to conduct a pair-wise comparison when the sample size was small (see chapter 4, paragraphs 4.28-4.38 for clarification).

2.22 A separate questionnaire was sent to medical records staff. This was an open questionnaire where within each section respondents were able to describe the different tasks they carried out for each order.

2.23 The total time commitment for each stage of an order under the MHCT and the 1984 Acts was calculated by summing the time commitment of medical professionals, MHOs and medical records staff. Due to the small sample size, and less exposure to subsequent stages of each order, total time commitment for every order (that is an aggregate of every stage) has not been produced.

Opportunity cost

2.24 Opportunity cost is the foregone benefit of the next best alternative use of resources. While the additional time spent undertaking the new tasks of the MHCT Act could have been spent treating patients in an outpatient clinic or doing some other activity, we did not explicitly ask in the questionnaire what other activities the medics/ MHOs would have been doing, so do not know what opportunities were foregone. However, it is possible to place a monetary value on these (as the costing exercise did), and as such what is expressed is an opportunity cost (in monetary terms, rather that alternative benefit terms). The cost of each stage of an order was estimated by multiplying each respondent's time commitment by the appropriate unit cost. Unit costs employed in the analysis are given in Table 2.2 below. Comparisons between the MHCT and 1984 Acts were undertaken using a paired samples t-test.

2.25 One approach to estimating the cost of staff time is to undertake a primary costing study. However, this is an extensive task which was outwith the scope of the study. Instead published unit costs were used (Netten and Curtis, 2006). Unit costs are estimated from NHS Trust and PCT financial returns in England, and consideration is required to use them in a Scottish context. Notably, the consultant contract is the same across England and Scotland, so the use of this data should not introduce bias in estimating the time costs of medical practitioners (although the GP contract is different between Scotland and England, but few GPs responded to the questionnaire).

2.26 England does not have a defined MHO role (the Approved Social Worker ( ASW) is similar), so it was necessary to estimate the unit cost of an MHOs time. In order to do this, the current range of MHO salaries was considered (Paul Noyes, personal communication) and PSSRU methodology for estimating unit costs was applied to the mid-point salary of £29,349 (range £27,039 to £32,658). The PSSRU methodology involves summing the salary, salary oncosts (national insurance contributions and superannuation), overheads (15% of salary costs for management and administrative overheads) and capital overheads (a fixed amount of £2,253 based on new build costs) and dividing this by the number of hours worked (37.5 hours per week for 42 weeks). This provided a unit cost of £29 per hour (not dissimilar to the hourly cost of an ASW (£30).

2.27 To estimate the total cost of each order required a summation of the costs of staff time with the costs of the Tribunal service and multiplying this by the number of orders/hearings. Likewise, it should also be possible to estimate the additional cost of the MHCT Act, by multiplying the additional opportunity costs by the change (increase or decrease) in orders/hearings, and adding the cost of the Tribunal service to this. Such an analysis requires information on the cost of the Tribunal and the number of orders/hearings. The costs discussed are conservative costs as they do not include the cost of the Tribunal service or the opportunity cost of the time commitments of other professionals who attend hearings. This information was not available at the time the research was carried out. Therefore it was not possible to estimate the total cost of each order at this time (for more information see chapter 7, paragraph 7.9). Costs here are based on the time commitment for psychiatrists, MHOs and medical records staff.

Table 2.2 Unit cost information

2.28 Cost estimations were subject to sensitivity analysis. In the first instance the unit costs employed were changed and the values with qualification costs (or monetary incentives in the case of MHOs) included were used. The use of costs which include qualification costs (that is investment costs) can be important when estimating the cost effectiveness of different approaches (in this instance the difference in cost between the MHCT and 1984 Acts) as one might expect the professional mix to change. In the second instance the estimation of total costs employed a range of costs and quantities.

Other information

2.29 During the course of this work it became clear that a number of other individuals, groups or organizations were also collecting information on aspects of the MHCT Act, including workforce issues. In some cases this information has been shared and has allowed for some comparison. Both the Mental Welfare Commission and the Mental Health Tribunal have published figures which have been used.

« Previous | Contents | Next »