« Previous | Contents | Next »
Listen
DEVELOPMENT OF TOOLS TO MEASURE SERVICE USER AND CARER SATISFACTION WITH SINGLE SHARED ASSESSMENT
CHAPTER SIX DEVELOPING THE TOOLKIT
6.1 The development and testing of tools and methodologies to assess user and carer satisfaction with their single shared assessment was a key objective of the commission.
6.2 The task was to produce methods, guidance and information which would enable evaluators to include service users and carers in the evaluation of SSA. Practitioners were consulted but the inclusion of other stakeholders was beyond the scope of this project. The Scottish Executive was clear that the tools and methodologies would be based on service users' and carers' own perceptions of their needs and experiences and they would accommodate a range of modes of communication.
6.3 The overall approach, ideas being developed and some specific sections of "the tools and methodologies" which came to be called "The Toolkit", were discussed in detail with the Project Steering Group.
6.4 In developing the Toolkit, a number of requirements had to be met. These included:
The Toolkit, of course, had to be deliverable; this meant that it had to accommodate the wide variation that existed at this time in the implementation of single shared assessment across Scotland. These variations included differences in terminology (some partnerships were not using the term SSA) and different interpretations for operational purposes of what constituted a SSA, as well as very significant differences in the pace and pattern of roll out of the SSA to all community care groups
It had to be feasible, i.e. consist of validated methodologies and approaches for evaluation which would be affordable to put into practice and practical - and if possible be perceived as an attractive option
It would be used with anyone who uses community care services from any section of the community and so must accommodate a vast diversity of individual circumstances
It had to provide methods which could be used systematically which would enable community care authorities to collect usable data and which was also consistent with an individualised approach to service users and carers
It had to successfully address the challenge of helping people to recall and talk about events which professionals call assessments but which for an individual may have been a chat or an unimportant route to getting a service or forgotten altogether
It had to incorporate best practice in terms of modes of communication and use of communication aids.
6.5 Decisions about methods and approaches and the content of the information and guidance which the draft Toolkit contained were made with all these requirements very much in mind. In this section we provide a commentary on some key sections of the Toolkit. The process of testing the Toolkit and other steps taken to validate the methods are described in the following 2 chapters.
Evaluation methods
6.6 4 evaluation methods are included in the Toolkit, i.e. for service users - a self administered questionnaire, an interview and the Guided Discussion and for carers, a questionnaire which could be self administered or used as an interview schedule.
6.7 Self completion questionnaires are attractive on the grounds of affordability and practicality, being cheaper than other methods to apply and easier to manage. Whilst there is evidence (see literature review, Chapter 3 above) that questionnaires are often not the preferred format for service users and carers and that responses may be based on inaccurate interpretation of the meaning of questions, we believed it was essential to include this method as an option. We followed guidance on good questionnaire design (e.g. Quereshi and Rowlands, 2003) as closely as possible, for example ensuring consistency of layout and questioning. We used straightforward questions and in the explanation of the questionnaire made clear that the questions were based on the views and experience of service users and carers about what matters in assessment. The final format of the questionnaire grouped the questions into 4 sections, each of which began with some general questions, followed by several more specific questions. The intention here was to allow the service user to answer only the general questions if they were unwilling to answer all the detailed questions in each section.
6.8 The Interview schedule consists of the same questions as the questionnaire. This ensures coherence in the evaluation and helps with data collation and analysis. In addition, the Questionnaire itself can be used for interviews; this cuts down on documentation. The format of the schedule, as described in the preceding paragraph, allowed the interviewer to exercise judgement in the number of questions they asked a service user: if the service user appeared to find the more detailed questions irrelevant, irritating or simply tiresome, the interviewer could focus solely on the general questions.
6.9 The Guided Discussion is a short schedule of 5 broad questions. The questions are based on the priority areas for enquiry which service users identified and broadly reflect the domains used in the questionnaire/interview schedule. This method was included because for some community care users the 2 other methods would be unhelpful and impracticable. The Guided Discussion is designed to be used by skilled, experienced interviewers and require interviewers/evaluators to ensure that they find out about and meet the communication requirements of each person. This method should help to ensure that people who are often excluded from research - are included; this may be with the help of a known person (e.g. advocate, family member) to assist with interpretation.
6.10 The carers' questionnaire was developed in recognition of the fact that an unpaid carer's role in the assessment of the person they cared for was important but different to that of the service user. The questionnaire was also designed to be used as an interview schedule should the carer prefer this method of enquiry. The carers' questionnaire was not designed to elicit views on a carer's own assessment.
Guidelines for assessors developed by service users and carers
6.11 The Guidelines for community care assessors were produced by the 4 local development groups (see page 76-83). In spelling out what would help to make assessments for services work well for service users and carers, the Guidelines provided the framework for the evaluation questions.
6.12 The work with groups made it clear to us that the way all assessments are carried out held considerably more relevance for service users and carers than the particular characteristics of the single shared assessment or the difference between the impact of the SSA process and that of its predecessors; the distinction in terms of name and process was of little interest. This is not to say that this would be the case for everyone who uses community care services. In the interests of ensuring that the Toolkit is inclusive and accessible, the guidelines and the information for service users and carers refer to assessments for community care and health services rather than to single shared assessments. Practitioners supported this practice.
6.13 The Guidelines are also intended to be relevant in evaluation interviews: for example, an interviewer might use one of the items in the Guidelines as a prompt for discussion with a service user or carer.
6.14 There has been a steady flow of enquiries about these Guidelines. Their wider applicability and relevance to the inclusion, participation and involvement agendas suggest that it may be appropriate to make these publicly available separately from the Toolkit as a whole. In the context of the Toolkit they are essential reading, as they represent in their totality the Local Development Groups' definition of "satisfaction".
Selection of respondents and consent issues
6.15 One of the guiding principles underpinning the development and design of the Toolkit is that everyone who has a SSA will be eligible to be involved in evaluating the SSA. In practice, involvement depends on being selected as a potential participant through a sampling process and on agreeing to take part.
6.16 The national survey brought in reports of at least one research project about SSA in which service users were asked for their views - with the exception of people with dementia, who were automatically excluded. The view taken for the Toolkit was that all SSA recipients would be automatically included.
6.17 Consent and capacity issues were considered very carefully and advice sought from expert colleagues as well as through other sources. The Adults with Incapacity Act, in particular the General Principles contained in the Act and the Explanatory Notes on the General Principles and definitions (pp 2-3) together with other guidance enabled us to develop clear guidance about asking for consent, what is meant by informed consent, the kinds of circumstances which would indicate that a carers' assent should be sought as a co-signatory to the consent form and when a carer would be asked to consent on behalf of the individual.
Summary
6.18 Key themes in developing the Toolkit were:
The Toolkit had to be both deliverable (i.e. able to accommodate different stages and patterns of implementation of Single Shared Assessment) and feasible (i.e. using valid methods which could be affordable and practical)
It had to be useable with anyone who used community care services
It had to use methods which would allow Joint Future Partnerships to collect usable data, whilst maintaining an individualised approach to service users and carers
The Toolkit included 4 evaluation methods to ensure it was inclusive: a self administered questionnaire; an interview; a Guided Discussion; and a carers' questionnaire which could be self administered or used as an interview schedule
Care was taken to make the questionnaire comprehensible and consistent in its layout. It was divided into 4 sections with general questions at the beginning of each section to allow service users to answer only the general questions if they wished
The Guided Discussion had 5 broad questions based on the priority areas for enquiry identified by service users: these areas broadly reflect the domains used in the other methods. It was intended for use by skilled interviewers where the other methods would be unhelpful or impractical for some service users
The Toolkit provided guidance on the use of the 4 methods, information on communication issues and advice on seeking consent. It also included the Guidelines for assessors developed by the Local Development Groups.
« Previous | Contents | Next »