The Scottish Government

CHALLENGES

SOLUTIONS

the concept of satisfaction

As summed up by Qureshi & Rowland "in order to obtain meaningful and useful feedback from service users it is important to recognise that satisfaction is multi-dimensional, that satisfaction ratings are affected by life circumstances and characteristics, and that the aim of services is not just to satisfy individual users." (Qureshi & Rowland 2003, page 15)
"Satisfaction" with public services can not equate with consumerist ratings of commercial services as quality indicators for reasons relating to widely varying expectations of standards, and to diverse and opposing interests of different stakeholders.
(Bauld et al, 2000 ; Qureshi & Rowland, 2003 )
"Satisfaction" as expressed by older service users has been found to relate to their personal circumstances (for example poor health, or positive outlook) as well as to more objective aspects of services.
Older service users are more likely to give positive responses for reasons including low expectation, fear of reprisal, lack of knowledge, reluctance to criticise.
(Bauld et al, 2000)

Global measures of satisfaction and standardised ratings should be avoided or at least supplemented by specific questions about particular services or aspects of care
Avoid reliance on closed questions only
Use approaches that will allow users to express their own priorities.
(Bauld et al, 2000 )
"Move away from satisfaction ratings per se, and develop questions which are meaningful and relevant to service users and their experience, and which reflect accepted ideas about service quality (derived from research with users and other stakeholders)"
Assess satisfaction on a regular basis rather than relying on one-off cross sectional surveys
(Bauld et al, 2000)
Incorporate mixed methods in design of satisfaction surveys (Bauld et al, 2000; Qureshi & Rowland, 2003 )

setting standards

Definition of satisfaction criteria has to bridge a gulf between 2 different conceptual and practical worlds - of services and of people who use them.
(Nocon & Qureshi, 1996; Nocon et al, 1997)
The need for explicit standards, to achieve scientific rigour, without being unnecessarily complex
Individual perspectives and diversity of expectations
(Bamford et al, 1999 )
The need to match the approach to collecting outcomes information to the type of intervention
(Bamford et al, 1999 ; Qureshi et al, 1998)

A development process which comprises
consultation with service users and carers and other stakeholders
use of the available knowledge base of "possible appropriate measures and impact of particular services"
development of further measures which are "valid, reliable, sensitive to change". (Nocon and Qureshi, 1996)
Range of stakeholders involved in standard setting, diverse service users and carers and staff at different levels, discussions carefully conducted and recorded, set out standards frameworks consisting of service process outcomes (i.e. desired impacts on individuals of way services are delivered, e.g. treatment by staff) and ultimate outcomes (i.e. desired impacts or end results of service interventions on individuals' wellbeing and quality of life)
"Process" outcomes, of particular relevance to this stage of measuring satisfaction with the assessment process, were agreed in projects involving people with physical disabilities, older people with diverse needs and backgrounds and carers (Bamford et al, 1999 and Qureshi et al , 1998).
Desired process outcomes were
being valued and treated with respect
being treated as an individual
having 'a say' in services
services having a good fit with existing sources of assistance or with family help, which should not be assumed
good value for money
a positive relationship with frontline staff (disabled people's project)
a good fit with cultural and religious preferences and requirements
Process outcomes agreed by carers included
a shared sense of responsibility
being valued and treated with respect
having 'a say' in services,
confidence in the quality of services and getting value for money
having a positive relationship with staff.
Build on the consensus achieved in defining desirable outcomes and on their "consistency with stated policy objectives of community care" to set specific standards (Bamford et al, 1999 page28).

Assessment

General lack of awareness and understanding of the assessment process (Arksey et al, 2000)
Assessment often happens at a time of traumatic change
People need support to make decisions, in terms of desired life outcomes (Nocon et al, 1997)
Underdevelopment of carers' assessments and the risk of separate assessment of carers detracting from the effectiveness of holistic assessments of the service user (Qureshi et al,2001)

use an outcomes approach within the assessment process itself which equates with holistic needs and goals identified by service users and carers (Qureshi ed., 2001)
how services are delivered (treatment as a person, fit with individual's life) are important aspects to service users and carers (as above Standard setting)

Constraints on collection of information from community care service user

People in receipt of community care services are likely to include a high proportion who are unable to respond to self administered questionnaires, which offer the most economical means of collecting information from a large number of people.
A significant proportion of people affected by cognitive impairment will be unable to contribute data which can be quantified for purposes of performance indicators or comparisons between community care authorities.
(Qureshi & Rowland, 2003)
Lack of consensus amongst service users and carers of preferred ways of collecting their views, with majority preference for individual interviews conflicting with general aim to minimise cost ((Bamford et al, 1999)

Maximise reliable use of self administered questionnaires by ensuring that questions are based on satisfaction criteria important to service users, that they are well designed and that there is consistent understanding of the questions asked.
Good design included questionnaires which were
based on "social exchange" principles, to "create trust, increase rewards and reduce social costs for respondents"
clear and consistent in layout and questioning approach.
Final testing of respondents' understanding of the questions (see below)by cognitive interviews to minimise possible ambiguities of wording.
Have realistic expectations about return rates for self administered questionnaires, develop a range of other methods including adaptation of the questionnaire as an interview schedule (home interviews were the majority preference found by SPRU), accept that data collected by more individualised means from hard to reach users may not be easily quantifiable.
Resolve tension between need for national performance indicators and local detailed information by setting a few compulsory questions and the rest to be decided as useful for local circumstances.
(Qureshi & Rowland, 2003)

Establishing reliable tests for survey methods

Pilot surveys carried out in one authority area may not reliably be extrapolated to other areas where organisational structures and working cultures prevail. Conventional field tests of questionnaires do not reveal whether respondents understand and answer questions in the way intended by researchers. (Qureshi & Rowland, 2003)

Focus on testing respondent responses to questions, with reference to cognitive processes involved in answering (understand question, recall information, assess information in relation to question, answer question). (Qureshi & Rowland, 2003)

Inclusive communication

Assumptions of what is "normal"
Lack of understanding of the complex barriers
Competing priorities
Undeveloped financial and practical resources
Patchiness of resources between areas
(McPake and Johnstone, 2002)

See Sources & Resources section of the Toolkit