The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 1.8. CONCLUSIONS: LESSONS FOR FUTURE RESEARCH

8.1 This literature review highlights a range of issues for defining and measuring QOL in relation to culture and sport. It flags up the paucity of empirical research in this area, and the need to fund well-designed, large scale research to explore and test the impact of culture and sport on QOL. The evaluation of the Culture Builds Community initiative in the US demonstrates the value of longitudinal studies that enable longer term outcomes to be identified. ³⁸⁷ While common in other policy areas, notably health, and in the US, where charitable foundations provide substantial funding for cultural policy research, research on the scale required is rarely sponsored within UK cultural policy. As it is, the findings of the studies reviewed here demonstrate that cultural and sport participation have a very small influence on the overall QOL of individuals.

The absence of research creates a theoretical vacuum, which poses a problem for developing indicators. One of the desired outcomes of this research was the establishment of social and economic indicators to measure QOL and well-being in relation to culture and sport. On the basis of current research evidence, this is obviously not possible. This is because:

• The evidence base supporting a causal link (rather than an association) between culture and sports participation and QOL does not yet exist. The economics literature suggests that the general benefits of participation and voluntarism for individual subjective well-being may apply to culture and sport, but as yet there is too little evidence to support a causal link. Elsewhere, the expectation of establishing a causal link has been questioned by experts, for reasons that are discussed below.
• Suitable datasets with which to investigate the link between culture and sports participation and subjective well-being have not yet been identified by economists ( See Annex 3);
• The small body of existing studies tend to concentrate on one or two individual aspects of QOL rather than embracing the concept of QOL as a whole. As a result we currently have just a partial view of the possible impact of culture and sports participation on QOL.
• Without this type of evidence it is difficult to develop meaningful indicators, ones that are rooted in a theory of cultural or sporting impact;
• The question of how indicators might take account of the quality of cultural and sports interventions has yet to be resolved.

8.2 However the review has highlighted two possible alternative approaches used elsewhere:

• to base indicators on areas where existing cultural impact research suggests an impact
• to regard cultural indicators as a "research tool" as demonstrated by Cultural Initiatives Silicon Valley, and proposed by Jackson and Herranz

"The context of community indicators may provide a research setting to develop 'theoretical or empirical research that speaks to how arts and cultural participation contribute to social dynamics'". ³⁸⁸

8.3 A key concern of public policy in all areas is the wish to demonstrate causality. The

review raises some important issues with respect to this. The findings of several of the sports/exercise and community level cultural studies certainly suggest that it is very difficult to identify a single cause relationship between participation and impact on QOL. They conclude that in practice a complex combination of variables are involved in determining impact, in other words, it is unlikely to be sporting or cultural participation alone that produces a particular outcome. In addition to this, Bowling and Zahava emphasise how

"…influencing variables can include a dynamic interplay between people's individual characteristics and their surrounding social structures…". ³⁸⁹

8.4 Another key issue for public policymakers is the need for research whose results can be extrapolated or generalised to the population as a whole. As we have seen, the majority of individual level studies of culture and sport and QOL do not allow this. In some of these cases this is explained by methodological weakness. However there is a much more fundamental issue about the generalisability of QOL research. As we have seen, QOL is a shifting, dynamic and culturally specific concept. In other words what contributes to the QOL of one person may change according to life stage, and circumstance. There will be differences in what constitutes QOL to populations living in countries at varying stages of economic development or with varying social and cultural values. Defining QOL involves ethical and political considerations and both community indicator and intellectual disability researchers stress the importance of identifying the dimensions of QOL that are important and valued either by a particular community or by individuals themselves. The relative importance of cultural participation to the QOL of individuals and communities may also vary widely. The scope for generalising from QOL research findings is therefore clearly limited. This is one of the drawbacks of the concept of QOL, and why some regard it as a not particularly helpful concept. ³⁹⁰

These points could lead us to the view that QOL is just not a fruitful subject for research: useful as an "organising concept", but just too complex to be "do-able". Alternatively, these points might lead to the view that the natural science research model, of which notions of "causality" and "generalisability" are part, may not be the most useful for this type of research subject. Perhaps the question to explore is, what other types of research approaches might best fit this purpose? The literature review points us in this direction, with the example of the grounded theory approach pioneered by The Urban Institute. Are there other research models that might help us study the underlying social mechanisms that determine cultural impact? For example, this might involve developing a theoretical model to explain the impact of culture or sport on QOL, or a dimension(s) of QOL, and both collecting data and using observation to test this out. ³⁹¹ Would these other types of approaches meet policymakers' needs?

8.5 The discussion of the issues involved in defining QOL leads to the conclusion that a "standard definition" of QOL for use in culture and sport research is perhaps not a realistic goal. As we have seen, there is no definitive definition of QOL that applies in all contexts. Instead this review has demonstrated that working definitions are shaped by research purposes and policy objectives. For this reason experts have focused instead on developing "guiding principles" for QOL researchers. ³⁹² So, while this review flags up some "bigger issues" to consider, it also points to more practical recommendations for future culture and sports research in this area. These might be viewed as lessons for "best practice":

• First, there is a requirement for conceptual clarity. Researchers need to state explicitly what kind of definition of QOL they are using, and what they intend to measure. In the first instance, they need to decide whether they are investigating QOL as a multi-dimensional or uni-dimensional concept. If the former, they need to decide whether they are investigating QOL as a whole, or whether they are focusing on one or more individual domains of QOL. If they are concerned with subjective well-being, using life satisfaction as an indicator, then they need to acknowledge that they are taking a partial view of QOL, focused on one aspect only. The same applies to studies focusing on social capital/community development.
• Related to this, if the objective is to pursue evidence of a single cause relationship between culture or sports participation and QOL, then the methodological challenges and complexities of measuring the multi-dimensional concept of QOL in its entirety, may mean that it is simpler in practice to measure subjective well-being alone, using satisfaction or happiness as a proxy. This is the route taken by most sports and culture studies reviewed here, but it does raise wider issues. Happiness research promotes the idea that this should be the ultimate objective of public policy, an idea taken up and promoted by the Cultural Commission in its Final Report. ³⁹³ Given what is known about the determining influence of psychological factors on perceived well-being, some experts have questioned whether "engineering gains in subjective quality of life" is a "realistic policy goal". ³⁹⁴ Moreover, there is a body of research that distinguishes between a (temporary) change in affect and lasting changes in levels of subjective well-being - how do we take this into account in research design, and, given that it rarely defines it, which type of well-being does public policy seek to address?
• Second, cultural and sports researchers need to consider how to define cultural participation. How culture is defined has a practical impact on the "do-ability" of research. Definitions of cultural activity or participation are either too narrow or "top down" to capture the full range of what people on the ground understand and engage with as culture, or they can be too broad for policy use and therefore compound the problems involved in identifying causality. ³⁹⁵ As Bygren et al have suggested: "Perhaps cultural behaviour is so intermingled with life as a whole that it is impossible to discern its influence". ³⁹⁶ This is an issue that has to be confronted.
• Thirdly, another issue is the need to counter any tendency to assume, or expectation, that every experience of cultural participation will produce similar outcomes. The way in which cultural or sports projects or events are organised, managed, delivered - all the variables that determine the nature or quality of the participants' experience - help to determine the outcomes. ³⁹⁷ This is rarely acknowledged in research design. Researchers need to consider how to allow for the variable quality of culture or sports projects in QOL research.

8.5 Finally, two pointers on the question of measurement.

• A key criticism of social impact research is the strong use of narrative and anecdotal evidence, and reliance on the "uncorroborated" self-report of participants. Given, as we have seen, that the subjective perceptions of subjects play a key role, sometimes the dominant role, in much QOL research, we should perhaps reflect on the standard dismissal of such data in the context of culture. The issue is perhaps not self-reporting, but how self-report data is collected, and whether this should involve more rigorous, theoretically based, measurement instruments, rather than reliance on a passive "yes/no" response to a prescribed list of statements into which participants have had no input. The application or adaptation of standardised instruments developed in other disciplines might be explored, but clearly there are many issues to be considered here that require specialist knowledge and expertise.
• A large number of QOL researchers argue that both combination and multiple methodologies best suit the multi-dimensional nature of the concept being studied. Such methodologies overcome the bias and constraints inherent in solely quantitative or qualitative techniques and produce stronger results on the basis of triangulating data from multiple sources. We need to explore more complex types of research design and learn from some of the large scale QOL studies conducted in policy areas such as health.

« Previous | Contents | Next »