The Scottish Government

« Previous | Contents | Next »

Listen

Tracking Homelessness: A Feasibility Study

CHAPTER FOUR: POTENTIAL RESEARCH DESIGNS

4.1 This chapter describes different approaches to collecting data about homeless people. These approaches have been developed over the course of the project to address two main criteria: to address areas of policy interest; and to reflect what is feasible in practice. There has been a particular emphasis on means of obtaining information about groups of homeless people who have been identified by the Scottish Executive as of particular interest and for whom there is currently a lack of information. These groups include hidden homeless people and people at risk of homelessness.

4.2 A major factor in undertaking any research on homelessness would be obtaining the samples of homeless people. Various sources for obtaining samples of the different groups of homeless people have been considered for this project. Because generating a sample frame from scratch is a labour-intensive and difficult (at times, impossible) process, using already available lists of (ex-) homeless people, for example from administrative data currently collected in Scotland or from surveys already undertaken, is desirable. Hence most of the approaches recommended make use of these information sources.

4.3 Research approaches that are considered in this chapter include: the use of surveys currently undertaken in Scotland and administrative datasets as sampling frames; adding questions to current surveys in Scotland to gain more information on homelessness trajectories; using administrative data to study homelessness; and obtaining samples of people at risk of homelessness. General issues related to the design of surveys are not covered in this report - but, where appropriate, recommendations are made about specific research designs as Annexes.

SURVEYS IN SCOTLAND

4.4 Since 2000, the Scottish Household Survey (SHS) has asked respondents a number of questions about prior experience of homelessness and these identify a fair-sized sample of ex-homeless people each year. This therefore makes the SHS an ideal sampling frame for any study of ex-homeless people.

4.5 In its current design, the British Household Panel Survey (BHPS) would not be useful for any study of homelessness. However, there is the possibility of adding questions about homelessness to it (see paragraph 4.18).

4.6 Other surveys in Scotland (including the Scottish House Condition Survey, the Scottish Health Survey, the Scottish Social Attitudes survey and the Labour Force Survey) were examined, but were found not to be of use for a study of homelessness.

The Scottish Household Survey

Background

4.7 The Scottish Household Survey (SHS) is commissioned by the Scottish Executive and is one of the largest surveys undertaken in Scotland. There are two parts to the SHS - the household interview is completed by the main income earner or his/her partner, and the individual interview is completed by a randomly selected member of the household aged 16 years or over. The individual interview contains questions on attitudes and experiences, and contains several questions relating to homelessness:

(1) Have you ever been homeless? That is, lost your home with no alternative accommodation to go to?
(2) How many times has this happened to you in the last five years?
(3) Have you ever applied to the local council because you were homeless?
(4) Have you ever had to sleep rough because you were homeless?
(5) Have you got your name on either a council or housing association waiting list?

4.8 Because of its relatively large sample size (approximately 15,000 interviews achieved each year) and the inclusion of the above questions related to homelessness 1, the SHS offers the potential to obtain a relatively large sample of people who have experience of, but have subsequently left, homelessness. In the 2001 SHS, 568 (4%) respondents reported that they had some experience of homelessness and 129 (1%) reported that they had been forced to sleep rough at some time because they were homeless.

4.9 In addition to identifying a group of ex-homeless people, the SHS could also be used to identify people living in insecure accommodation and/or hidden homeless people by including extra questions to the SHS. This is discussed in more detail below.

Following up the SHS

4.10 The information collected about experiences of homelessness in the SHS is obviously useful in itself for investigating characteristics of people with experience of homelessness. The information would also be useful in generating a sample of people with experience of homelessness which could then be studied further using either a one-off retrospective quantitative or qualitative survey. This is an ideal opportunity to study a group of particular research interest - people that have broken the cycle of homelessness, and would therefore be able to provide crucial information on 'routes out'. (Because these people have been included in the SHS, it is unlikely that they were homeless at the time of the survey.) A follow-up full-retrospective study would allow more detailed information about their experiences of homelessness and their routes in and out of homelessness to be collected, either through a quantitative survey or qualitative work.

4.11 Qualitative work would be more suitable than a quantitative study to explore the process of moving out of homeless. This is because of the more detailed information that can be collected in a qualitative study. Details of a possible qualitative research design are included in Annex 3.

4.12 One concern about undertaking a quantitative survey is the relatively low rate of respondents that are willing to be re-contacted. In 2001, the questions on homelessness in the SHS identified 568 respondents that reported some experience of homelessness. Of those 568, however, only 72% (about 400) gave permission to be re-contacted 2. If a follow-up study of a sample from the SHS were to be considered then efforts should be made to improve this re-contact rate 3.

4.13 A second concern is that the sample of ex-homeless people obtained is likely to under-represent the long-term homeless (Rossi, 1990). Therefore quantitative estimates based on this sample are likely to over-represent the experiences of the short-term homeless. This bias should, at the very least, be acknowledged when results are published.

Adding further questions about homelessness to the SHS

4.14 One approach to circumvent the attrition from any quantitative study using the SHS and obtain some (albeit less detailed) information from SHS respondents would be to add additional questions (or a module of questions) about homelessness to the main survey. These additional questions would be answered only by those that report experience of homelessness (i.e. about 500 each year) and so would not impact on the length of the survey for the majority of respondents.

4.15 At the time of this research project a review of the content of the SHS was underway and thus consideration of the potential for new questions on homelessness to enhance understanding of this area was seen as an important element of the study. There is obviously a limit to the number of questions that can be added and hence the depth of information that could be collected. However, adding questions would be useful either if it were not planned to undertake a follow-up study of SHS respondents identified as having past experience of homelessness or to supplement any further study by ensuring a minimum level of information from each eligible respondent. This would generate information on a sample of ex-homeless people for minimal costs and effort. (Some questions that could be included in the SHS are listed in Annex 4.)

Obtain samples of other types of homeless people from the SHS

4.16 In addition to obtaining samples of those with previous experience of homelessness, it would also be possible to identify households in the following groups of homeless people from the SHS (definitions taken from the HTF first report, Scottish Executive, 2000):

• those whose accommodation is 'unreasonable' or is overcrowded and a danger to health;
• persons with only a short-term permission to stay, such as those moving around friends' and relatives' houses with no stable base;
• persons with no legal rights or permission to remain in accommodation, such as squatters or young people asked to leave the family home;
• tenants or owner-occupiers likely to be evicted (whether lawfully or unlawfully).

4.17 Some of these groups could be identified using questions already included in the SHS, others would require additional questions to be added. For example, because only permanent members of the household should be interviewed for the SHS, it is unlikely that an interview would be undertaken with anyone with short-term permission to stay and hence such people would not at present be identified. However, one could include a question in the SHS to check whether there is such a person in the household. (Questions that could be added to the SHS to identify these additional groups of homeless people are included in Annex 4.) Identifying these groups of homeless people through the SHS would be of particular policy interest because it would enable information to be gained on 'hidden' homelessness.

The British Household Panel Survey

4.18 The British Household Panel Survey (BHPS) is run by the Institute for Social and Economic Research at the University of Essex. The BHPS is comprised of a household interview and an individual interview with everybody aged sixteen or older in the household. Since 1999, the sample size in Scotland has been boosted to about 2,000 households. There are some core topics that are included each year - other topics are rotated or are included as one-off modules.

4.19 Each year some space in the BHPS is allocated to academics - the BHPS team decides which of the subject areas/questions to include from the applications made. Therefore, there is scope for including questions about homelessness in the BHPS. This could range from a few questions to a whole module (about twelve questions) at the discretion of the BHPS team. Any such application would need to utilise the longitudinal element of the BHPS as opposed to seeking just cross-sectional information. Thus a research question would need to draw on information already collected in the past and link this to the new homelessness questions.

4.20 The number of people in the BHPS that would report having experience of homelessness in Scotland is relatively small. Of the (approximately) 4,000 people interviewed for the individual survey (in which the questions about homelessness would be added), about 160 4 would report experiences of homelessness. Therefore, it would probably only be worth adding questions to the BHPS for a study of the whole of Great Britain.

ADMINISTRATIVE DATA

4.21 Administrative data are collected on people in a range of homeless situations and, in theory, should offer opportunities to study homelessness. This section assesses the feasibility of studying a range of types of homelessness either using administrative data directly or by obtaining samples for subsequent follow-up studies.

4.22 The main sources of data on homelessness identified during the course of our research are from local authority HL1 Returns, the Rough Sleepers Initiative (RSI) monitoring database run by the Glasgow Homelessness Network (GHN) and the Shelter database. The discussion in this section therefore concentrates on these three sources. It is possible that information from smaller organisations might also be useful, for example if a study was concentrated in a particular area.

Administrative data related to homelessness

The HL1 Returns

4.23 Local authorities collect information from every household applying for housing as homeless and record this into their own databases. Every quarter a core of information for each applicant is forwarded to the Scottish Executive, regardless of the status of the application. Although each local authority records the name of the applicants in their own database, this information is not passed onto the Scottish Executive - the record is identified by a unique application reference from which it is not possible to identify the individual/household.

• 4.24 The HL1 returns collect a wealth of information about each applicant. This includes:
• household composition;
• whether slept rough in last 3 months (this is collected for "a member of the household" rather than for applicant);
• postcode of last settled address;
• reasons for becoming homeless;
• statutory assessment;
• housing outcome;
• whether the household has applied previously to the same local authority, and if so the reference number of the most recent previous application.

4.25 At present, the applicants are only asked to give consent for their details to be sent to external organisations in order to check the validity of their applications. This consent would not cover forwarding any information that could be used to identify the applicants to the Scottish Executive (or an organisation working for the Scottish Executive) for research purposes.

Shelter Database

4.26 Shelter Scotland is the national campaigning charity for homeless and badly housed people, providing housing advice through its four Housing Aid Centres (there are also two affiliated advice centres) and Shelterline, a national 24 hour free phone help line covering the whole of Scotland. All contacts between its staff and clients through these Housing Aid Centres and Shelterline are recorded on a database.

4.27 This database includes:

• the full name of the client;
• date of birth, gender and ethnic origin;
• household size and type;
• homelessness status - including whether classed as priority and local connection;
• employment;
• whether roofless;
• referring agency - which indicates whether client is in contact with another agency;
• type of problem - includes homeless, in rent or mortgage arrears relationship breakdown, victim of domestic violence;
• any actions taken/outcomes.

RSI monitoring

4.28 In order to monitor the second round of RSI projects, a database system was set-up by the Glasgow Homelessness Network (GHN), which is used by 68 of the 77 RSI projects. Information is collected by the projects about every individual that makes contact. This information is sent to the GHN and contained in a central database. Using a 'unique' ID code consisting of a client's initials, date of birth and gender, any applications to different projects funded by the RSI can be identified.

4.29 At the point of first contact, a wide range of information is collected including reasons for and history of rough sleeping. However, the only pieces of information required to create a new record are: name, age or date of birth, gender and whether roofless at time of referral. Identifying information such as National Insurance numbers and contact addresses, including forwarding addresses for re-housed clients are not collected.

4.30 Other information that is entered into the database (if collected) includes:

• reasons for sleeping rough;
• last accommodation before sleeping rough;
• history of sleeping rough;
• age first slept rough;
• whether barred and reason for being barred;
• if moved from other local authority;
• where referred and outcomes.

4.31 The projects are asked to enter as much information as possible, however often this is not possible - for example, if the project has contact with a client for a very short time. As an example, the recent figures for all referrals for RSI projects in Glasgow show about 40% item non-response for key information such as alcohol and drug related problems.

Obtaining samples of homeless people from administrative data

4.32 From the administrative datasets described above it would (in theory) be possible to obtain samples of homeless people to include in a semi-retrospective study. However, this does depend on the co-operation of the data holders and appropriate consent being obtained (see Chapter 5).

4.33 The information collected for the HL1 returns would allow samples of all types of homeless people to be obtained. However it should be noted that the people that apply to their local authorities may not be fully representative of the homeless population. It has been argued, for example, that a lower proportion of single homeless people than other groups of homeless people apply to local authorities (Fitzpatrick et al., 2000). However, it is possible that this will change in September 2002 when local authorities are obliged to offer temporary accommodation to all homeless households (including single homeless people) under the Housing (Scotland) Act 2001. In any case, even with its limitations, HL1 returns constitute by far the most comprehensive single dataset on the homeless population in Scotland.

4.34 In order to adequately cover groups such as single homeless people and rough sleepers who may be under-represented in the HL1 returns, it would be necessary to sample the clients of a range of voluntary agencies. The Shelter and/or RSI databases would be useful in this regard. (In Glasgow, for example, it is estimated that RSI funded projects account for about a third of all homeless service provision.) If a study was targeted in a particular geographical area, then it would be feasible to improve coverage by also sampling from non-RSI projects. However, this would not be possible for studies of larger geographical areas.

4.35 To follow up a sample of homeless applicants for interview, contact information would be required. For the HL1 returns, contact information could only be obtained by contacting the relevant local authorities directly 5 (this information is not passed on to the Scottish Executive). This would require co-operation from every local authority from which a sample member is likely to be recruited. If contact with people sampled from the local authority lists was attempted as close to the date of application as possible, then it is likely that it would be possible to make contact with the majority of sample members.

4.36 Obtaining contact information from other administrative datasets would be more difficult. The Shelter database does include a contact address, although for some groups of homeless people (e.g. rough sleepers) this might not be sufficient to make contact directly. The RSI dataset does not include contact information, although it would be possible to obtain a last point of contact or to contact the person via the RSI project directly. Making initial contact with homeless people selected from any administrative dataset could involve considerable effort. Some of the issues involved are described in Chapter 5.

Monitoring homeless people's pathways using administrative data

4.37 One approach to undertake a semi- or full-prospective study of homelessness would be to use only administrative data (i.e. to have no direct contact with the sample at all). The samples of interest could be selected from any source - including from administrative data itself - and relevant information extracted from administrative datasets at various points in time.

4.38 The amount of information available from administrative sources is substantial. However, as Chapter 3 highlighted, the key weakness of tracking people through administrative databases is that no information would be available about those that make no contact to any of the relevant organisations during the study period. This therefore implies that nearly all the interesting measures that could be obtained from administrative data would be (potentially) heavily biased. Therefore, there is a limit to the amount of useful quantitative information that could be inferred from using solely administrative data, and any figures would need to be treated with extreme caution.

4.39 As an example, a potential research question might be to estimate what proportion of a sample of people sleeping rough at a particular time point are still sleeping rough one year later. If this sample was tracked using administrative data over a course of the year then, for each individual, there will be one of three outcomes - they are identified as still sleeping rough, they are identified as not sleeping rough or there is no further information about them. For people in the third category one would have no idea if they were sleeping rough or not. Therefore, as a large number of people are likely to fall into this category, it would not be possible to obtain the desired estimate.

4.40 That is not to say that analyses of administrative databases are not worthwhile. Administrative databases are ideal for obtaining cross-sectional estimates and also some retrospective information (for example, the HL1 includes questions on experiences of rough sleeping). In addition, they are useful for studying some specific longitudinal research questions. For example, because the HL1 return now identifies repeat applications, it is now possible to track households' applications to their local authority. However, it seems unlikely that undertaking a prospective study that used just administrative data would adequately cover the key policy issues identified in this study.

FULL PROSPECTIVE STUDIES FOCUSSING ON AT RISK SUB-GROUPS

4.41 As Chapter 3 outlined, the most challenging approach to longitudinal work with homeless people would be to follow a group of people from the point before they experience homelessness, through their homeless careers and onwards after they stop being homeless (a full prospective study). Such a study may be particularly interesting from a policy perspective because it could aid the prevention of homelessness.

4.42 Since the proportion of people in the general population who become homeless is relatively small, following a representative sample of the population to see who became homeless would require a very large sample and would be very expensive for the return it might give. However, since quite a lot is known about the characteristics of people in the homeless population, it might be possible to sample groups of people that might be considered at risk of homelessness and include them in a longitudinal research study.

4.43 Groups that might be included in such a study since there is evidence that they are at higher risk of homelessness include:

• young people leaving care;
• people leaving drug rehabilitation units;
• people leaving prison;
• people leaving mental health facilities;
• people discharged from the army;
• people facing eviction.

4.44 Undertaking a study purely to identify whether people from these groups become homeless is likely to be impractical, and may also be considered dubious from an ethical point of view. Thus any longitudinal work with these groups should probably be driven by policy-makers with direct responsibility for their overall well-being and should cover a wide range of topics, enabling a 'holistic' approach to be taken.

4.45 From a homelessness policy perspective, quantitative longitudinal work with people in these groups might be valuable in identifying:

• the proportions of people in high risk groups who actually become homeless;
• any statistical variations in the characteristics and circumstances of those who do and do not become homeless; and
• the interventions and projects that appear most effective in providing routes out of homelessness for people from high risk groups.

4.46 Qualitative work among high risk groups might also be useful in gaining more detailed information about the reasons some do and some do not become homeless, and why certain types of projects seem to be more or less successful in resolving homelessness for people in these groups.

4.47 In the following discussion of tracking people at risk of homelessness we focus on one of the groups identified above, young people leaving care, because the existing evidence suggests that they experience the very highest risk of homelessness. While the sampling strategies may vary a little for other groups, many other issues would be common across the groups and relatively similar research designs could probably be employed. Some information is also given on sampling the other groups.

Obtaining a sample undertaking follow up research with young people leaving care

4.48 There has been some previous work following up young people leaving care in Scotland. However this has tended to be small scale in nature and to follow people up only once, shortly after they leave care. For example, in a recent study in Scotland, a sample of 100 people across a few local authorities were interviewed just before they left care (or retrospectively for a small sample) and then again six months later having left care (Dixon and Stein, 2002). This study had an initial structured interview and more qualitative six-month follow up, and considered outcomes related to accommodation as well as employment or training, social networks, health (including drug use) and offending. This study identified that around 40% of the young people had some experience of homelessness within the six-month period. This is based on young people's own views of considering themselves to have been homeless, including sleeping on friends' floors.

4.49 Around 1000 young people leave care in Scotland each year. Sampling of these people would be possible via local authority social work departments.

4.50 From the previous studies undertaken it does seem possible, and potentially of interest to other policy makers, to think about a general purpose longitudinal study of care leavers. With a large enough sample this might provide valuable information on the numbers of this group who become homeless, and over time, the factors that affect successful housing outcomes. A wealth of other data might also be generated and the lead for such a study should perhaps come from those directly involved in assessing the success of care services. A proposed research method is described in Annex 5.

Generating samples of other at risk groups

4.51 Similar studies could be undertaken for the other at risk groups. We have undertaken some preliminary work to investigate whether similar research has been undertaken and what sampling frames are available. This is summarised in this section.

People facing eviction

4.52 Research tracking people facing eviction has been done by following up people identified as being at risk of eviction from seven years of the BHPS (Boheim and Taylor, 2000).

People leaving prison

4.53 Information about all prisoners in Scotland is contained in a database called the Scottish Prisoner Information Network (SPIN) which is held by the Scottish Prison Service. This would seem to be the most useful sampling frame for people leaving prison. However, although SPIN does include a contact address, it is the address which the offender gives to the court and in many cases this address is not correct.

People discharged from the army

4.54 Research has been undertaken on people discharged from the army. For example, a study of reproductive outcome and child health among 53,000 UK veterans is being undertaken by the Department of Epidemiology & Population Health at the London School of Hygiene and Tropical Medicine (see www.lshtm.ac.uk/eph/eu/gulf_war/ for further information). For this, the Ministry of Defence gave names and addresses of service personnel to the research team. For any study of homelessness, the co-operation of the Ministry of Defence would be required to generate the sampling frame.

People leaving mental health facilities

4.55 In undertaking a study of people leaving mental health facilities, the ideal approach would be to generate a sample of people in the relevant institutions and then trace them after their discharge. Some form of 'trigger' would be needed so that the institutions informed the researchers of the discharge as it happened. This would ensure that no-one was missed from the sample.

4.56 A database that might serve some purpose in sampling people currently or having been in mental health facilities would be the Scottish Morbidity Records (SMRs). These records include information on all inpatient and outpatient hospital activity and have been used as a sampling frame for other research purposes. It would be unlikely that there would be a problem with the Scottish Executive having access to the records. However they would not be useful in obtaining samples of people at the point of leaving mental health facilities - as they would only indicate whether individuals were still resident or had been discharged.

LONGITUDINAL EVALUATION OF PROGRAMMES/ PROJECTS

4.57 A further area of research for studying homelessness is the longitudinal evaluation of particular projects or programmes. At the moment some projects do attempt to follow people over time so that they might gain better information on how their work is affecting long term outcomes, however this work tends to be fairly small scale and involve only short term follow up.

4.58 In this section we make some suggestions on how longitudinal evaluations might be standardised to be able to deliver greater strategic benefit by allowing comparison between approaches. We then look at one programme which is about to take place, and which might offer great scope for some longitudinal evaluation - the Glasgow Hostel Closure Programme.

Longitudinal evaluation of projects

4.59 Many evaluation studies for homelessness projects are designed only to evaluate processes of the project and short-term outcomes. They are therefore able to address questions such as:

• How did the project operate?
• Which were the things (e.g. processes) that worked well and which worked less well?
• Did the project have short term benefits for the clients and what were they?
• Did the clients feel they benefited from the project?
• How satisfied were the staff working on the project?

4.60 This is all valuable information, in particular when used by the projects themselves to improve the delivery of services. However, often what is of particular interest to policy makers is the long-term benefit for clients and the success or failure of different interventions.

4.61 It would therefore be helpful for projects to move away from evaluations which focus primarily on processes of administering the projects and to include assessments of the long-term benefits of the clients. Ideally all longitudinal evaluations of projects should begin with an initial assessment of the clients and should monitor outcomes ('soft' as well as 'hard') over a period of time. By introducing an element of standardisation into this process, it would be possible to compare outcomes for different projects. This would then be of strategic benefit to policy-makers in seeking to address questions such as:

• Which types of projects tend to work well/ less well?
• What are the types of people for whom different types of projects are most/least successful?

4.62 The Homelessness Monitoring Group could become involved in setting up a standardised approach to evaluation. Any attempts to standardise evaluations should recognise that evaluations for many projects will be largely or entirely qualitative in nature due to the numbers of clients using the project and the types of skills and resources available for the evaluation work.

4.63 It is hoped that development of a standardised approach to evaluation will be implemented for the longitudinal evaluation of the Glasgow Hostel Closure programme (this programme is discussed below). It could also be that some of the instruments being developed under the Glasgow Hostel reprovisioning process might be useful in assessing the long-term outcomes of a wide range of projects.

Glasgow Hostel Closure programme

Background

4.64 Glasgow currently has 28 hostels providing around 2,000 places. Most of these provide accommodation for single homeless people. Within this hostel population men represent 75% of the clients and a third are under 25 years of age. A relatively high proportion of Glasgow's hostel population are long-stay clients with some people having spent 20 years in the same place. Despite this, four in ten clients have been in their present accommodation for less than three months.

4.65 It is now planned that the large traditional hostels will all be closed over the next five years - although in reality it may take longer. All existing hostel clients will be resettled, and the procedures will also change for those who approach as 'new' clients to the system. Prior to resettlement all clients will receive an assessment. In addition, assessments are being introduced for new clients approaching the homelessness services.

Evaluation of the Glasgow Hostel Closure programme

4.66 The Glasgow Hostel Closure programme will be subject to some form of evaluation. However the planning of this does not appear to have been extensively developed at this stage, therefore some suggestions could be fed into the process. To evaluate the Glasgow Hostel Closure programme, a sample of homeless people should be selected from those that undertake the assessment interview and followed-up over time to assess the outcomes of the different resettlement options. Given the size of the population, quantitative work would be feasible, which could allow the following questions to be addressed:

• What is the relationship between clients' assessments for support on leaving the hostel settings, and the resettlement programmes on which they embark?
• What kinds of resettlement programme work for those with different support needs?
• What outcomes are achieved in terms of key indicators? (This might include accommodation provision, holding down a tenancy, receiving addiction services, gaining employment, skills training, improved social networks etc.)

4.67 Qualitative work with some members of this group would also provide additional information on topics such as:

• Why some people with apparently similar assessments and who embark on similar resettlement programmes are successfully re-housed and others are not?
• What makes some projects or combinations of project types particularly successful or unsuccessful?

4.68 Recruiting a sample for such a study should be possible, although it would not be entirely straightforward. A suggested design for the evaluation of the Glasgow Hostel Closure programme is included in Annex 6.

CONCLUSIONS

4.69 The conclusions about the potential research designs are summarised in the following table. For each design, the table shows the types of homelessness that could be studied, the type of longitudinal research undertaken, whether it analyses positive or negative outcomes, the suggested methodology and any concerns with the design.

« Previous | Contents | Next »