Scottish Government

OPERATIONAL EFFECTIVENESS OF THE CHILD PROTECTION LINE

3.1 There are a number of operational effectiveness issues that are prevalent at both a national and local level. These are considered in this section of the report across the following main areas:

• management of the Child Protection Line;
• performance of the Child Protection Line;
• effectiveness of the marketing campaign.

Management of the Child Protection Line

3.2 The operation of the CPL nationally has been managed by the Essentiagroup, under the management of the Scottish Government working group since the CPL's launch in February 2007. The child protection policy team within the Scottish Government provide the overall strategic and operational management of the CPL. The Essentiagroup are a well-established consumer contact and communications company, managing many high-profile Government funded Scottish-based and UK based telephone lines including:

• Connexions Direct;
• Sexwise;
• NHS Smoking Helpline;
• FRANK.

3.3 The Scottish Government working group and the Essentiagroup have developed effective and efficient processes and protocols to successfully manage the operation of the Child Protection Line. Effective systems have been established to monitor callers to the CPL. The Essentiagroup, in partnership with the Scottish Government, has developed user-friendly, comprehensive and transparent monthly monitoring reports. These provide information on the level and type of calls received through the CPL.

3.4 The content of the monthly monitoring reports has evolved since the establishment of the CPL. These amendments reflect changes in the type, and amount of information collected by advisers. The working group has been proactive in ensuring the information collected by advisers in response to feedback to the Scottish Government from Child Protection Committees is fit for purpose and reflects the developmental nature of the CPL. The ability for such changes to be actioned efficiently by the Essentiagroup demonstrates their flexible and client-focused approach to fulfilling the remit of the CPL.

"The quality of service provided by the Essentiagroup was excellent. The regular reporting mechanisms were of a very high standard, and contained relevant information." (Child Protection Committee Lead Officer)

3.5 Robust processes were established for advisers to efficiently deal with calls and the transfer of calls to local areas. The remit of the CPL warranted clearly defined and transparent processes to ensure advisers were trained and knowledgeable and to ensure calls were handled and transferred to local areas using a standardised approach. The Essentiagroup in partnership with the Scottish Government developed effective processes to ensure this occurred and that advisers benefited from Scottish Government refresher training.

3.6 A call flow model defined the process for how call advisers should deal with calls. This model encompassed all stages of a live call and provided guidance to advisers across all possible call scenarios. For example:

• fully informing callers of the remit of the CPL and how the information they provide will be used;
• dealing with calls that relate to children at risk of immediate harm, mid-call transfers and general enquiries.

3.7 Dealing with calls that could not be transferred immediately to local areas was a key remit of the CPL. This became a higher priority when it was established through the monthly management information that over 40% of calls received about specific children or families were not able to be transferred immediately to local areas. At this point, the Scottish Government working group had discussion with all participating Child Protection Committees, reinforcing the importance of CPL referrals being able to be transferred to local areas with ease. The processes developed by the Scottish Government working group and the Essentiagroup for such calls ensured that:

• information from callers was passed to local areas as quickly as possible;
• service leaders oversaw the continued efforts to transfer calls;
• callers did not become lost in the system.

3.8 The skills and training of the call advisers to effectively deal with calls was a key feature of the operational model. The robust processes to allow call advisers to effectively operate the CPL were underpinned by:

• comprehensive staff training;
• a supportive staff environment.

3.9 Call advisers' training was specifically tailored to provide an understanding of the policy context in which the CPL was operating, and practical training in dealing with calls.

3.10 Providing comprehensive support to call advisers was an important feature of the Essentiagroup's approach. The remit of the CPL resulted in call advisers having to deal with difficult and sometimes traumatic calls from the general public. The call advisers spoke highly about the support they received from service leads within the organisation and the organisation more widely, both in relation to providing advice on how to deal with particularly difficult calls, and to talk through any difficult calls they may have taken.

3.11 The Scottish Government has effectively managed the relationship with the Essentiagroup. The organisation has been actively involved in the working group, providing regular feedback on the operation and performance of the CPL. Their presence has also ensured they are central to discussions around the feasibility of any required changes to monitoring processes and delivery models.

Delivery Model

3.12 The call-centre delivery approach adopted for the operation of the CPL was an issue for some local areas. There were specific concerns at a local level in relation to the ability of this delivery model to effectively meet the needs of callers to the CPL.

3.13 The perceived skills of the call advisers to effectively to deal with difficult callers was a common concern. Local areas valued the comprehensive training provided to advisers to ensure they were well-equipped and knowledgeable to deal with the type of calls they received through the CPL. However many felt that using non-professionals in the operation of a child protection line was potentially damaging.

"I was concerned about the advisers on the ( CPL) line, particularly as the majority did not have a prior background in child protection." (Child Protection Committee Lead Officer)

"Practitioners and Child Protection Committees weren't that happy with the set-up of it - they wanted professionals to man the phones." (Child Protection Committee Lead Officer)

"There is no support or response at the end of the CPL - it's just another layer or loop in the system… another person to tell your story to." (Child Protection Committee Lead Officer)

3.14 The strike rate (proportion of calls answered) achieved by the CPL was raised as an issue. An average strike rate of 86% between February 2007 and October 2008 was achieved. However, a number of local and national stakeholders had perceptions that a higher percentage of calls were not answered. The figures provided for the Domestic Abuse helpline suggest that for the period February 2009-April 2009 a strike rate of 82% was achieved. This suggests that comparatively the strike rate achieved for the Child Protection Line appears sufficient.

3.15 The delivery model of the CPL is one of the major factors influencing the strike rate. However, some level of unsuccessful calls is unavoidable in the operation of a telephone line, due to the random timing of in-bound calls. The delivery model for the CPL was variable and was selected dependent on expected call demand. The ongoing monitoring of call demand, allowed decisions to be made quickly on which delivery model would best meet the level of calls received.

3.16 The delivery model can impact on the strike rate in the following ways:

• if higher volumes of calls are received than have been predicted;
• from the pattern of calls i.e. if a dedicated model is employed and calls are received within a short time period when all available advisers are on calls this will impact on the strike rate;
• from the duration of calls.

3.17 The average strike rate achieved by the CPL does appear to be at a sufficient level, compared to the Domestic Abuse helpline. The Scottish Government has taken a flexible approach to the delivery model, reacting to expected call demand, timing of marketing, and cost considerations. Quality assurance and effective management of the CPL has encouraged decisions on the delivery model to be based on clear rationale and evidence.

3.18 Continual monitoring of the strike rate is required by the Scottish Government to ensure an optimal approach. For example, the strike rate achieved in January 2009 was 63%. This is due to a change to the bureau delivery model. This is an unacceptable strike rate if an appropriate level of accessibility to the Child Protection Line is to be maintained.

Transfer of Calls to Local Areas

3.19 The transfer of referrals from the CPL to local areas has been effectively handled by the Essentiagroup. Issues with the transfer of referrals, where evident, relate to the operational model and overall remit of the CPL, as opposed to the management of the CPL by the Essentiagroup. There were pertinent issues in relation to the ability of some Social Work departments to accept live calls, as a high proportion of these were not able to be transferred immediately. The call advisers followed standardised procedures to ensure that the transfer of referrals, if not able to be done immediately were transferred as soon as possible.

3.20 The level of information provided by callers to advisers was often disparate and limited. Many callers were reluctant to disclose comprehensive personal information about either themselves or the child or family for which there was a concern. This caused difficulties in the identification of these children or families at a local level, often causing frustrations at a practitioner level.

3.21 Improving the quality of information provided by callers would be difficult to resolve easily. Caller apprehension in providing information, due to possible repercussions, may be one possible reason for this poor quality. Conversely, providing identifiable information on children and families will be more difficult for callers who are not relatives. There is no evidence that this lack of information is due to the skills of the call advisers. As highlighted in paragraph 3.5 there was clear evidence to suggest that call advisers were dealing with call referrals using a standardised and robust approach.

3.22 Out-of-hours transfers to local areas varied in their success. The Essentiagroup experienced considerable variation in their ability to successfully transfer calls to out-of-hours teams. Call advisers reported that there were some areas for which it was very easy to transfer referrals out-of-hours. In other areas there was more reluctance by the out-of-hours teams to accept the referrals, instead specifying that the advisers should attempt the referral again during normal working hours.

3.23 There were no pertinent issues raised at a local level in relation to out-of-hours transfers. This is perhaps surprising considering the high level of calls that were unable to be transferred immediately to local areas by the Essentiagroup and the reluctance of some areas to accept out of hour's calls. The Social Work practitioners that were involved in this evaluation and the Child Protection Committee lead officers however did not raise any concerns about their capacity to deal with calls. This does not necessarily mean that this is not an issue at a local level, but instead this was not an issue for the sample of Social Work practitioners (within one Child Protection Committee area) involved in the evaluation. More in-depth and wider consultation with Social Work practitioners may uncover greater prevalence of such issues.

Monitoring & Tracking of Referrals at a Local Level

3.24 There was generally ineffective monitoring and tracking of referrals at a local level. Although the Scottish Government has issued guidance to local areas on monitoring and tracking referrals, seven Child Protection Committees did not have effective systems in place to monitor referrals. For Child Protection Committees with established monitoring systems, there was often found to be significant disparity between the number of referrals recorded by the Essentiagroup, and the number of referrals recorded by local areas. This again highlights difficulties with the tracking processes at a local level.

3.25 Insufficient co-ordination and quality assurance at a local level was the reason for poor referral monitoring in some areas. Local areas without tracking systems often reported having had initial plans for tracking to be undertaken, only finding later that this had not been actioned. The requirement for effective monitoring and tracking was perceived to be of low priority by local areas or there was a general lack of awareness of this requirement by front line staff.

3.26 Areas with effective tracking systems had put in place specific systems and protocols to ensure that staff were fully aware of the need to identify referrals from the Child Protection Line. For example, one rural area reported developing a memo system, ensuring that all practitioners in the local area who were dealing with referrals were producing regular feedback on progress and outcomes to the Child Protection Committee lead officer, although this would be less effective when scaled up to cope with larger volumes.

3.27 Inconsistent tracking has resulted in difficulties in measuring the impact of the CPL. Understanding how calls have been dealt with once they have been referred to a local area, and therefore understanding the 'added value' of the CPL at an individual case level is severely hindered by inconsistent tracking.

3.28 Guidance on the process and need for referral monitoring, although provided by the Scottish Government, was not enforced. Local areas were generally autonomous in their approach to monitoring, adopting their own approaches to complement existing local referral processes. It is this lack of enforcement that has in the long term affected our ability to fully understand the value of the CPL at an individual case level.

3.29 Areas with established monitoring and tracking systems also faced difficulties in identifying how referrals at a local level were made (Table 3.1).

Table 3.1: Tracking Issues

3.30 Tracking at a local level should be improved significantly if the impact of the CPL at a child and family level is to be fully explored. The inability for local areas to accurately identify referrals from the CPL, may suggest a wider issue, which although external to the remit of this evaluation, suggests that local areas need to tighten their approaches to monitoring and tracking of referrals and cases at a local level. In the case of the Child Protection Line, strengthening quality assurance procedures at a national and local level to allow the identification of referrals is required in order for the impact at a case level to be fully understood.

Effectiveness of the Marketing Campaign

3.31 The powerful child protection message portrayed by the marketing was the most positive aspect of the Child Protection Line. The majority of Child Protection Committees, including those who had not been actively involved in the distribution of marketing materials, positively rated the message portrayed.

3.32 There were elements of the marketing campaign that were felt to be more powerful than others. The telephone box campaign was singled out as being particularly successful.

"The publicity was far more than we could generate on a local level e.g. the telephone box posters were very good at raising awareness."

"The Child Protection Committees involved directly in the marketing pilot have reported a greater response attributable to successful media campaigns. For those not directly involved in the pilot the success is difficult to measure. The greatest impact had been the bus shelter poster as they had a powerful visual image which often triggered a response to local services." (Child Protection Committee Lead Officer)

"It was a well thought out strong message." (Child Protection Committee Lead Officer)

3.33 Greater resource allocation to allow more widespread and sustained marketing was a key requirement of local areas. The shortness of the marketing campaign ³ was perceived as being detrimental to the ability of the campaign to maximise its impact on public awareness. This limited the power of the awareness raising message at both a national and local level.

3.34 Local areas would have valued a much higher profile marketing campaign, sustained over a longer period of time. The use of more widespread and powerful mediums (e.g. TV & radio) was also a common suggestion from local areas.

3.35 The review of other telephone lines and campaigns provides useful learning about key success factors in marketing campaigns. For example:

• public awareness of campaigns was greater when marketing was sustained and intensive over a substantial period of time;
• utilising a range of marketing strategies was beneficial in embedding campaign messages.

3.36 Clarity of the marketing message and the knowledge of CPL operators were also viewed as being key to operating an effective CPL. Learning from other telephone lines and campaigns would be beneficial in further developing and maximising the impact of the CPL.

"The campaign was not high profile compared to other national campaigns - for example the Domestic Abuse campaign, for which there is a TV advert on every night."

"It needed a large-scale campaign with much more resource including TV and radio adverts. This campaign fell somewhere in-between."

"It would have been better to do some TV and radio adverts to really reach everybody."

"Although the intention was maximum coverage, even I wouldn't be able to quote the telephone number. It isn't very catchy."

3.37 It should be noted that the CPL marketing campaign was not intended to be significantly high-profile. The intensity of the marketing campaign was based on resources and capacity constraints. For example, call demand generated by the marketing campaign was specifically intended to be at a level which the Essentiagroup delivery model had the capacity to handle effectively. The Scottish Government had intended for the CPL to generate 300 calls per month with the marketing campaign. The call levels achieved after the launch of the marketing (Table 3.2) indicates that the CPL was relatively successfully in achieving this aim.

3.38 Variation in engagement with the marketing campaign across local areas was detrimental to embedding a national child protection awareness raising message. The opportunity for local areas to opt fully, selectively or not at all in the marketing was not helpful in creating a powerful and consistent message nationally. Child Protection Committees to some degree had demanded this flexibility from the Scottish Government. The willingness of some local areas to only engage in the marketing campaign selectively, or not at all had been accommodated by the Scottish Government who had attempted to meet the needs of local areas, through reacting to actual demand for the marketing materials. However, with the aim of the CPL to create a single national gateway it could be perceived that local areas were provided with too much flexibility in their approach to marketing, leading to disparate and inconsistent campaigns across local areas.

3.39 It was perceived by some local areas that greater mandatory direction at a national level would have supported the establishment of a coherent national campaign. A number of local areas, particularly those fully engaged, commented that there was more emphasis placed on accommodating local areas' approaches to the national campaign, rather than creating a powerful and consistent campaign nationally.

3.40 There were a minimal number of concerns in relation to the portrayal of the marketing message. These concerns related to the imagery and one local area did not approve of the imagery on the posters:

"The pictures of the children were the stereotypical white; poor dishevelled looking children which people always think of."

"I felt that the message portrayed contradicted other existing children and young people focused campaigns which aimed to empower young people to speak up if they had a concern."

Line Performance

3.41 The monthly management information collated by the Essentiagroup enables a more detailed understanding of the performance of the Child Protection Line. This information provides the context for the Child Protection Line across the following key areas:

• call demand;
• call outcomes - children and families;
• call outcomes at a local level.

3.42 The management information has been analysed for the period February 2007 to October 2008. Comparative analysis for the period February 2007 - January 2008 and February 2008 - October 2008 also provides a valuable insight into the influence of the marketing campaign on the performance of the CPL. A detailed analysis of the management information can be found in Annex B

Call Demand

3.43 Call demand to the CPL generally is relatively low. On average there were 120 calls to the CPL per month between February 2007 - October 2008. This increased considerably after the launch of the marketing campaign, to 221 calls per month. As already highlighted (paragraph 3.38), the Scottish Government intended for the marketing campaign to generate 300 calls per month.

3.44 There was a significant increase in call demand after the launch of the marketing campaign. Call volume increased by 270% from February 2008 (see Table 3.2 below). Although it is not possible to attribute this increase directly to the launch of the marketing campaign, there is good evidence (as shown in paragraph 3.44) to suggest the marketing campaign contributed to this increase in call volume. Other extraneous factors that may influence an increase in call demand include wider media attention on child protection issues (high-profile child protection cases), and resulting heightened public awareness.

3.45 The strike rate of the CPL has remained relatively consistent since its establishment. The strike rate refers to the proportion of calls that were answered by the call advisers. The average strike rate since the CPL launch was 86%. The strike rate before the launch of the marketing campaign was 84%, with there being a slight improvement in the strike rate after the launch of the marketing campaign (87%). As outlined in paragraph 3.16, this increase may directly relate to the change in delivery model after the launch of the marketing campaign. That is, the move from a bureau model, to a dedicated model of advisers operating the CPL.

Table 3.2: Call Demand

3.46 Call demand for the CPL is low compared to the perceived level of calls received through local access points. A number of Child Protection Committees provided anecdotal information on the level of calls they received through local access points. These were reported to significantly surpass the level of referrals they received through the Child Protection Line. This has a view that was consistent both across Child Protection Committees that had engaged fully or selectively in the national marketing campaign, and of course those who had not engaged.

3.47 Information calls account for a high proportion of total calls received to the CPL since its launch. Over 900 calls to the CPL were information calls:

• 61% (n=627) of these were in relation to general information;
• 28% (n=284) of calls referred to a national agency (e.g. Parentline, NSPCC etc);
• 5% of calls were requests for campaign specific information.

3.48 The Child Protection Line has played a distinct role in dealing with information calls which may have previously been dealt with directly through services in local areas. This suggests a positive role of the CPL in:

• reducing demands placed on local areas to dealing with such information calls;
• potentially raising awareness of child protection issues at a more preventative level i.e. encouraging the public to better inform themselves if they needed to report a child protection concern in the future.

3.49 Comparisons between the level of call demand with other Scottish and UK based telephone lines are difficult to accurately assess. Evaluation and research findings of similar signposting telephone lines are not available in the public domain. Evaluation findings for advice/guidance focused telephone lines are available; however, variation in their remit makes comparisons in call demand and outcomes difficult. However, for illustrative purposes, comparing the Child Protection Line with other advice/guidance focused telephone lines is useful in placing the call volumes received in context (See Table 3.3).

Table 3.3: Average number of calls per month

3.50 Call volumes to the Child Protection Line are significantly lower (per month) compared to other existing telephone lines. There are a number of potential reasons for other telephone lines yielding much greater call volumes than the CPL. This may include:

• more intensive and sustained marketing;
• a wider focus on providing advice/guidance based support;
• a larger target audience;
• a greater need for demand for such telephone lines;
• a lack of local 'substitute' services.

3.51 It is not possible to fully understand within the remit of this evaluation why there is such disparity in call volume.

Call Outcomes - Children/Families

3.52 Calls to the CPL about specific children and families increased after the launch of the marketing campaign. There was over a five-fold increase in the number of calls received through the CPL about specific children and families. Between the period February 2008 - October 2008 there were 474 calls about specific children/families, compared to 88 calls between February 2007 - January 2008.

3.53 Calls were less likely to be successfully transferred immediately to child protection teams after the launch of the marketing campaign. 51% of calls were not able to be successfully transferred immediately after the launch of the marketing campaign, compared to 23% before its launch. This change may be due to the following:

• Social Work teams not having the capacity to answer calls from the CPL advisers;
• an increase in out-of-hours calls, and therefore increased difficulties in successful mid-call transfer during these times;
• increased call volume may impact on the ability of calls to be transferred immediately to local areas.

Table 3.4: Call Outcomes

3.54 Increased call volume about specific children and families indicates the following positive outcomes:

• an increased awareness of access points to services that the public may have previously been unaware of;
• a greater willingness to report child protection concerns as a result of the marketing.

3.55 The CPL is successfully attracting callers with valid child protection concerns. Concerns about neglect, and drug and alcohol abuse were most prevalent across all calls. These trends were seen both prior to and after the launch of the marketing campaign. For example:

• 17% of calls were about unattended or unsupervised children;
• 10% were about children appearing hungry, tired or untidy;
• 10% were about the misuse of drugs and alcohol.

Table 3.5: Area of Concern

3.56 There were changes made to the monitoring of children/family concerns in August 2008, which provided more detailed information on specific concerns of callers. This was a definite improvement but was introduced too late to inform this evaluation.

Call Outcomes at a Local Level

3.57 There was considerable variation in referral levels across local areas. This may reflect variation in marketing campaign engagement across local areas and indicates a possible relationship between the extent of marketing campaign engagement and referral levels. For example the highest level of referrals (between February 2007 - October 2008) for areas fully engaged in the marketing campaign were in:

• North East Scotland - incorporating Aberdeen, Aberdeenshire and Moray (79);
• Glasgow (65);
• Edinburgh (64);
• Fife (46).

3.58 An increase in referrals to individual Child Protection Committees coincided with the launch of the marketing campaign in February 2008. As highlighted previously this suggests public awareness about the CPL and its remit increased through the marketing campaign.

3.59 Areas that were fully engaged in the marketing campaign, generally received a much higher number of referrals. Conversely, there were some areas who although fully engaged in the national marketing campaign, received a relatively low level of referrals, for example:

• Scottish Borders (6);
• Argyll & Bute (9);
• West Dunbartonshire (11);
• Falkirk (12).

3.60 Referral levels to areas who had engaged in selective elements of the marketing campaign were less consistent. For example, areas such as North Lanarkshire (28), West Lothian (22), and South Lanarkshire (21) received a greater level of referrals. Other areas received considerably less referrals (Inverclyde, Stirling and Western Isles).

3.61 The four areas that 'opted out' of any form of marketing as would be expected received a very low level of referrals. These included:

• Clackmannanshire (3);
• Orkney Islands (0);
• Shetland (1).

3.62 East Lothian did not engage in the marketing campaign; however it received 21 referrals through the CPL. The prevalence of the marketing campaign in the neighbouring area of Edinburgh may explain this high level of referrals.

3.63 Marketing engagement is an important factor influencing referral levels in local areas. However, disparity across local areas that have employed the same marketing approach suggests the presence of other contributory factors, including:

• demographics of local areas i.e. rural vs. urban, population density;
• a lack of willingness within some local areas to report child protection concerns;
• an existing good awareness of local access points to services;
• level of need within local areas.

3.64 Variation in referral levels across areas that were fully engaged in the marketing campaign are particularly interesting to consider. Densely populated urban areas such as Glasgow and Edinburgh, with well recognised and prevalent child protection issues, will always yield a greater level of referrals than less populated rural areas.

3.65 Figure 3.1 helps to demonstrate the possible relationship between referral levels and the level of engagement in the marketing campaign. 'Calls per 10,000 residents' shows the expectation that more densely populated areas are more likely to generate CPL calls. There is evidence of a relationship, suggesting that increased marketing engagement generated more referrals.

Figure 3.1: Calls per 10,000 residents by local authority and marketing engagement

Source: YCL data, 2009 and National Statistics, Population Mid-Year estimates, 2007

Website Performance

3.66 The Scottish Government launched the Child Protection Line website in February 2008. A more detailed analysis of the website usage can be found in Annex B.

3.67 The website attracted a reasonable level of visitors. 3,269 visits were made to the website between February - October 2008. Of these, 78% (n=2,565) were unique visitors. This suggests that 22% (n=704) visited the website more than once, but there could be fewer than 704 unique individuals.

Table 3.6: Overview of Website Usage

3.68 Signposting to the website was generally directly typed in. The majority of visitors to the website typed the address in directly (86%, n=2,795). There were only a small number of visitors who had found the website through search engines or referring websites (7.7% and 6.8% respectively).

3.69 Visitors to the website have fluctuated over time (see Table 3.7). As would be expected, the peak level of visitors was greatest at the time of the launch.

Table 3.7: Visitors and Visits by Month