The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER SEVEN VALUE FOR MONEY

7.1 A key consideration when evaluating a public sector intervention is the extent to which it has delivered value for money.

7.2 Value for money (VfM) is the term used to assess whether or not an organisation has obtained the maximum benefit from the services it provides, within the resources available to it. It not only assesses the cost of service provision, but also takes into account the mix of quality, resource use, fitness for purpose, timeliness and convenience to judge whether or not, when taken together, good value has been achieved.

DETERMINING VALUE FOR MONEY

7.3 In the absence of a study with a sufficient longitudinal element to measure change over time ( e.g. in a firm's productivity or an individual's wage rates), the snapshot data from the SQW survey and the other sources presented in this report can be used to indicate the likely value of SUfI's input.

7.4 In devising our approach, we have built on authoritative literature sources which provide insights into how the value for money of SUfI's activities might be assessed. These are discussed below. Data from the present study will then be considered.

Valuing benefits to individuals

7.5 As far as individuals are concerned, the returns to some form of learning or training are often related to increases in wages, with the scale of the returns being greater the higher the qualification obtained.

7.6 For example, based on an analysis of UK Labour Force Survey ( LFS) data, Dickerson (2006) asserts that 'on average, returns to low level qualifications are negligible or zero. At all other levels, the returns are quite substantial - approximately 12-15% for both level 2 and level 3, and rising to more than 20% for level 4 and level 5. Interestingly, Dickerson also highlights the difference in the type of return between academic and vocational qualifications, with the former being much greater than the latter at all levels apart from level 5.

7.7 Research by Blundell (1996 &1999) found that individuals undertaking employer-provided or vocational training boosted the real wage of men by, on average, some 3.6% (for on-the-job training) and 6.6% (for off-the-job training). For women the effects of training in the current job were 4.8% (on-the-job training) and 9.6% (off-the-job training). Training in previous jobs also had a positive effect on wages. These estimates refer to training which did not lead to a qualification. Higher vocational qualifications boosted male wages by about 8% on average, and female wages by about 10%.

7.8 Arulampalam et al. (1997) looked at the effects of both training and educational courses on wage growth, for males only, over the period 1981 to 1991. The results indicated that receipt of training increased wages by about 11% over the period 1981 to 1991 (comparing those who experienced at least one training episode to those receiving none). The size of the effect falls after the first five years of completing the training event by 0.44% per month.

7.9 Greenhaugh (2002), summarising the literature, concludes that the estimates using British data indicate wage gains after training for workers of all ages in the range of 3% to 10%, while for younger workers the gains are estimated from 11% to 21%.

7.10 Although this study does not allow any direct measurement of the impact of the additional learning attributed to the intervention, it is possible provide indicative values by using these studies as a guide to the scale of effect that might be expected.

Employers

7.11 In terms of employers' investment in training, the most commonly applied measure of the impact of this investment is on the productivity of the enterprise. Keep and Mayhew (2002) state that 'the overall conclusion from the majority of studies is that there is a positive relationship between increased levels of skill and productivity, but little agreement about the magnitude of the effect'. Nonetheless, this enables a positive value to be placed on the participation of employers in training, as a result of contact with SUfI.

7.12 Attempts have also been made to establish a relationship between investment in training and company profitability, although here the evidence is more equivocal (Keep and Mayhew, p3).

7.13 Other factors which may be taken into account include 'hard' measures such as: plant down-time; rates of absenteeism; and labour turnover, as well as 'softer' measures, such as levels of customer satisfaction; and employee motivation and commitment (Keep and Mayhew, pp 14-15).

7.14 While there appears to be general acceptance of a positive relationship between enhancing the skills of the workforce and productivity, this is often tempered by an acknowledgement that this impact usually requires other aspects of people management to be effective.

7.15 One of the most influential studies, and of more utility for the present study, is that of Dearden, Reed and Van Reenen (2000). On the basis of their analysis of Labour Force Survey data which focused on the effect of training on value added per head in the UK, these authors conclude that 'an increase of five percentage points in the proportion of employees trained is associated with a 4 percent increase in productivity' (p 53). In a further paper in 2005, the same authors report the impact of training on productivity was calculated to be twice the effect on wages.

SCOPE OF BENEFITS

7.16 The purpose behind the allocation of public funding to SUfI is to engage two key target groups in learning: (1) individuals who are least likely to be interested in learning and (2) SMEs. Drawing on the data derived from the study, the anticipated benefits accruing to both individuals and employers who have initiated contact with SUfI can be assessed in the light of the costs involved.

7.17 In order to approach an assessment of value for money, the benefits arising from SUfI are summarised below before considering these in relation to the total costs incurred. In Table 7.1 the key quantifiable outputs and outcomes, as indicated by SQW's survey results, are summarised. In order to assess value overall relative to costs, the net additional benefits determined from the survey samples are extrapolated to the appropriate populations

7.18 The value of benefits to individuals from enhanced wages and benefits to employers from enhanced productivity are assessed below, as advised by the literature review. However, given the objectives of SUfI, it is also appropriate when assessing value to take into account two other quantifiable measures: individuals engaging with learning and individuals gaining employment.

7.19 It is also appropriate to ensure that "softer" benefits are not missed out of the value for money assessment, even though quantification is not always possible. Therefore, these are also summarised in what follows.

QUANTIFYING THE BENEFITS

7.20 Producing quantifiable measures of the outcomes of engagement with SUfI through learndirect scotland or learndirect scotland for business, on the part of either individuals or companies, and certainly in the short term, is extremely difficult.

7.21 The two main reasons for this are: the fact that, in the case of both individuals and companies, the desired changes in attitude and behaviour are often protracted processes, which require longitudinal research or long-term monitoring to identify; and the complexities involved in attributing impact to specific interventions, when a myriad of influences could be brought to bear.

7.22 However, findings from both the surveys and the different waves of the LTS are indicative of the benefits which have accrued. These are used together with the evidence from literature on the wage returns accruing to training and a series of assumptions to produce, what is best described as a set of indicative estimates of the additional impact of SUfI's activities. Providing some broad estimates is necessary in order to draw any conclusion on value for money.

7.23 Although learndirect scotland received just over 135,000 enquiries in 2005/06, the survey was based on telephone calls rather than all sources (which also includes web, e-mail and fax enquiries). It is unclear what the differences would be, but in developing the indicative estimates, we have extrapolated solely on the basis of telephone calls. This means that the results will underestimate the impacts as they will not include those influenced to undertake learning through web-based (or other) forms of contact. This assumption is particularly important for the business service where the number of calls is a much smaller proportion of enquiries.

7.24 The forecast for 2005-06 was for calls to represent 60% of all individual enquiries. With a projected total of 135,429 enquiries, this would mean 81,257 telephone enquiries. For learndirect scotland for business, the total of 19,428 enquiries, the forecast was for calls to represent 18% of enquiries. Applying this to the projected total of 19,428 enquiries gives 3,497 calls. However, from the KPI data we also know that learndirect scotland's activities engaged 5,257 SMEs in learning in 2005-06. This is a better estimate of the base number of SMEs.

7.25 The following table uses these figures along with the survey data to produce estimates of the number of people engaged in learning and the difference that intervention has made.

Table 7.1: Summary of key benefit estimates

Source: SQW ( LDS = learndirect scotland)

Benefits to individuals

7.26 The following is a summary of benefits to individuals:

• the relatively high proportion of individual enquiries from localities of social deprivation (49 per cent in the LTS survey), pointing to success, on the part of learndirect scotland, in achieving penetration among one of the key target groups, that is the 'hard to reach'. Ultimately, however, it may represent the most powerful indication that SUfI is making inroads into a key area - the achievement of a significant cultural shift in the mindsets of traditional 'non-learners' may not be readily apparent for some years, but these early indications are distinctly promising
• around a quarter of those who subsequently completed courses could be classified as 'new learners', in that they had previously undertaken no formal learning since leaving compulsory education
• the work-relatedness of the motivation for contacting learndirect scotland on the part of the majority of callers (Paras 4.17-4.22) suggests that immediate benefits will be derived, either in finding a job or enhancing career prospects
• from the LTS findings, 35 per cent of all respondents had undertaken learning since contacting learndirect scotland, representing a substantial rate of 'success'
• as well as attracting significant numbers of 'new' learners who subsequently attain a qualification, a quarter of those who were in the process of learning expressed an interest in undertaking further study in the future (4.44). This provides evidence of SUfI's legacy, through its contribution to effecting cultural shifts in attachment to learning
• the less tangible aspects of the beneficial impact of contacting learndirect scotland, and subsequently engaging in learning activity, such as increased self-confidence and self-esteem, were also prominent in the findings, with over a third of the SQW survey respondents citing 'feeling more confident' or 'feeling happier'
• also, importantly, shifts in attitudes towards learning were apparent, with 38 per cent of the respondents claiming to have developed a better appreciation of the value of learning as a result of their experience with learndirect scotland. The difficulties of deriving a meaningful measure of the amount of this benefit have already been explored, but the fact that there was survey evidence to support these claims should not be discounted
• there are other aspects of the effects of increased training activity which are more difficult to quantify. These include levels of job satisfaction, attachment to the organisation on the part of employees, and heightened aspirations and expectations (mentioned by a fifth of respondents to the SQW survey).

7.27 Aside from a concentration on 'hard to reach' groups, the survey findings suggest that those who were who were unemployed and seeking work benefited considerably from their contact with learndirect scotland. This is highlighted by the finding that 22 of the 109 respondents in the SQW sample who had not been employed had since found employment.

7.28 The surveys found that 11% of the sample of those that had completed courses had moved from not being employed into employment. For the population of those that have completed courses (4,875), this would represent 536 individual learners. If all those that are currently studying, completed courses, this would generate a further 2,592 moving into employment. Added together this gives a total of 3,128. Analysis of this sub group indicates that while learndirect scotland was instrumental in helping them find the specific course, many may have enrolled for some form of learning anyway. Among the 22 in the sample, three (14%) were unlikely to have enrolled in a course without learndirect scotland. This indicates that of the population of callers, we estimate that there were 438 additional people moving into work as a result of contact with learndirect scotland.

7.29 Because these figures are in some cases based on relatively small sub-samples, they should be treated with caution and regarded as indicative rather than as formal estimates.

7.30 It is not possible to determine whether these represent additional jobs within the economy. This would depend on whether their new employment was at the expense of other prospective employees. However, it does represent a contribution to increasing the size and level of skills of the workforce and provides evidence that the support has brought new people into the labour market.

7.31 These benefits generally contribute to raising skills within the workforce and increasing the prospects of those not in work. The extent to which SUfI is responsible for these is reflected in Table 7.2 which shows the gross and net figures, using the survey evidence to estimate whether clients would have enrolled for some form of learning anyway.

Table 7.2: Gross to net benefits

Source: SQW
Table notes:

Valuing the impacts to individuals

7.32 The earlier analysis of the literature suggested, broadly, that a period of training is associated with wage returns of between a 3% and 10%. The amounts vary significantly depending on the nature and duration of the training and the circumstances of the employee. To provide an indicative estimate of the effects of the additional training generated by contact with learndirect scotland for business, we have assumed a wage premium of 5% ².

7.33 Using the estimate of 4,470 new learners generated through the Training Partners and calls to learndirect scotland for business, and assuming that those training earn an average of £15,100 ³, a 5% increase in wages would represent an increase of £755 each. This would represent a collective increase in wages of £3.4m.

Benefits to SMEs/employers

7.34 As far as SMEs are concerned, although there is clearly more to be done in creating awareness of learndirect scotland for business, the success, particularly of the TP service, is undeniable. The fact that 77 per cent of SMEs which had engaged with the Training Partner service subsequently claimed to have adopted a planned approach to learning points to significant benefits being derived, on the basis of Dearden et al's (2000) assertions about the impact on productivity.

7.35 There is also evidence from the SQW survey of firms of additional training activity emanating from the input of learndirect scotland for business. While it is possible, notionally, to quantify the benefits accruing to SMEs, in the form of reduced labour turnover, reduction of skills shortages, a more skilled and flexible workforce, productivity gains and company growth and competitiveness, this would require in-depth studies of individual organisations, from which proxies could be derived. Consideration should be given to undertaking such studies in the future, if the full costs and benefits of the impact of SUfI, and therefore the value for money which it represents are to be gauged.

7.36 Nonetheless, the finding that 99 of the 200 SMEs included in the sample had instigated some form of training since calling learndirect scotland for business suggests that considerable immediate benefits have been derived, even allowing for the contention that in only 21 of these cases was the training directly attributable to the learndirect scotland for business intervention.

7.37 Using the projected estimate of 5,257 businesses engaged, the earlier analysis suggested that there were around 4,470 additional employee learners and that this would result in wage increases of around £3.4m.

7.38 The paper, by Dearden ⁴ et al, building on the findings reported in 2000, makes two key observations. Firstly it demonstrates that raising the proportion of workers trained in an industry by one percentage point (say from the average of 10% to 11%) is associated with an increase in value added per worker of about 0.6%. Secondly, it finds that the impact of training on wages is only half as large as the impact of training on productivity. Logically, rational employers would only pay wage increases up to the value of any productivity benefits. Using wages as a proxy for the increase in value added, therefore does not take into account the likelihood that employers also benefit from training, through increased productivity (from which a proportion is paid to the employee).

7.39 The estimate of £3.4m represents thin additional wages would therefore represent the minimum additional value added generated. The evidence above suggests that the benefits to employers, through productivity improvements could be double this ⁵. On the assumptions discussed, this would lead to an estimate of around £6.8m of additional GVA for businesses.

7.40 These estimates are based on one year. In practice the new skills will generate benefits over a longer period of time, although subject to some depreciation. A further factor is that these estimates do not allow for the costs of training, either the direct costs or the foregone output during the period of training.

7.41 While the results relate only to learndirect scotland for business service, many of the individual callers to the main service will also undertake training which will improve productivity. We know from the survey that 22% cited "looking to improve career prospects/gain a promotion" as a motivation for their engagement. Among callers, this represents 1,501 additional learners. If this learning was sufficiently work-related it would add considerably to the economic benefits.

7.42 Finally, we stress again that these extrapolations should be treated as indicative of the scale of benefit rather than formal estimates.

COSTS

7.43 In assessing value for money, it is relevant to compare the benefits that accrue with the costs of the intervention(s) responsible. Although a detailed breakdown of costs for the operation of all aspects of SUfI are available, in the context of assessing value for money it is appropriate to consider the overall costs against benefits.

7.44 In total, the funding bid, set out in the 2005/06 operating plan was for £8.5m. In addition a further £133,000 was invested in a number of other projects. The figures exclude separate funding related to ILA Scotland (£4.0m), which operates separately from learndirect scotland activities.

Table 7.3: Costs (Operating Plan 2005/06)

7.45 The total funding allocated to marketing in 2005-06 was just under £1.6m. Of this, approximately 45 per cent (£711k) was allocated to consumer marketing ( i.e. targeted at individual learners). This produced a total of 135,000 contacts in the year, which equates to a cost of £5.27 for attracting each call.

7.46 Moreover, approximately two thirds of the sample had contacted learndirect scotland as a direct result of the advertising. This would indicate that the £711k investment had generated a response from at least 50,000 individuals.

7.47 In terms of the contact centres, the total funding allocated to their activities in 2005-06 was £917,000. Of this, approximately 93.5% (£857k) can be attributed to the cost of handling calls from individuals ⁶. Taking the figure of 135,000 contacts, the average cost of handling each one is around £6.35. When marketing and contact centre costs are combined, the cost of each contact is £11.62.

7.48 SUfI's recently formed Inclusion Team has a remit to promote learning in local communities and amongst key disadvantaged groups. The team's budget of £163k in the current year is limited. SUfI aims to avoid incurring direct costs, by tapping into existing mainstream mechanisms and activities delivered by other agencies.

7.49 The Inclusion Team has attempted to measure the cost of reaching individuals. They have calculated, for example, that an average event costs about £3,000 to deliver and attracts around 100 people, representing a cost of £30 to reach one person. For larger events, where costs are higher, the cost per individual can go up to £75. The wider benefit, however, is brand building for learndirect scotland. Again, the difficulties of measuring the benefits derived from this activity are considerable. Not only would it be unreasonable to expect that some direct causality for subsequent participation in learning could be attributed directly to attendance at such events, but the wider impact on societal attitudes to learning would be practically impossible to gauge. This model is scheduled for appraisal in 2007.

7.50 It is axiomatic that the cost of engaging with the most disadvantaged individuals, who have not undertaken any learning since leaving school and do not feel that learning is relevant to them, will be high. If this is the case, then this overall level of response is made even more impressive when, as evidenced by the findings of the LTS (para 4.11), almost half of those contacting learndirect scotland were from socially excluded areas.

The cost of engagement with SMEs

7.51 Of the total marketing budget of £1.6m, approximately 27 per cent (£433k) is allocated to business marketing ( i.e. targeted at SMEs). In 2005-06, it was projected that there would be 19,000 calls to the learndirect scotland for business helpline. Thus, the cost of attracting each SME call roughly equates to £23. Certainly, when we consider that each SME that subsequently engaged in training led to an average of 5.4 people going into learning, this is very effective.

7.52 Around 6.5 per cent of the total contact centre costs (£60k) is attributable to learndirect scotland for business. Given the projected number of 19,000 contacts, the average cost of handling each one would therefore, be around £3.10.

7.53 When marketing and contact centre costs are combined, the notional average cost of each SME call to the helpline is around £26. When considered in the context of traditional antipathy among SMEs to engage in training, and to respond to external initiatives, this represents a relatively small investment, especially as it could be expected that, having contacted the service, those SMEs were sufficiently motivated to follow up their interest and to make future investments into workforce training and development.

7.54 The cost of the TP service is £768k ⁷. The number of 'new' SMEs visited over a three-year period is 2,562; therefore the average number of 'new' SMEs visited per annum is 854. This puts the cost of visiting each 'new' SME at approximately £899.

7.55 Some indication of the cost of taking this activity a step further can also be calculated. The number of 'new' SMEs which adopted a planned approach to learning as a result of TP intervention, over a three year period, is 1,983 - roughly equivalent to 661 per annum. Using the same crude method of calculation, the cost of each 'new' SME adopting a planned approach to learning is approximately £1,162. However, this represents planning that will impact on a number of employees in each business

CONCLUSIONS

7.56 Clearly, not all the benefits can be quantified, although they should be taken into account in assessing value for money. For purposes of ready comparison, it is appropriate to compare the total public sector investment of £8.6m with the quantifiable element of net additional economic impact and to argue that, as it discounts the "softer" benefits for practical reasons, it offers a conservative view.

7.57 The results that have been calculated should be considered both indicative and partial, but in our opinion provide a basis for asserting that the contribution of SUfI intervention offers good value for money. Extrapolating from the survey results to produce indicative values we conclude that the overall public investment of £8.6m has generated the following:

• around 6,800 additional learners
• directly moved around 400 people into work
• engagement of approximately 4,500 additional SME employees in learning
• an increase in GVA in Scottish businesses of at least £7m ⁷.

7.58 There are a number of factors that make us believe that the actual impact on GVA is potentially greater than this:

• it excludes the value of bringing new people into the workforce (400)
• there are estimated to be a further 1,500 additional learners that contacted the main learndirect scotland service motivated by career prospects. Their higher output is likely to contribute to business value added, but is not included
• the estimates represent the impact for one year. The impact of the training will lead to continuing productivity benefits over a longer period of time (although depreciating)
• there is evidence from the surveys of significant contributions to individuals' quality of life and future engagement in learning.

7.59 For these reasons the actual contribution of SUfI is likely to be considerably higher than the £7m calculated above. This is in addition to the quality of life benefits reported.

INTERPRETING THE FINDINGS

7.60 Notwithstanding the estimates of value offered above, it is important to note that in the case of SUfI, the issue of assessing value for money is far from straightforward, for a number of reasons. SUfI is not a provider of learning: it is a broker of learning. As such, it provides one component of the process by which individuals, or companies, embark on learning activity. The weight of the SUfI intervention in leading up to the decision to participate, in comparison to the contributions of other components, such as family and peer pressures, workplace encouragement or requirement, or advice and guidance from other sources, will vary considerably between individuals and organisations.

7.61 Second, much of the work of SUfI, whether targeted at individuals or companies, is essentially attempting to inculcate attitudinal or cultural change. Deep-seated changes of this nature may not appear until several years after the programme, and in this time other factors may have played a role achieving impact (Wainwright 2002; Maguire and Killeen 2003). Thus, the time period over which outcome measures are applied is extremely important, as it could be argued that longer-term outcomes, for the individual, the organisation, and society as a whole, are likely to reflect more truly the benefits of SUfI's provision.In measuring the long-term impact of an initiative or programme, it can, again, be difficult to establish causality.

7.62 It should be emphasised that SUfI is driven as much by aspirations as by concrete objectives: it aspires to engage those people who are the most 'hard to reach', as they have exhibited no inclination to participate in learning, or even to consider that learning was for them.

7.63 Rather than looking at a value for money metric, it is more realistic to regard SUfI as part of the education, training and guidance infrastructure in Scotland, whose worth lies in its ability to challenge and influence the status quo and provide an essential link between various key agencies, allowing other initiatives to be more effective than they otherwise would be. Adding in these dimensions of SUFI's strategic added value makes the process of quantifying the organisations value for money even more complex.

OVERALL VALUE FOR MONEY

7.64 The total allocation of funding to SUfI in 2005-06 was £8.5 million, excluding VAT, as core budget.

7.65 Our analysis of the business related benefits of SUfI activity demonstrates a reasonably strong return on investment for the Scottish economy. Of the elements of the service that can be valued, we estimate a contribution to business GVA of at least £7m. When we also consider that this only represents part of the service, the total return, including contributions to employability and quality of life, is further strengthened.

7.66 Overall, therefore, the funding allocated to SUfI represents good value for money when viewed in the context of the potential contribution of the service to the longer-term economic and social goals of Scotland.

« Previous | Contents | Next »