Evaluation of Regional Selective Assistance (RSA) in Scotland: 2000-2004

Listen

CHAPTER THREE ESTIMATING THE BUSINESS LEVEL IMPACT OF RSA FINANCIAL ASSISTANCE

Introduction: the fundamental econometric problem

3.1 The essential econometric problem is to determine the impact that RSA had on firm performance. The first stage is to determine whether RSA-assisted firms grow faster than non-assisted firms. As is discussed in detail below, and in Appendix 2, there are certain econometric considerations associated with this. These are whether there are sample selection issues, or whether, in the case of inward investors, the availability of RSA is essentially determined at the same time as the investment decision. The results of this analysis are in themselves informative, as they highlight distinctions between types of firms, and the different processes in the allocation of RSA.

3.2 In these data, there are three potential measures of performance: output, employment, and a proxy for productivity (i.e. sales per employee). Output is perhaps the most intuitively appealing, though also the most problematic, as output in this context is measured by sales, which of course are the outcome of two structural equations, both of which include the price of the final good, which is unobservable. As such, one can estimate a sales equation without the identification problem, but it is likely to perform poorly. Productivity can be measured in any number of ways, though this again relies on good data, and proxying this as output/employment, without good information on the capital stock is again unreliable. The third measure is employment, which in general is perhaps the least appealing measure, as observed employment is an outcome of the supply and demand effects in both goods and factor markets. However, for the purposes of RSA it is particularly attractive, as one of the main aims of the RSA Scheme is to boost employment. Notwithstanding these caveats we undertake estimations on all three measures of performance in order to shed light on the ability of RSA assistance to address overall economic objectives within the Scottish economy, and especially productivity.

3.3 Within an econometric framework, the standard approach is to regress employment on a measure of RSA incidence, and a set of control variables, to determine whether RSA support does indeed boost employment. Notice that this assumes a direction of causality here that may or may not be testable within the data, that firms in receipt of RSA tend to grow faster, rather than, for example, firms with high growth potential are better at obtaining RSA. This is discussed at length in the work by Harris and Robinson (2005) on RSA and its effects. With a survey of cross sectional data, ascribing this causality can be problematic, and while our survey was conducted at a point in time, we do have employment and output data for the period prior to, and after RSA was awarded. We can, therefore, to some degree address this causality problem. The analysis presented here offers two approaches to the problem. The first is an approach based on all of the firms in the sample, irrespective of whether or not they are RSA beneficiaries, while the second looks at the subset of firms that are assisted.

3.4 There is one further potential problem, that is particularly important for policy evaluation, and that is the nature of the relationship between firm performance and policy instruments. It is reasonable to assume that "good" firms - those with high levels of managerial competence, good products, good processes, highly innovative etc. will be those that will perform well and grow. Equally, it is reasonable to assume that such firms are also those that are attractive for policy makers, especially at a local level looking to boost output and employment. Therefore, if one were to run a simple regression of growth against RSA, and detect a positive result, this positive implied effect of RSA may be subject to an upward bias, as it may be that better performing firms are also better at attracting RSA. In other words, we may have a strong correlation, but not necessarily a reliable inference for policy makers. Equally, if RSA is to an extent targeted at ailing firms, then a standard regression based on ordinary least squares OLS will understate the importance of RSA to the recipient firm compared with the counterfactual.

Addressing the problem of selection or endogeneity

3.5 The econometric issues surrounding this type of problem are discussed in more detail in Appendix 2, therefore here we summarise. There are numerous ways of addressing the issue of sample selection, all of which rely in some sense on being able to treat the policy instrument, not as an exogenous variable (something determined outside of the system) but as an endogenous variable, that is that the likelihood of a firm receiving RSA support is in part dependent on the quality of the firm. Before discussing the alternative econometric approaches, it is important to consider the nature of the sample of firms that one is dealing with in such studies. In theory, all firms within a geographic (and sometimes sectoral) domain dictated by the particular policy are potential recipients for support. One then has information on all recipients (and sometimes either unsuccessful applicants and even successful applicants who subsequently did not follow through), but no information on those firms that either considered applying and did not, or those firms that for whatever reason did not consider applying.

3.6 One possible solution, therefore, to this is to use a sample selection "Heckman" model. This is discussed in detail in Appendix 2. These are common in industrial economics, for example in relating ownership change to firm performance - is it that firms who have been taken over perform better (or worse) or is it that good (or bad) firms are the targets for takeover.

3.7 This approach firstly involves estimating a Probit model, which seeks to explain the probability of a firm being a recipient of the policy. The purpose of this is to test whether there are any common factors in RSA beneficiaries, and also to capture additional characteristic differences between assisted and non-assisted firms that could not be identified when the sample was constructed. 16 If, however, the probit fails to identify any systematic patterns in the recipient firms, then subject to the other considerations above, one can again revert to OLS. The results presented here are suggestive of selection issues, or at least that one can explain a high proportion of the probability of a firm obtaining RSA with just a few key variables.

Limitations and alternatives

3.8 However, it is also fair to say that this approach has its limitations, and while we assume that this approach is appealing to policy makers it is important to consider alternatives. This also depends on what one assumes with respect to the process of the allocation of RSA financial support.

3.9 Before coming to the estimation, it is important to consider the nature of the data discussed above. RSA beneficiaries broadly fall into two categories. Firstly, single plant domestically owned enterprises seeking relatively small amounts of funds, and secondly large (typically foreign) multinationals seeking large amounts. The work by Harris and Robinson (2005) focuses to a large degree on the latter, while it is clear that in terms of both size and scope the former group became more important in the later years of the sample period. Rather than simply pooling all of these firms together, we therefore present three sets of estimates, one for the full sample, and one for each of these two subgroups separately.

3.10 The sample selection model essentially assumes that there exists a given set of projects or potential projects, some of which are eligible for RSA. Investors then bid for RSA funding, and based on a set of criteria, funding is then awarded to those projects to which the scheme is most applicable. This seems most appropriate for those firms who are UK owned single site firms. Here, RSA is less important in the location decision than it is for multi-plant firms, though the evidence hitherto suggests that it is important for future development.

3.11 However, for foreign-owned firms (or indeed multi-plant firms more generally, be they locally or foreign owned) the allocation decision is rather different. This is not a matter of selecting "potential winners" or dealing with applications to determine whether an existing investment project meets a set of criteria, so much as RSA being determined simultaneously with the investment and location decision of the firm. The case studies, and a large literature on the location decisions of firms confirms this, see for example Driffield (2004), and as such the econometric problem with determining the effect of RSA on the employment growth of foreign subsidiaries is perhaps not so much sample selection as endogeneity.

3.12 It is relatively easy to test whether a variable may be treated as exogenous, though in general these tests are recognised as having only relatively weak power. If a variable cannot be treated as exogenous, a common way of dealing with such problems in econometrics is to find an "instrument" - that is, a variable which is correlated with the endogenous variable, but is determined outside of the model. A fundamental problem in policy evaluation is that there are seldom good "instruments" for policy initiatives - as they are in general designed in some sense to correct market failure, and the policy maker attempts to gather as much information as possible about the firm when implementing the policy. We test the assumption of exogeneity of RSA in the employment growth equation. This is strongly rejected in the case of multi-plant firms, and borderline for the full sample and for UK singles (the probabilities that RSA is exogenous are 0.06 and 0.14) respectively. It is well known however that such tests are sensitive to the choice of instruments, and to the issue of identification which is further discussed in the annex to this chapter. The convention, therefore, with such results is to proceed with caution, on the understanding that OLS is likely to produce biased results.

3.13 Perhaps the most simple is the straightforward "two stage least squares" or "instrumental variables" approach outlined above - subject to being able to obtain a suitable instrument. Finally, one can adopt a more conventional sample selection model, which tests for sample selection bias, but this only estimates the growth effects on the selected firms. This is similar to Heckman's "other" famous model, the treatment model.

3.14 Once one has determined whether endogeneity or sample selection bias exists, the issue is then how to proceed, and how to distinguish between these approaches. Here, we rely on what is intuitively appealing and consistent with previous literature on small firms, FDI and financing, along with some basic econometric tests.

3.15 In order to generate the final models, we draw upon a range of variables (obtained through the bespoke survey for this evaluation) derived from theory that have been shown to relate to the performance of firms and plants (e.g. R&D, innovation activity, business strategy, management capacity) as well as the descriptives from the previous chapter in order to generate some priors. These include, for example, that nationality is important, along with past growth, and the regional embeddedness of firms may also in part determine the probability of a firm applying for, and receiving RSA assistance. These priors were then coupled with a general-to-specific approach, where successive models were run, excluding insignificant variables until a parsimonious form was obtained for each of the three samples.

Timing of effects: some issues

3.16 What we are modelling is the performance (particularly employment given the objectives of the scheme) of the RSA-assisted business after an input of a grant assistance and embedded within this model is the counterfactual position represented by a group of non-assisted businesses. In the case of the safeguarded component of assistance, either on its own or with assistance to create jobs as well, we can make the assumption that the employment performance of the firm or plant in the period 2004-06 would have been different from unassisted firms or plants as a consequence of receiving the financial subsidy. For example, and taking the positive outcome of RSA assistance, they may have declined less quickly than similar unassisted businesses. Further, the intervention to safeguard jobs may have served to keep the firm or plant in business and as a result enables it to be 'present' at the start of the 2004-06 period. As such, its performance in this period is included in the model which seeks to assess whether the RSA assistance parameter is significantly associated with net employment creation in the 2004-06 period.

3.17 There is another broad issue with regard to the timing of the financial assistance received, especially with respect to the safeguarded jobs. The conceptual problem here is that we are conducting a cross-sectional econometric analysis to isolate the effects of RSA assistance when in effect the assistance has come at varying 'distances' back from the start of the modelling period for employment change - that is, 2004. At the design stage of this evaluation these issues were considered and the decision was taken that within the constraints of an evaluation using cross-sectional survey data modelling the effects of RSA assistance received in the 2000-04 period for the 2004-06 period was the only feasible option available. However, we have undertaken some sensitivity analyses which adjust the assistance and impact period to confirm the robustness of this conclusion.

3.18 This decision does, however, create a number of specific issues. First, the nature of the RSA assistance package allows businesses to draw down the monies offered over a 3-year period which means that a business receiving an offer in the first quarter of 2004 will not perhaps have fully realised the benefits of assistance and therefore, the model of employment change in the 2004-06 period may under-estimate the effects of assistance. Second, assistance received to safeguard jobs at the start of the period for this evaluation (i.e. 2000 or 2001) will have had its effect on the firm or plant and to model employment change in a period far removed from the point of assistance may be problematic and lead to an over-estimate of the effects of assistance. We argue, however, that the assistance may have kept the firm in existence for the following 6 years (i.e. between 2000 and 2006 when we surveyed the firm). This is, of course, conjecture as we simply do not know what the answer to this question is. Third, there is an assumption that the actual realisation of the effects of RSA assistance that was received by businesses in 2002 or 2003 will be fully captured by the models presented below. This may not be the case and again may under-estimate the effects of RSA assistance.

The probit estimates

3.19 The first stage in estimating the impact of RSA on beneficiaries is, therefore, the development of a series of Probit models of the probability of receiving RSA. There are two purposes for doing this. Firstly, as outlined above, to test for any selectivity bias in RSA and its subsequent growth effects, and secondly to identify any elements of the targeting of policy which are not 'controlled' for by the structuring of our sample survey of non-recipients. Initial estimates are reported in Tables 3.1 to 3.3. Three models are reported for different sub-samples, with slightly different specifications. We estimate the model for the full sample of Scottish firms, and then separate models for single plant and multi-plant firms.

Table 3.1: The Full Sample Probit: Dependent variable is receipt of RSA (0/1)

Parameter

Estimate

Dp/dx 17

t-statistic

P-value

Constant

-1.136

-0.348

-2.78 **

[.005]

Employment 2004

0.002

0.001

2.20 **

[.028]

Employment growth 02-04

1.022

0.313

3.32 **

[.001]

Business Plan

0.281

0.086

1.74 *

[.0853]

Firm Age

-0.008

-0.002

-1.81 *

[.071]

Exporter

0.592

0.181

2.54 *

[.012]

R&D

0.486

0.149

2.45 **

[.014]

Partnership

-0.692

-0.212

-1.46

[.144]

MD has at least 20% equity

0.366

0.112

1.78 *

[.074]

Local customers

-0.481

-0.147

-2.20 **

[.028]

National customers

0.499

0.153

2.03 **

[.043]

Customers in EU

0.712

0.218

1.85 *

[.065]

Sector dummies

yes

Number of observations = 235
Scaled R-squared = .293755
Number of positive obs. = 120 LR (zero slopes) = 72.2823 [.000]
Mean of dep. var. = .510638 Schwarz B.I.C. = 186.751
Sum of squared residuals = 42.5181 Log likelihood = -126.695
R-squared = .276023
Fraction of Correct Predictions = 0.736170

3.20 Overall, it is perhaps surprising that none of the nationality or ownership variables are significant in explaining the probability of firms obtaining RSA. However, what is noticeable is that variables that may be considered indicative of dynamism are important in explaining the probability of a firm receiving RSA. Exporters, or firms with a national customer base are some 30 per cent more likely to be RSA beneficiaries than firms who only serve their local market. Equally, firm age is negatively associated with RSA, while firms that carry out R&D at the site in question are 15 per cent more likely to be RSA beneficiaries. There is some evidence that firm ownership matters, in that firms where the managing director has at least a 20 per cent ownership stake in the business are more likely to be RSA beneficiaries in Scotland, while partnerships are less likely to receive RSA. Both of these effects however are relatively weak. 18

Table 3.2: The UK Single Plant sample: Dependent variable is receipt of RSA (0/1)

Parameter

Estimate

Dp/dx

t-statistic

P-value

Constant

-0.599

-0.209

-1.82 *

[.068]

Employment 04

0.008

0.003

2.22 **

[.026]

Business Plan

0.467

0.163

2.84 *

[.005]

Firm age

-0.011

-0.004

-2.39 **

[.017]

Exporter

0.197

0.069

2.88 **

[.004]

R&D

0.322

0.113

1.43

[.153]

MD has >20% equity

0.286

0.100

1.26

[.209]

Number of observations = 165
Scaled R-squared = .127822
Number of positive obs. = 97 LR (zero slopes) = 21.4645 [.003]
Mean of dep. var. = .587879 Schwarz B.I.C. = 121.499
Sum of squared residuals = 35.0151 Log likelihood = -101.075
R-squared = .124092
Fraction of Correct Predictions = 0.654545

3.21 With relatively few variables, the model predicts well, and again suggests that internationally orientated firms are more likely to be beneficiaries of RSA, as are younger firms. The key variable here is the Business Plan variable. SMEs that have an existing business plan (usually associated with seeking finance from elsewhere) are more likely to obtain RSA than those without 19. Overall, these results are encouraging for policy makers, in that young, dynamic firms with international links are more likely to be beneficiaries of RSA than those with merely a local focus. This may be in part a function of the eligibility criteria, but still suggests that the allocation of RSA resources worked well. This is potentially good news for policy makers, in that it suggests that domestic recipients are more dynamic than average with higher rates of growth. This is again important when seeking to relate RSA to subsequent growth.

Table 3.3: The Multi-plant Sample: Dependent variable is receipt of RSA (0/1)

Parameter

Estimate

Dp/dx

t-statistic

P-value

Constant

-1.050

-0.323

-4.04 **

[.000]

Employment 04

0.001

0.000

1.62

[.105]

Employment growth

1.449

0.446

3.07 **

[.002]

Exporter

0.345

0.106

2.16 **

[.032]

R&D

0.528

0.163

1.85 *

[.064]

Number of observations = 99
Scaled R-squared = .258158
Number of positive obs. = 41 LR (zero slopes) = 26.5357 [.000]
Mean of dep. var. = .414141 Schwarz B.I.C. = 69.9698
Sum of squared residuals = 18.2345 Log likelihood = -53.8868
R-squared = .240870
Fraction of Correct Predictions = 0.717172

3.22 The essential problem here is that very few inward investors are not at some point in receipt of RSA. As a result, while the matched non-beneficiaries include a sample of multi-plant firms, many of these are UK firms rather than foreign ones. Equally, it is reasonable to assume that domestic multi-plant firms also have a degree of choice as to where within the UK they locate. This is, therefore, suggestive of an endogeneity problem rather than a sample selection problem. However, country of ownership is not related to RSA, but these results again suggest that policy makers favour firms that are exporters and carry out R&D locally. What is perhaps equally surprising is that the model predicts over 70 per cent assisted/non-assisted accurately, with a relatively balanced number of beneficiaries and non beneficiaries, with only a few key variables.

3.23 Bringing the results of the probit estimates together we can justify the rationale for our modelling approach as the selection issue had been clearly revealed. Put another way, we might conclude for example, that firms that are better able to identify new investment projects, or are in more innovative or high growth industries, may be more likely to obtain RSA, as they are more likely to put forward viable proposals. Equally, firms better able to draft good proposals may be better able to generate growth. The modelling work to derive estimates of the impact of RSA assistance in the employment growth equations now seeks to control for this selection issue.

The employment growth equation results 20

3.24 In this section we report the estimates of the impact of RSA assistance during the 2000 to 2004 period on subsequent employment growth (over the 2004 to 2006 period). In each case our performance models are estimated for log growth rates in employment to allow for the standard log-normal distribution of business growth rates. There are two reasons why we focus on employment. Firstly, that employment is the key measure of performance from the perspective of the scheme, and secondly without having detailed information on prices, then employing turnover as a measure of growth is likely to be biased. As suggested above, and given the considerations outlined in Appendix 2, the preferred approach is to assume sample selection for UK single site operations, that is, that there is a link between management ability and the ability to get RSA, and endogeneity rather than sample selection for the multi-plants. For comparison, we present the results from both estimations for the full sample of firms. Further, the OLS estimates are also presented, though we advise caution when reading the OLS results for the reasons outlined above and in Appendix 2.

3.25 The results presented in Table 3.4 suggest that, overall, there is indeed a sample selection issue, but that it acts if anything to understate the growth effects of RSA. This means that RSA has a positive effect on employment, but for the sample overall RSA did not go to the best performing firms. The specific reasons for these results become clear when one looks at the UK single plants and multi-plant subgroups separately. However, these results are broadly good news. RSA is positively related to employment growth, and the estimates are broadly consistent across all of the estimators. It is also clear that many of the individual factors associated with RSA in Table 3.1, are negatively associated with growth. UK multi-plant firms have grown faster than others, while Japanese and SE Asian firms have not. Firms that source through imports have grown slower, and not surprisingly, firm size is inversely related to growth.

3.26 RSA is again strongly associated with firm growth, but again this suggests that it is given to firms who perform worse than average. As such, the OLS coefficient (column 3 in Table 3.4) on RSA is biased downwards compared with the Instrument Variable or sample selection estimates (columns 1 and 2 in Table 3.4).

Table 3.4: The Full Sample

Heckman Stage 2

Instrumental Variables (2 SLS)

OLS

Variable

Coefficient

t-stat

Coefficient

t-stat

Coefficient

t-stat

Constant

0.011

0.21

0.011

0.17

0.099

2.44

RSA

0.595

4.33 **

0.570

3.64 **

0.270

5.72 **

Employment growth 02-04

-0.102

-1.25

-0.119

-1.20

0.018

0.26

Employment04

0.000

-1.66

0.000

-1.01

0.000

-1.30

R&D

-0.109

-2.06 **

-0.117

-1.90

-0.057

-1.21

Ownership SE Asia

-0.274

-2.11 **

-0.301

-2.11 **

-0.288

-2.35 **

Cross price elasticity

-0.092

-1.35

-0.100

-1.33

-0.058

-0.93

Selection term 21

-0.218

-2.49 **

R-squared = .233322
Adjusted R-squared = .176531
LM het. test = .017380 [.895]
Sum of squared residuals = 19.4174 Ramsey's RESET2 = .369453 [.544]
Std. error of regression = .320527
F (zero slopes) = 4.10844 [.000]
Log likelihood = -49.5643

R-squared = .188872
Std. dev. of dep. var. = .353216 Adjusted R-squared = .173374
Sum of squared residuals = 23.6903
F (zero slopes) = 2.31629 [.007]

R-squared = .212917
Adjusted R-squared = .163724
LM het. test = .267645 [.605]
Durbin-Watson = 1.97178 [<.807]
Sum of squared residuals = 21.3860
Ramsey's RESET2 = 1.20139 [.274]
F (zero slopes) = 4.32823 [.000]
Log likelihood = -55.2708

3.27 As discussed in detail in Appendix 2, and in the case studies, there are good reasons for assuming that the growth patterns of single plant firms, and plants owned by multi-plant (including multinationals) may differ. Also, the mechanisms by which RSA was allocated to these types of firms may also differ. The key issue in the case of domestically owned single plants appears to be one of sample selection - that is where firms who are better at obtaining RSA may be more successful for other reasons. In the case of multi-plant firms however, the issue is more likely to be endogeneity, that is, that the award of RSA is determined along with the development path of a given plant, not as a separate issue. This is discussed in more detail in Appendix 2, but for this reason, we adopt the selection model for domestically owned singles, and the instrumental variables approach for the multi-plant firms. For comparison however, we report the OLS estimates in each case.

Table 3.5: Domestically-Owned Single Plants Growth Models (number of observations = 165)

Heckman Step 2

Instrumental variables

OLS

Variable

Coefficient

t-statistic

Coefficient

t-statistic

Coefficient

t-statistic

Constant

-0.220

-1.93

-0.175

-1.51

-0.051

-0.81

RSA

0.619

3.10 **

0.590

2.54 **

0.258

4.70 **

Employment 04

-0.003

-3.37 **

-0.003

-2.88 **

-0.001

-2.34 **

R&D

-0.106

-1.71

-0.091

-1.34

-0.040

0.71

Product innovation

0.125

2.27 **

0.084

1.23

0.084

1.56

Growth as main objective

0.075

1.33

0.045

0.66

0.075

1.38

Selection term 22

-0.251

-2.08 **

R-squared = .242649
Adjusted R-squared = .187841
LM het. test = .103027 [.748]
Sum of squared residuals = 16.8722 Ramsey's RESET = 1.12324 [.291]
Std. error of regression = .333169 F (zero slopes) = 4.42724 [.000]
Log likelihood = -46.2217

R-squared = .184579
Adjusted R-squared = .171284
Sum of squared residuals = 21.7896
F (zero slopes) = 2.71877 [.004]

R-squared = .202320
Adjusted R-squared = .154554
F (zero slopes) = 4.23571 [.000]
Ramsey's RESET = 1.58765 [.209]
LM het. test = .096817 [.756]
Sum of squared residuals = 19.5336

3.28 The results for the domestically-owned (i.e. Scottish) plants are strongly suggestive of the sample selection issue, though perhaps not in the manner that one would imagine (Table 3.5). The probit results discussed above suggest that larger firms within this group are more likely to receive RSA, but that after receiving RSA firms do not grow as fast as the non-assisted firms. However, it is also clear that beneficiaries do better than they would have done without RSA. The reason for this is that RSA is offered in order to secure jobs, as well as to create new ones. In the case of domestic singles 23, the total "secured jobs" protected by RSA in Scotland was 819 over the period, while the number of new jobs that firms claimed would be created was 1,435. This explains the negative sample selection term as only RSA beneficiaries can have "secured jobs" in this way. The process is that on receipt of RSA a given number of existing jobs are created, hence the positive correlation between RSA and employment growth. However, given that firms can apply for RSA support to secure jobs, it is likely that some of the beneficiaries would have reduced employment without RSA. As such, while these firms do better than they would have done without RSA, they are unlikely to do as well as the control group who have not applied for funds to secure jobs.

Table 3.6: Multi-plant firms (including foreign-owned) (Number of observations: 99)

Heckman step 2

Instrumental variables

OLS

Variable

Coefficient

T stat

Coefficient

T stat

Coefficient

T stat

Constant

0.038

0.59

0.034

0.50

0.179

2.44

RSA

0.260

1.64 *

0.278

1.89 *

0.256

2.76 **

Employment 04

0.0002

-1.15

0.0002

-1.21

0.0003

-1.37

New products

0.094

1.25

0.091

1.24

0.052

0.48

Standardisation of processes

-0.096

-1.39

-0.099

-1.48

-0.212

-2.14 **

Sample selection term

0.022

0.21

LM het. test = .061653 [.804]
Sum of squared residuals = 22.2900
Adjusted R-squared = .134919
Log likelihood = -68.6265
F (zero slopes) = 3.18345 [.003]
R-squared = .196710
R-squared = .311899
Adjusted R-squared = .250734
Sum of squared residuals = 7.99033
F (zero slopes) = 2.95541 [.006]

R-squared = .312366
Adjusted R-squared = .242830
LM het. test = .644124
Sum of squared residuals = 7.98340
F (zero slopes) = 4.49214 [.000]
Log likelihood = -15.8460

R-squared = .196710
LM het. test = .061653 [.804]
Sum of squared residuals = 22.2900
Adjusted R-squared = .134919
Log likelihood = -68.6265
F (zero slopes) = 3.18345 [.003]

3.29 For the multi-plant group (Table 3.6) the impact of RSA is smaller than for the domestic singles, but still significant and positive. The three estimators are very close together in terms of the impact that RSA has on employment, which is perhaps not surprising as the Inverse Mills ratio does not suggest a significant sample selection bias here. In general, however, these models do not perform as well as the domestic single plant models. The most likely explanations for this are the relatively small sample size, and the large degree of heterogeneity in the sample, ranging from relatively small UK-owned firms with perhaps only two plants, through to subsidiaries of foreign multinationals.

3.30 These results highlight the distinctions between the two types of firms, but even further suggest at least two groups within the multi-plant sample. For example, multi-plants that have been seeking to grow, or introduce new products have been successful in this, though this appears unrelated to RSA. This is all indicative of there being a group of inward investors for whom RSA was an incentive to stay in Scotland rather than to come to Scotland. This result is robust to the inclusion of the country dummies. There is no positive relationship between employment growth and jobs created or secured by RSA. Indeed, while firms may have promised new jobs when in receipt of RSA, these are not related to subsequent employment growth. This is not to say, necessarily, that such firms do not meet their job targets, merely that multi-plants in receipt of RSA employ the number of people specified at the outset, and then seldom grow in the short term, and indeed grow slower than non-assisted firms. The number of secured jobs, and new jobs created by these firms is equal at 1,317, so it is perhaps not surprising that the models do not work well.

Capturing safeguarded jobs in the model: sensitivity analysis

3.31 As we discussed above an important issue that emerged in the discussion of these results is the ability of the employment growth model to capture both elements of the RSA assistance package, that is, job creations and safeguarded jobs 24. We have already seen in Chapter 1 that the financial assistance provided to firms and plants under the RSA Scheme was a combination of job creations only, safeguarded jobs only or a mixture of the two. It is our view, following the reasoning set out above, that a substantial proportion of the employment effects of the financial assistance received by a firm or plant in the period 2000-04 will be captured in the model of employment change in the 2004-06 period, irrespective of the proportion of job creations and safeguarded jobs in the assistance package offered.

3.32 However, in the calculation of the cost-per-job estimates for UK singles and multi-plant firms presented in the next chapter we may understate the importance of safeguarded jobs, as the estimation is based on the net employment gain in the period 2004-06. If employment retention was not an issue for unassisted firms, then the cost-per-job calculation may understate the importance of RSA assistance, as the relative importance of job retention (i.e. 8,531 jobs) may be understated. Nevertheless, a large proportion of these jobs are included in the sense that the performance of these plants in the 2004-06 period may be associated with the financial assistance received at some stage in the 2000-04 period. That is, as we state above, the assistance received may have enabled them to be still in existence in the 2004-06 period, and as such 'counted'.

3.33 Does the nature of the over- and under-estimates average themselves out over the RSA beneficiaries in the sample? In short, we do not know the answer to that question for sure but what we can do is re-specify the employment growth model in order to introduce a sensitivity analysis into the results. Two approaches were considered to test for this. The first was to effectively impose a two year time constraint for all of the employment effects to have occurred, and to estimate for several subsets of the data over a two year moving window. This was rejected for several reasons. Firstly, one is concerned that all of the employment gains from RSA may take longer than two years to materialise, and secondly that this would render it impossible to carry out a proper sample selection model based on previous performance. In addition, many of the sample sizes would become small, making it difficult to capture all of the inter-firm variation, and the other determinants of employment growth.

3.34 Therefore, we undertook an alternative sensitivity test, by carrying out the estimation discussed above for the sub-sample of assisted firms that obtained RSA between 2002 and 2004, effectively seeking to ensure that we captured the "safeguarded jobs" effect within the employment growth period, which we re-specified to be 2002-06. While we have several reservations about this, such as sample size and the fact that the matching process with non-recipients was not designed to allow for this (though it is not clear how one could practically do this other than by doing a matching process for each year before constructing a stratified sample), this was done, and the results summarised below.

3.35 Rather than present all of the results again for the sub-sample, Table 3.7 provides the coefficient on the RSA term for the various estimation procedures for the basic employment growth equation, across the full sample and the two sub-samples. It also provides the coefficient on the "grant size" variable for the final sample selection model.

Table 3.7: Employment Growth Model (2002-06) - Sensitivity Analysis

Full sample

Domestic (Scottish) singles

Multi-plant firms

Coefficients on RSA term:

Heckman

0.215

(1.64)

0.363

(1.66)

0.095

(0.437)

Instrumental variables

0.220

(1.62)

0.370

(1.34)

0.072

(1.15)

Coefficient on Grant size

(sample selection model)

0.077

(0.392)

Sub-samples too small to be valid.

Sample size

( RSA recipients)

214

(76)

134

(56)

80

(20)

Note: (t values in parentheses).

3.36 It is noticeable when comparing the models that in each case the coefficients and implied elasticities are smaller for this sub-sample than for the overall sample. However, the fact that there is less time for the created jobs to occur and be captured by the model will reduce these numbers so we cannot be sure they represent good estimates of the RSA effect. More importantly, these differences are not significant, though obviously these point estimates would suggest marginally higher cost-per-job estimates. From these results we conclude that the concern that using all the 'in scope' assisted firms/plants in the 2000-04 period in the econometric analysis would lead to a failure to capture the full "secured jobs" effect is unlikely. In other words, the tendency of the base line estimation to understate job retention is less important than the tendency of imposing a window of arbitrary (shorter) length to capture the full employment effects of RSA assistance.

Summary

3.37 Overall, these results, which include a counterfactual element, are broadly supportive of RSA interventions and encouraging for policy makers. The major points to emerge from the analysis can be summarised as follows:

  • RSA assistance did have an impact on employment growth: assisted firms do better than they would have done without RSA
  • Within the context of eligibility criteria for the Scheme the allocation process appears to have worked well with those firms who received financial support from RSA being younger, more dynamic and internationally focussed than those that did not.
  • The determinants of RSA allocation differ between single and multi-plant firms. R&D is more important for the multi-plant firms, while age is only important (inversely) for the single plant firms.
  • This highlights the need for sample selection modelling as one has to allow for the fact that such firms may have grown faster than average anyway, or that firms best able to obtain RSA support are "better" than the average.
  • Even when one allows for sample selection or endogeneity in the effects of RSA, there is a positive relationship between RSA and employment growth.
  • For the multi-plant group the impact of RSA is smaller than for the domestic singles, but still significant and positive.
  • The results suggest that there exist at least two groups within the multi-plant sample. Inward investors that have been seeking to grow, or introduce new products, and have been successful in this, though this appears unrelated to RSA. The second is a group of inward investors for whom RSA was an incentive to stay in Scotland.

Page updated: Thursday, March 20, 2008