< Previous | Contents | Next >

A Review into Exam Results Issues Concerning the Scottish Qualifications Authority

4. FINDINGS - MANAGEMENT AND ORGANISATION

4.1 Introduction

The Scottish Qualifications Authority is a Non-Departmental Public Body (NDPB) which was established by the Education (Scotland) Act 1996 to replace the Scottish Examination Board and the Scottish Vocational Education Council.

The SQA Management Statement and Financial Memorandum5 is a public document which sets out the relationship between the SQA and Scottish Ministers, together with the policy and financial framework within which the SQA is required to operate and discharge its functions and responsibilities.

The SQA is responsible to the Minister for Enterprise and Lifelong Learning who has the sponsorship role for the SQA. That Minister also has an interest in the SQA related to the Minister's responsibilities for further education, work related and work based qualifications, and in relation to entrance to higher education. The SQA is also responsible to the Minister for Children and Education for that Minister's interests in qualifications in schools.

4.2 Functions and Responsibilities in Relation to the SQA

In summary, the SQA's functions are to:

• devise qualifications, determine the entitlement of a person to qualifications, and award and record such qualifications;
• review and develop qualifications;
• approve education and training establishments as being suitable for presenting persons for such qualifications, including assessment of persons undertaking education and training;
• accredit qualifications;
• exercise a quality assurance function in relation to establishments which offer its qualifications; and
• provide advice to Ministers on any matter to which its functions relate.

The SQA derives its principal income from education authorities, further education colleges, training organisations, higher education institutions, the enterprise network, employers and independent schools by means of entry charges.

The SQA is required by statute to submit an annual report on its activities during the preceding financial year to Scottish Ministers by 30 September each year.

In summary, powers and duties of the Scottish Ministers include, inter alia, the ability to:

• give the SQA (after consultation with it) direction of a general or specific nature with regard to the discharge of its functions. In respect of this power, the written evidence by the Scottish Executive Enterprise and Lifelong Learning Department (SEELLD) to the Enterprise and Lifelong Learning Committee states⁶ that "in general policy terms the power of direction is viewed as a measure of last resort, to be used only in exceptional circumstances where there is a lack of co-operation by, or conflict with, the organisation concerned";

• appoint at least 12, but not more than 19 members of the SQA, including the Chairman. The SQA may appoint up to five members proportionate to those appointed by the Minister, but the SQA must ensure that total membership is not less than 15. The present Chairman, David Miller, was appointed on 18 September 1996, having previously been Chairman of both SEB and SCOTVEC. The Chairman was re-appointed on 1 April 2000 for a further two years. The Chief Executive is also a member of the Board;

18 Board members were appointed on 20 January 1997. 11 of the initial appointments were for a four year period and seven were for two years. A further nine Board appointments were made on 16 March 1999 on the expiry of the initial two year appointments and to fill two resignation vacancies. In line with Scottish Executive policy⁷ departmental officials do not attend meetings of the SQA's Board of Management, Finance, Planning and General Purposes Committee or Audit Committee but officials have the right, in consultation with SQA, to attend SQA Board meetings for specific items when a departmental input is required; and

• make, after consultation with the Chairman, the appointment of the first Chief Executive. Ron Tuck was appointed Chief Executive on 6 January 1997 on a four year contract. The appointment followed open advertisement, and an interview panel chaired at Head of Department level and including independent membership.

The SQA Board, which consists of the Chairman, Chief Executive and up to 23 others, is required to meet at least four times a year. The SQA Board is required, inter alia, to:

• comply with any direction given to it by the Minister;
• provide the Minister with advice on any matter to which its functions relate;
• send a copy of minutes of meetings and related papers of the Board and SQA committees to the Department;
• submit an annual report;
• have an independent Accreditation Committee to accredit qualifications and meet such requirements as are specified by the SQA. This requirement reflects the need for a clear separation between SQA's awarding and accrediting functions to avoid any perception of unfair competition with other awarding bodies whose qualifications are accredited by SQA; and
• have an Audit Committee.

The SQA currently has the following committees⁸ in addition to its required Accreditation and Audit Committees:

• the Chairman's Committee, which "resolves significant items of business which arise in between the scheduled Board meetings, and makes decisions, advises or monitors on other matters as agreed by the Board. It meets when appropriate.";
• the Finance, Planning and General Purposes Committee "responsible for the SQA's financial and corporate planning. It meets three times a year";
• the National Qualifications Committee, which "monitors and develops national qualifications …. It meets three times a year";
• the Higher National Qualifications Committee, which "monitors and develops Higher National Qualifications. It meets twice a year"; and
• the Scottish Vocational Qualifications Committee, which "monitors and develops SVQs and other workplace provisions for which SQA is the awarding body. It meets twice a year".

Scottish Executive officials may attend meetings of the Qualifications Committees and the Accreditation Committee as observers when matters of policy are being discussed.

4.3 Responsibilities of Chairman, Members and Chief Executive

The Chairman of the SQA is appointed to chair the Board of Management of the SQA, and is responsible to the Scottish Ministers "for the overall direction and management of the SQA within the policy framework and any annual guidance set by the Secretary of State". ⁹

SQA members "hold office in accordance with the terms of their individual appointments" and are responsible for ensuring that the SQA acts within its statutory powers. They also have a role "in monitoring the implementation of these policies and are required to monitor progress on achievement of targets at least on a six monthly basis". ¹⁰

The Chief Executive's primary responsibilities include, inter alia:

• implementation of policy for qualifications, assessment and quality assurance in the national qualification system;
• overall management and direction of the SQA;
• development of the SQA's strategic and policy options;
• ensuring the SQA is organised, provided with the necessary information systems and other facilities to achieve objectives; and
• "within 18 months of assuming substantive powers, and thereafter periodically, review the level and structure of SQA staff below Chief Executive level in order to ensure that numbers and grading of staff are commensurate with their functions, and the requirements of efficiency, effectiveness and economy". 11

4.4 Planning and Reporting

In terms of planning, the SQA is required to prepare a Corporate Plan each year which sets out the SQA's:

• mission, role and values;
• corporate goals; and
• key objectives for the following three years.

The Corporate Plan should:

• include performance indicators;
• be used for internal monitoring and review; and
• be reviewed annually.

"Performance against these objectives and indicators must be reported, together with any necessary comments, in the Annual Report"¹²

The SQA is also required to have internal audit procedures and practices to standards specified in the Government's Internal Audit Manual and Information Systems Manual. As well as establishing an Audit Committee, the SQA is required to "establish a programme of cyclical reviews of areas of its activities which take full account of audit risks identified in an audit needs assessment exercise." ¹³

In terms of quality management, SCOTVEC was accredited to the internationally recognised ISO 9001 Quality Management System but no formal quality management approach was adopted by the SEB. The SQA intended to implement ISO 9001 across the entire organisation by 2002 and a significant step in achieving this has been the publication of guidance documentation on "Quality Assurance Principles, Elements and Criteria" and the distribution of a number of assessment and quality assurance guides to Centres. The SQA has a stated commitment to achieve ‘Investors in People’ status, with the 1998/99 Annual Report stating a target date for accreditation of September 2000. This date has now been postponed.

The most recent published annual report of the SQA¹⁴ noted Higher Still as having been "our biggest challenge this year" and stated that "effective and efficient communications is a priority and IT is a vital element in this. The development of the Awards Processing System (APS) is central to our communications strategy - it incorporates all the functions needed to administer a national qualification system."

Similarly, in all of the four Corporate Plans that the SQA has been required to produce to date, the significance of Higher Still is referenced. The ‘Environment’ section of the first (1997-2000) Corporate Plan, for example, states that "We are on the brink of further momentous change - the introduction of Higher Still …".

This emphasis is echoed in the Chairman's Introduction to the following Corporate Plan (1998-2001), which states that in respect of the demands which face the SQA the "most significant of those challenges will be the incorporation by the SQA of the revised post-16 qualifications into the Scottish framework".

The two latest Corporate Plans make even more specific reference to Higher Still in key objectives 4, 7 and 8 in the 1999-2002 Plan, and in key objective 1 in the 2000-2003 Plan.

There is, therefore, clear evidence of the recognition by the SQA of the importance and significance of Higher Still to its activities in both documents that it is required to produce - its Annual Report and its Corporate Plan.

However, during the course of this review, it became clear that the SQA did not develop an holistic plan, corporate or operational, to manage the actual implementation of Higher Still effectively within its operations. Indeed, it also became clear that the SQA did not - and does not - have either a business plan or strategy document below the level of its Corporate Plan, or an up-to-date information strategy.

Each of the 21 Units within the SQA produces a plan based on corporate goals and including a set of targets and performance indicators. The key objectives set out in the Corporate Plan are developed as a collation of targets. However, there is no integrated operational plan that draws the Unit plans together to create an agreed plan for effectively translating organisational objectives into an operational action plan to achieve those objectives.

More specifically, there appears not to have been a comprehensive assessment by SQA management of the:

• operational implications for SQA of the implementation of Higher Still during 1999/2000;
• additional workload for SQA arising from this implementation;
• additional resources, including finance and personnel, required to cope with the additional workload;
• changes to operational processes and procedures to cope with the new or changed requirements of Higher Still;
• staff training required to enable staff to handle the new or changed requirements effectively and efficiently;
• feasibility of achieving effective implementation within the timeframe, given the above assessments - had they been conducted; and
• risks associated with implementation within the timeframe.

These issues were raised with current and former members of the Senior Management Team (SMT) during the course of the review. The former Chief Executive, for example, accepted that the implementation of Higher Still, the development of APS and the related organisational restructuring of the SQA should have been considered together and that, had this been done, it would have allowed a better risk assessment. Similarly, both the former Director - Awards Division and the current Director - Development Division agreed during interview that there was no organisation wide plan for addressing the inter related implementation issues.

This is not to say that there was no planning. Indeed, the SMT had undertaken the planning that was required of them in terms of the Corporate Plan, and the Awards Processing System (APS) project¹⁵ is one example of some of the detailed planning that did take place. However, there is no evidence of management having taken an holistic approach to assessing the implication of and the resources required for the changes facing the SQA. Ultimately, in the absence of such holistic and rigorous planning, the risk of operational difficulties during implementation was increased substantially.

4.5 Management, Structure and Personnel

It is clear¹⁶ that both the Chairman and the Chief Executive have active roles in the management of the SQA. However, the boundaries of those roles are less clearly stated. Paragraph 6.1 of the Management Statement and Financial Memorandum states that the Chairman "is responsible … for the overall direction and management of the SQA …", whilst paragraph 6.4 states that, amongst the Chief Executive's primary responsibilities is "… the overall management and direction of the SQA's activities …". The role of the Chairman in Diet 2000 is considered in Section 4.1., together with that of members and SQA committees.

Ron Tuck was appointed Chief Executive of the SQA, following an open competition, in January 1997, three months before the formal creation of the Authority. His immediate background was in education policy development rather than operational management and his actual operational experience at the time could, perhaps, be viewed as modest for a Chief Executive of a new organisation with approximately 550 full-time equivalent posts and an annual budget of some £27 million. In addition, the new Chief Executive faced the significant management task of merging two organisations with their own distinct operational processes, infrastructures and cultures.

On the creation of the SQA in April 1997 the Chief Executive inherited the management and staff of the two predecessor organisations - the SEB and SCOTVEC - including five former Directors. The Chief Executive decided to appoint all former Directors to his Senior Management Team. A new structure was drawn up with 21 Unit Head posts. After internal advertisement, the majority were filled by people who were previously direct subordinates of former SCOTVEC and SEB Directors. Together with the SMT, this constituted a Management Team of 27 members.

It was clear from discussions during this review that many, including members of the SMT and the Management Team, considered this an inappropriate and ineffective management and operational structure.

Subsequently, one Director retired in 1998 and was not replaced and, more importantly, the Director responsible for Operations and Information Technology retired in April 1999. The second retirement was particularly significant in coinciding with a major systems project, and represented a loss of operational experience from the organisation. The post was advertised but no appointment was made. Instead, it was decided to merge directorate responsibilities. These retirements and directorate changes resulted in the organisational structure in figure 1, which was the organisational structure for Diet 2000.

Figure 1 - SQA Organisational Structure (from April 1999)

Under this structure, the Director - Awards Division has operational responsibility for approximately 50% of the staff of the organisation as well as the Operations Unit and the Information Unit, two of the Units which were critical to the success or otherwise of Diet 2000. However, the Director - Awards Division intimated during this review that he had no previous experience of or responsibility for operations, and limited direct management experience of information systems.

It is also clear from the evidence presented during this review that far less success had actually been achieved than the Chief Executive believed, in merging the two predecessor organisations, SEB and SCOTVEC, into one entity with a shared purpose and common culture.

Contrary to the view of the CE, who stated that "in terms of culture it was ‘getting there’, with the exception of Operations Unit" other members of the SMT stated that there "were cultural differences that were important", "the merger of the two organisations didn't happen", there was "lots of in-fighting between SCOTVEC and Dalkeith …" and "no real identity for SQA as an organisation".

These comments were echoed by many other contributions to the review from within SQA, indicating internal tensions within the SQA which, at the very least, would not help in the achievement of effective operational processes during a period of significant change. The continued split-site operation was also cited as a contributory factor in the continued divisions.

Indeed, perhaps illustrative of these and other issues is a paper by the Qualifications Managers in the Development Division in December 1999, their key points being:

• "the Division has not yet gelled into a unified whole";
• "communication within the Division is a key problem and officers feel isolated";
• "(there is) a lack of training to support roles and responsibilities";
• "unrealistic expectations on timescales to provide information";
• "everything seems to be changing all the time";
• "it is difficult to understand what (the) priorities are";
• "management (is) not willing to call on the expertise of staff and seek its views on how to approach particular situations";
• "officers are concerned about the way APS is being implemented - old and new systems should have been run in parallel until the new system proved itself to be effective";
• "never before have officers felt so overwhelmed by all the things that have to be done and that cannot be done";
• "helpline/support facilities (to Centres) are inadequate and teachers are desperate for help because there has been an erosion of support structures. This message must be transmitted to the Scottish Executive";
• "there must be some contingency to cope with staff shortages"; and
• "the general feeling that there is very little openness and honesty in communication between management and officers."

The Chief Executive managed the SQA through a Senior Management Team consisting of himself and his three fellow directors who were responsible for:

• Awards;
• Development; and
• External Relations and Corporate Services.

The SMT met fortnightly. The wider (25-member) Management Team met quarterly and the two subsequently-created Organisational and Qualifications Policy Groups were supposed to meet every two months. The SMT was clearly intended to be the leading and ultimate planning and decision-making group within the SQA.

However, discussions with members of the SMT and others during this review suggested that it did not function entirely effectively. It has already been noted that there was a lack of a business plan or strategy, an up-to-date information systems strategy and an integrated plan to address the challenges of Higher Still and related issues during 1999/2000. One Director stated a "tendency to fudge things" as a characteristic of the SMT, and that the perception of staff was probably that "we were not a team". Another stated that "a lot of problems stem from … the structure … the Chief Executive being too consensual". In addition, the former Chief Executive and other directors also commented on the almost complete lack of useful management information. Such information is key to effective monitoring by senior management and it is surprising that more robust action was not taken to investigate the lack of management information at an earlier stage.

Up to the end of June the meetings of the SMT were recorded in the form of an Action Grid which noted agenda and minute references, agreed actions, actionee(s) and a target date for each reference. The Action Grids were often quite extensive, running up to eight pages, with a large number of very diverse issues and actions incorporated. However, there is no clear evidence from the Grids to demonstrate that the SMT applied sufficient rigour in management of the issues arising out of implementing Diet 2000 since the references to Diet 2000 specifically are few and limited in detail. For example, for the period between 13 January and 30 June 1999 there are eleven Action Grids consisting of approximately 51 pages. Only six of these Action Grids reference Diet 2000, giving rise to approximately 30 actions, 14 of which arise from the Action Grid of 30 June which was dedicated entirely to Diet 2000 issues. Understandably, the later Action Grids give more emphasis to Diet 2000, culminating in the special 30 June Action Grid. Arguably, however, this detailed consideration was too late into the process to rectify the earlier weakness of a lack of a strategic and holistic focus.

So, whilst a great many issues were being discussed at the SMT meetings, most generally relevant to the issues and problems being faced by SQA, there is no sense from the Action Grids of a prioritised, holistic focus on the Diet 2000 problems and how to address them. This is, perhaps, surprising since some stakeholder organisations had been raising issues directly with the SQA since, at least, the first quarter of 2000.

Moreover, one of the Directors of the SMT signalled his growing concern to his fellow SMT members during the period March — May 2000. For example, in an e-mail¹⁷ of 3 March 2000 to his SMT colleagues, he stated:

"I'm reluctant to press the panic button but I am becoming very worried … I think that we need to take full stock of:

• what is going wrong;
• why and whether we have remedial action in hand;
• what more we could do;
• how we best handle and resolve individual complaints …

This brings me to the special SMT. I have to say I think it is remiss of us that we put off consideration of these issues at our last SMT , moreover a 90 minutes or thereabouts SMT to consider these and other issues is hardly sufficient".

Similarly, in response to an e-mail on 19 May enquiring about possible items for the SMT meeting on 30 May, he responded to the Chief Executive in similar terms, asking:

"Are we going to major on Diet 2000? As you know, I'm distinctly worried - I've found recent events / development very disturbing and I have grave doubts re 10 August … If I, as a member of SMT, have these concerns, I hate to think what unit heads are imagining / worrying about and therefore we need to be very frank with them as to:

• how the situation has arisen;
• what steps are in hand to prevent a debacle on 10 August;
• what degree of confidence we can have in those steps;
• what contingency plans we have in place if those steps are not successful;
• how we prevent a repetition next year;
• is anyone to be held accountable for current difficulties … "

Again, on 30 May, more than 12 weeks after the first of the e-mails cited above, he e-mailed at length¹⁸ to his SMT colleagues, stating:

"… I am just very worried and if I am rattled I know key colleagues will be too … Setting aside APS delays and non-submission matters, we seem to have lurched from crisis to crisis in recent weeks with problems coming out of the woodwork and instant reaction to make up for a lack of planning … I refer to the crisis in Operations … the identified backlog and the need to move to shift working, the acquiring of 67 extra staff … the ETP work transfer … SQC testing requirements / live certification on going problems … HN entry problems … early Ops / IT restructuring … significant backlog in processing estimates …

As a member of SMT, I've expressed my serious concerns about the overall situation to Ron (Tuck) - I am not confident we are completely on top and I suspect management team colleagues must have their concerns too …".

4.6 Review and Audit Systems

The use of an ISO 9001 Quality Management System within the former SCOTVEC part of the organisation provides the discipline of critical-processes reviews. However, no systematic review of critical procedures took or takes place across the organisation as a whole.

A small team, known as the Key Process Group, led by the Quality Enhancement Manager, has undertaken some work process mapping in the External Relations and Corporate Services Division and the Development Division. However, no performance measures have been placed on any of the processes and they have not been reviewed to eliminate duplication and non-value-added practices. This, in the main, is due to the complex 21-unit organisational structure within SQA which is perceived as complicating and obscuring the lines of authority within processes. Interface issues are, therefore, difficult to resolve.

There are no formal service level agreements in place between the SQA and its Centres, and a general lack of performance measures and effective process-benchmarking. There is no evidence of changes to processes and documentation being signed off, and changes made were not communicated to all interested parties. Here again, the 21 unit structure does not allow ease of flow of information.

It is apparent from the Internal Audit Services Annual Report 1999/2000 that reviews of the APS and Operations Unit were not part of the programme of internal audit work for 1999/2000 agreed with the Audit Committee. Whilst these areas had originally been identified for inclusion within the 1999/2000 plan, a decision was reached to defer these runs until after the APS system had been fully implemented. Given the significance of the development of APS to the organisation, the deferral of these reviews could, perhaps, be regarded as surprising.

4.7 The ‘Diet 2000 Group’

Evidently, very serious concerns were being expressed at a senior level within SQA from at least early March 2000 in respect of the SQA's ability to cope with Diet 2000. Earlier concerns had led to the establishment in November 1999 of a ‘Diet 2000 Group’ by the SMT, but this Group contained no member of the SMT. The Diet 2000 Group first met on 17 January 2000 and managed to meet weekly by video conferencing.

A report from the Diet 2000 Group¹⁹ stated that the Group was "to deal with the issue - is the SQA capable of successfully delivering the 2000 Diet with the same quality assurance and rigours of past years?"

Whilst the establishment of the Diet 2000 Group was a sensible approach to addressing the issues that the SQA was facing, it is clear from the Group's minutes and report that it:

• had too narrow a remit. For example, the report of the Group stated²⁰ that "… certain business decisions required to effect the delivery of Diet 2000 would fall within the remit of other groups, e.g. the APS Project Board, or require action from staff in other Units …";
• did not have effective communications links throughout the organisation; and
• relied on the SMT being "… aware of strategically focused issues impacting on Units and this allows appropriate action to be taken".

The Diet 2000 Group did achieve some successes in addressing a number of important issues. However, on the question of whether it had met its remit the Group's own report, which was published in June 2000, concluded²¹ as follows:

"The effect of these key tasks being delayed has been to effectively compromise the successful achievement of the Diet. It has also become apparent that complete openness and honesty was not universally adopted".

As well as the timing of the report, the last sentence is, perhaps, also significant in view of the incorrect and misleading information that was subsequently communicated with regard to the issues that the SQA was facing, and its progress in resolving them.

The report also stated²² that "significant areas identified which impacted on the successful delivery of the Diet were:

• lack of planning and resource management;
• lack of entry data being available by 31 October…and the knock-on effects to processing and quality assurance;
• new software not available within projected time scales and the time to progress testing…..;
• Qualifications Managers unaware of what was required of them……;
• lack of clarity in policy decisions;
• incomplete test specification data….;
• poor communication between units of SQA;
• an unwillingness to accept help;
• contracted timetable for the 2000 Diet;
• late production of information required by the invigilators;
• difficulty in recruiting markers;
• late receipt and processing of internally assessed unit results;
• lack of co-ordination; and
• absence of contingency planning.

All of these issues were worked through by the Diet 2000 Group…. Action in all areas has not been undertaken…."

4.8 Individual Performance

Performance of individuals within the organisation has been cited by the SQA as one of the main reasons for the failure of Diet 2000, and particularly the management style and actions of the Head of Operations Unit, Jack Greig. Indeed, the concern over the Head of Operation Unit's performance in the run-up to Diet 2000 were so serious that he was, we understand from the SQA, suspended from the SQA under its Discipline Policy and Procedure on an allegation of gross negligence. It is understood that the gross negligence alleged was that of failing to plan adequately for, and prepare his Unit staff and systems for, Diet 2000. Given the central role of the Operations Unit in the exam Diet, such failings would undoubtedly have adversely affected the operational effectiveness and efficiency of the exam Diet. Disciplinary action was subsequently obviated when the Head of Unit retired on 30 September 2000, under the terms of a previously offered early retirement package.

It is not for this review to consider the allegation of gross negligence against the former Head of Operations, and the performance of the Operations Unit during Diet 2000 is considered in chapter 5 of this report. However, it is clear to us from the evidence presented that the Operations Unit was not adequately prepared for the challenges of Diet 2000 and that many of the contributing factors to the failure of Diet 2000 stemmed from the Operations Unit. There is, for example, evidence to suggest that Operations Unit failed to:

• assess the implications of the changes being introduced in respect of Diet 2000;
• review and redesign operational procedures to cope with the known changes; and
• make staff aware of the new qualifications that they would be dealing with, and train them accordingly in order that the Unit could cope efficiently and effectively with the demands of Diet 2000.

All of these failings had a significant impact on Diet 2000 and it was the Head of Unit's responsibility to have addressed them.

However, from the evidence presented to this review, it is equally clear that some members of the SMT must also share some of the responsibility for the failings within the Operations Unit. This is on the basis that the Chief Executive and the Director - Awards Division were aware for some time of concerns which had been expressed about the Head of Unit's performance.

Evidence to this review from both the former Chief Executive and the former Director - Awards Division indicates that they were both aware of allegations regarding both the appropriateness of the Head of Unit's management style and his operational effectiveness. Notwithstanding this, they consciously agreed to allow him to continue in post for Diet 2000 on the basis that he was, as the former Chief Executive put it, "believed to be technically competent", and that he had a substantial operational knowledge and experience not otherwise available to the SQA, the removal of which would leave the SQA exposed. Ironically, during the course of Diet 2000, a decision was taken to merge Operations Unit with another Unit for Diet 2001 purposes. This decision was intimated to Jack Greig in early 2000 and at the same time he was also told that whilst he would be entitled to apply for the new Head of Unit post so created, he was (in the view of his informant) unlikely to be appointed.

It is clear from evidence presented during this review that at least the former Chief Executive and the former Director - Awards Division of the SQA’s SMT were aware of the allegations in respect of the Head of Operations Unit’s performance and the risks of retaining him in post. However, there is little evidence to suggest that they developed alternative strategies to address this potentially serious weakness in their management team and critical area of operational process. There is, in particular, no evidence of any focussed effort to transfer his knowledge and experience of the systems and processes to other individuals, to minimise organisational risk and exposure. Indeed, a recurrent theme amongst many SQA contributors to this review has been the lack of strategies to transfer knowledge between groups and individuals, and to develop robust, multi-skilled capabilities within staff.

4.9 Chairman, Members and Committees

As previously noted²³, the Chairman is appointed to chair the SQA’s Board of Management and is responsible "for the overall direction and management of the SQA". During interview for this review, the Chairman stated that he "assumed operational aspects were debated and dealt with between the Scottish Executive and SQA Executives" at regular SE/SQA liaison meetings.

Members are appointed individually to the SQA Board of Management and they have inter alia "a key role in monitoring the implementation … of policies and are required to monitor progress on achievements of targets at least on a six monthly basis" ²⁴.

In his submission to the Inquiry into Governance of the Scottish Qualifications Authority, the former Chief Executive stated that "while we were open and honest in our reporting both to our Board and Scottish Executive officials, there turned out to be problems of which I (and my senior management colleagues) were not fully aware.

Every Board meeting since the formulation of SQA included a discussion of APS developments, normally as a discrete item, but sometimes in the context of an overall review of key objectives. At every Board meeting from March 1998, there was a full report on Higher Still developments."

In his contribution to this review, the Chairman echoed these sentiments and there appears to be agreement between the Chairman and the Chief Executive that the information being given by the Chief Executive was being given in good faith and was accepted as such.

In reviewing the Minutes of the Board of Management meetings over the past year (September 1999, December 1999, March 2000 and June 2000) it is relevant to note that:

• while there are earlier references to aspects of Higher Still and APS as separate items, there are no specific references to Diet 2000 until the Minutes of the June 2000 meeting. This is surprising given the evidence we have seen of serious concerns being expressed within the SQA on Diet 2000 from at least early March of 2000;
• the tone of the Minutes on APS, Higher Still and Diet 2000 do not reflect the extent or degree of concern that was clearly being felt within the SMT of the SQA, as exampled in particular by one of the Directors’ series of e-mails in March to May. This is surprising given that of the four SMT members, three were present at the September meeting, all four were present at the December meeting, four were present at the March meeting (following the first of the e-mails on the subject and a specially convened SMT) and two were present at the June meeting;
• the Board Minutes of APS, Higher Still and Diet 2000 items record limited interrogation of the SMT by the Chairman and Board members on these issues. This is, perhaps, also surprising given the collective expertise and experience of the Board members, some of whom had close associations with colleges and schools, and others of whom had roles in the:
  • Association of Directors of Education;
  • Association of Scottish Colleges;
  • Educational Institute of Scotland;
  • Headteachers Association of Scotland;
  • Scottish Council for Independent Schools;
  • Scottish School Board Association; and
  • Scottish Secondary Teachers Association.

   

  Even if it had been the case that individual SMT members were not conveying the full extent of their concerns to the Board, more rigorous questioning on these critical matters by individual Board Members might have been expected. It may, of course, simply be that such questioning occurred but was not fully reflected in the Minutes. However, Board Members have a responsibility to satisfy themselves on the accuracy of the Minutes. Without this, there is neither an ‘audit trail’ nor full accountability.

Members of the Board interviewed during the course of the review stated that appropriate questions were asked and assurances were given by the SMT that the issues identified would be resolved. This is consistent with Ron Tuck’s view of events. In his evidence referenced earlier, Ron Tuck stated that he "found Board discussions (this year) rigorous and probing".

4.10 The Scottish Executive

Although outwith the strict remit of this review, it is worth commenting for completeness on the operational relationship between the SQA, SMT and the Scottish Executive.

The formal relationship between the Minister and the SQA is set out in the Management Statement and Financial Memorandum, which outlines a very limited role in respect of operational management consistent with the policy of delegation of maximum responsibility to an NDPB. Ron Tuck noted this in his submission to the Inquiry into Governance of the Scottish Qualifications Authority, adding that "it should be noted that communications between SQA senior management and officials, from … SEELLD and SEED were regular and open. On my initiative, we had instigated a practice of holding a half-day joint meeting every two months to review key aspects of SQA's work and to enable us to keep abreast of developments in Government policy. There was other contact via meetings or telephone as circumstances demanded".

The two-monthly meetings became known as the Scottish Executive/SQA Liaison Group Meeting and were attended by representatives of SEELLD, SEED and the SQA. Latterly, the meetings were jointly chaired by Alistair Aitken (SEELLD) and Eleanor Emberson (SEED). It was these meetings that the SQA Chairman had in mind in his reference²⁵ to operational matters being dealt with at a meeting between Scottish Executive and SQA executives.

A review of the Notes of Meetings of this Group for the last year (August 1999, November 1999, February 2000 and May 2000) reveals no note of discussion of either the APS development or of difficulties being encountered in Diet 2000. This again is, perhaps, surprising given Ron Tuck's comment on "open" communication, the degree of concern that existed within the SQA SMT and, indeed, the fact that the Scottish Executive was aware of some of the issues from March 2000 at least. However, it is recognised that additional meetings were held to discuss specific issues, such as the meeting on 28 April 2000 on APS, which offers some explanation for the lack of discussion at the May meeting on this issue.

However, Ron Tuck's view on this relationship, as stated in his submission to the above Inquiry, is perhaps telling and consistent with his view on the SQA SMT's relationship with its Board.

"The SQA Board and the Scottish Executive could act only on the basis of information provided by SQA senior management. While we were open and honest in our reporting both to our Board and Scottish Executive officials, there turned out to be problems of which I (and my senior management colleagues) were not fully aware". Specifically with regard to the Scottish Executive, the former Chief Executive stated his opinion that "… it was impossible to identify any practical support they could offer".

4.11 Conclusion

The evidence presented to this review, and that available from other inquiries, suggests a wide range of management and organisational weaknesses within the SQA which contributed to the many problems encountered during the 2000 examination Diet, some of which have existed since it was first constituted and include:

• an SMT which, at times, perhaps lacked the required breadth and depth of management experience to lead the merging of the SEB and SCOTVEC into the new organisational entity - the SQA, and address the challenges faced with Diet 2000. Particularly latterly, it could be argued that the SMT lacked sufficient operations, information systems and finance experience (it should be noted that the organisation does not have a qualified Director of Finance; it is not, however, suggested that the latter weakness contributed to the problems encountered during Diet 2000);
• the failure to design a management and organisational structure based clearly on the functional requirements of the new ‘business’ rather than rely on inherited structures and the practices of the predecessor organisations. The structure was neither effective nor efficient in the new circumstances, and proved insufficiently robust when faced with the challenging logistics of new qualifications and a major IT systems implementation;
• the failure to develop business and operational plans for the organisation that clearly set out the current situation, the business objectives, and the strategy and actions by which the organisation would achieve its objectives.

More specifically, the SMT appears to have failed to grasp the holistic nature of the challenges facing them during 1999 / 2000 and consequently failed to:

• define the implications for the SQA of the various initiatives that they were required to implement during 1999/2000;
• assess the ability of the organisation, as it was, to achieve effective implementation within the available timeframe;
• detail the changed circumstances, assess the increased or changed workload, estimate the additional resources, for example people, equipment, storage, and finance that would be required;
• prioritise actions and deploy resources effectively to achieve effective implementation;
• develop a detailed operational plan for managing the programme of change and the specific projects required;
• develop a detailed risk assessment;
• report accurately on all of the above to the SQA Board and to officials of the Scottish Executive;
• recognise that an effective merger of the two parts of the organisation had not been achieved and that many units and staff were still working to previous guidance, despite new operational requirements;
• deal adequately with significant and recognised personnel difficulties within key units, or to use the Performance Management System to assist in this;
• request and specify appropriate management information requirements in the new computer system; and
• properly manage internal and external expectations by communicating accurate, clear and realistic appraisals of the difficulties which were being experienced.

< Previous | Contents | Next >