The Scottish Government

« Previous | Contents | Next »

Listen

Part 2 Stakeholder Consultation

Stakeholders

Individuals from key stakeholder groups were nominated by SEHD for interview. 33 interviews have been completed and 5 more are due to take place. A full list of interviewees is available drawn from the following stakeholder groups ²:

• PG Deans
• Chairs of Boards
• CMO and CDO
• National Panellists
• Academy of Royal Colleges
• HR
• Medical Directors
• NES
• SEHD
• Chief Executives
• Current SpRs
• Newly appointed consultants
• Clinical academics

Methodology

Semi-structured interviews were conducted lasting approximately 60-90 minutes. The majority were conducted face-to-face with a smaller number of telephone interviews. Interview question areas as detailed in the project proposal included:

• Review of the current system identifying strengths and weaknesses.
• Potential or existing barriers to fair recruitment identified by stakeholders.
• Identification of the selection criteria for consultant recruitment.
• Views and expectations of what makes a valid and reliable selection process. How might evaluation of such a process take place in the future?
• Future issues for selection in the context of MMC, global recruitment and diversity eg short-listing and interview methods, scoring mechanisms.
• Patient safety implications.

Emerging themes from stakeholder interviews

A framework of issues consistently emerging from the interviews is presented here. Information can be classified into six key themes. The themes are:

Theme 1. Recruitment and selection process
Theme 2. Professional standards and quality assurance in recruitment
Theme 3. Selection criteria
Theme 4. Selection methods
Theme 5. Fairness and diversity
Theme 6. Future perspectives

Theme 1. Recruitment & selection process

This theme relates to the efficiency and effectiveness of the recruitment process.

1.1 Cost and resourcing of the process.

The resourcing required to run the current process was perceived by some to be excessive and inefficient, particularly for smaller employers. In reviewing current practice, many believed that the size of the interview panel and cost of engaging National Panellists (both in panellists attending in their own region but also in releasing consultants to go to other regions) was expensive, where often the costs were perceived to outweigh the value-added to the process. ³ For example, one stakeholder commented "Most people involved in the process value the external verification of clinical competence, but the numbers [of interviewers] and time involved is costly and doesn't seem to add a great deal…it could be improved". There were some more negative views, illustrated by the comment , "The whole process needs to shift into the 21 st century". However, the majority of respondents recognised that consultants on the whole have been appointed effectively for years, but that it could now benefit from a review to optimise efficiency and effectiveness. Some alternatives to reduce costs and resourcing were suggested including reducing the numbers of individuals on the interview panel (e.g. National Panellists, University representatives). For example, one interviewee suggested " I'm not sure we need to have two National Panellists, we could have a checklist to follow…so we can demonstrate and be sure that the person is ok and on the specialist register". Other suggestions included introducing paper-based elements of the process, rather than face-to-face.

1.2 Flexibility of the Regulations.

The regulations were perceived by some to lack flexibility to allow for different local needs. For example, in some areas, consultants may be appointed with a limited, if any, teaching role. In these situations, one University representative on the panel was seen as sufficient. It was reported that this lack of flexibility did not allow HR to make changes to practice in line with their status as an equal opportunity employers. "Our hands are tied in HR to make any meaningful changes in line with the rest of our recruitment practices. We need changes to the regulations in order to provide us with an impetus for change, otherwise it will remain the same." A consistent theme was the current regulations needed a review and updating, illustrated by the comment "historically it has been professionally driven but now the regulations need a significant review…the Regulations are out of date". Many stakeholders welcomed the opportunity to contribute to the review; " there's so much that relies on custom and practice so it's important to have this review".

1.3 Logistics & administration of the process.

The logistics of convening a panel with national panellists was viewed by most employers and HR operational staff as difficult. HR representatives reported that while they could engage their own clinicians and non-executives to sit on interview panels, obtaining interview dates from National Panellists was more challenging. This was attributed to a various factors. First, the HR personnel responsible for this task were often junior staff, and it was often perceived that they had insufficient 'credibility' with the panellists. HR reported that requests for dates were sometimes ignored. There were clear examples of how departments tried to remedy this. For example, clinical staff in the department that was recruiting would identify the national panellists they wanted to sit on the panel, and using informal contacts, they would agree dates directly with the panellist. HR would then be advised who could be approached and who was available. This was viewed as a practical solution to ensuring attendance of national panellists for interview dates.

Second, there was a perception that there were a small number of national panellists who volunteered to be on the list for career reasons and because of this were not always committed to attending. For many administering the process, some believed that this explained why there is a long list of panellists, but difficulties in obtaining attendance at interviews. However, from the national panellist perspective, there were concerns that they were given insufficient notice to cancel clinics, so that they could attend the interview. Six weeks notice is the minimum required to ensure clinical work is not disrupted and all panellists interviewed gave examples of being asked to attend interviews at very short notice. Some panellists reported that they do have regular days or sessions that they do not have clinics and could attend interviews on these days within a shorter time frame if this was more widely known.

1.4 Time taken between a consultant retiring or resigning to the appointment of a new consultant.

Even through consultants are required to give three months notice to leave a post, often new appointments are not in place by the time the consultant leaves. Whilst there were various reasons for this, the time taken to set up panels was consistently seen as a significant problem. The process was reported as slow in a number of places, for example in identifying available panellists, obtaining feedback on the job description and in obtaining shortlisting results. Time delays reportedly had an impact in several areas, including:

i loss of candidates to other posts; in some areas if the interview panel is not set up quickly, applicants drop-out as they attend other interviews and are offered posts elsewhere.
ii fewer handover opportunities; if the existing consultant has already left handovers which were viewed as important to the new consultant, the other members of the team and the patients were fewer.
iii cost of locums; some areas have to find locums to fill the post until a new appointment is made, which is not seen as ideal.
iv vacant posts; where locums are not put in, there is disruption to other members of the team and delivery of service while the gap is carried.

1.5 Candidate perspectives.

Delays in setting up panels reportedly cause anxiety and uncertainty in candidates. HR operational staff reported that they regularly deal with calls from applicants concerned about interview dates and whether they should apply for another post coming up. Both current applicants and newly appointed consultants were able to clearly identify areas for improvement in the current process. For example, there was a perceived 'lack of information' for many throughout the appointments process. Many were sympathetic to personnel in HR departments and believed that they were often overstretched or unable to deal with queries appropriately. How candidates are treated through the process, including on visits to the departments, could influence their decision to take a position. As one newly appointed consultant commented, "I think how you are dealt with throughout each stage can really reflect how they might treat you in the future. I certainly thought twice about a position after how I was treated on a visit".

For those individuals who are currently applying for posts, there was some lack of clarity regarding what is required of them in the appointments process. For example, they would like more information on what the interview involves so that they can fully prepare themselves. Many valued the opportunity for visits the department, however some were unsure how this information was used in the process. Some basic information and standardised guidance that clarifies some of these aspects would be welcomed by all.

1.6 Impact of "internal markets" within Scotland for recruiting consultants.

Neighbouring Boards may have vacancies at the same time and are competing for the same candidates. Because of the current pool of applicants and 'internal market' conditions, this meant that it was vital to be first to interview and appoint. One national panellist gave an example of this: " we had set up the panel and done it all by the book…we got interview dates set up and booked our national panellist. However, another region suddenly set up a panel and managed to clear it in a week and we lost out…I'm not sure how but they were cleared to be a national panellist for the day and we lost some really good candidates. I get a feeling that there are rules that sometimes get broken… so we need to make sure it's all on a level playing field".

Ensuring national panellists are independent of a recruiting location is viewed as particularly important in this context.

1.7 Role of HR in recruitment processes.

The role of HR in the process was primarily one of support and administration, ensuring paperwork was available, rooms are booked and candidates and interviewers were looked after. Those undertaking this role were typically more junior. This is illustrated by the comment "i n terms of HR, they are most often treated as the 'hand-maidens of the process' and sometimes lack the level of professional influence required…why aren't Board level HR directors involved". Whilst most HR staff were capable to undertake these roles, it was reported that it was a more challenging job for them to be advisors to the process. For example, in being able to intervene when best practice was not being followed or when advice was required. Given the level and status of panel members, it was reported that HR representatives would "need to be very strong to say something to panel members about appropriateness of questions being asked……or to advise the candidates that they did not need to answer a particular question". In most cases, this role was typically taken by the Chair. As such, it requires the Chair to be knowledgeable and motivated to apply best practice recruitment practice.

1.8 Composition and role of interview panel members.

While most were able to describe the rules governing panel composition (as per the Regulations), the specific role of panel members was less clear, other than " representing" their stakeholder group. Some suggested a need to see a clearer argument for the 'value added' by each member of the panel, with potentially more specific guidance. For example, the role and effectiveness of non-execs was challenged by some, "People choose non-execs that will go along with the decisions. Some come at the drop of a hat".

1.9 Time allocated to the selection of individuals to senior level appointments.

In practice, forty minutes for a panel interview was a common structure. A recurrent theme from stakeholders was that for many, although this had been a reasonably effective approach, there was scope for improvement. Many commented that forty minutes did not seem sufficient for the job level of consultant. "It seems a short amount of time to examine whether they will be good enough for one of the most responsible, highly paid posts"; "with 40mins, 8 people, and 2 questions each, we barely have time to ask the questions… how can this tell us much about a candidate?" Stakeholders consistently commented on how this time was spent, with a focus on the clinical verification of a CV. "Often we spend 30mins going through a CV where the doctor is just agreeing that they did spend time in x, y and z unit. Surely some of this can be assessed by a paper-based exercise?". Others questioned the value of what is uncovered during the interview, "we have between 8 and 10 people spending the best part of half a day to interview 2 applicants and all that happens is we go through their CV. There must be a better way to use the time to assess them, given that we are about to offer them a job for next 25 years". Importantly, all stakeholders believed that the current process does identify appropriate people. However, the vast majority of those interviewed said that the process could be significantly improved.

1.10 Opportunity to assess non-clinical aspects.

It was consistently reported that there was not enough opportunity to assess non-clinical aspects that localities perceive as important (e.g. team working, leadership, motivation, interpersonal behaviour, future plans etc). Most stakeholders felt that this was an area for significant improvement. Some suggested the introduction of a two-tiered process, where clinical competence was one aspect, and the other was the behavioural/attitudinal aspects and fit with the team. This is reported in further detail under selection criteria section (see Theme 3 Selection Criteria).

1.11 Consistency of decision-making processes on panels.

In making appointments, the decision making process used by interview panels varied between localities, and even sometimes within the same Board. Where more than one candidate was appointable, some panels decided with a show of hands (ie vote who should be appointed), some had a broader discussion before the Chair summarised, some involved the national panellists, others did not. Decision making processes appeared to have emerged over the years, often depending on the expertise and preferences of the Chair and local practices. There was some interest in understanding the mechanisms that other localities used and whether there was evidence for the effectiveness or efficiency of one approach over another.

1.12 Panel interviews as a networking opportunity for interviewers.

A positive aspect reported of the current system was the additional benefits to participating in the panel that are often not formally recognised. For example, participating in a panel was perceived by some as providing an informal networking opportunity for specialists, some employers and non-executives. It was also reported that there were opportunities to learn from external panellists, to share and cross fertilise ideas, which was viewed as positive.

1.13 Documented evidence of the effectiveness of the current process.

Many stakeholders view the current appointments process as appropriate. Specifically, this is how interviews for consultant level appointments have traditionally been conducted, so that interviewers and applicants are used to the system. "A lot of this is about custom and practice……we've always done it this way and the regulation has meant that there has been little appetite or flexibility for change." Similarly, stakeholders have also noted that although there is no documented evidence to suggest it isn't working, there are clearly significant areas for improvement. For those who support the current system, they do not see any compelling evidence that any changes to the system would make for a better outcome. There is no documented evidence to say that the wrong people have been appointed, and even though there were anecdotal examples of consequences of poor appointments and consultants moving on, these were not documented in a quantifiable way to be able to make a full evaluation of effectiveness. Some stakeholders suggested that there should be some primary research conducted in this area so that the results are appropriately documented. All stakeholders were able to clearly identify areas for improvement, even those who supported the current process and structure. In this respect, stakeholders believed that it was not a case for change for changes sake, but there was clearly a case here to make improvements to the current processes. This view is encapsulated in the statement, " the current selection process obviously works but it really does need modernising and streamlining. As it stands the panels are geared towards assessing clinical competence but there are many other aspects that need attention…we can't ignore this aspect as medicine is changing rapidly and so are the structures that govern it".

Key Finding:
The consistent message from all stakeholders is that the current process for consultant recruitment is not working efficiently. There is also evidence to suggest that the effectiveness could be improved, given the personnel involved. This is not to say there are grounds to suggest the wrong people are being appointed, rather that there are lost opportunities in current practice. The process by which appointments are made is viewed as bureaucratic and in need of change. Improvements are required to enhance the experience for applicants, interviewers, HR and employers.

Theme 2. Professional standards & quality assurance

This theme relates to the mechanisms used to judge clinical competence, the impact upon patient safety, and the role of the external advisor in the recruitment process.

2.1 Role and remit of national panellists.

For most stakeholders, although there was broad agreement, the precise remit and boundaries of national panellists was unclear. Most stakeholders clearly described the role as having two key elements. First, in recommending to the Board whether candidates are clinically appointable/unappointable for that post, and second the role of an external reviewer on the panel, to ensure fair practice and professional standards. However, there were variations in the description of the activities involved in these two areas. For example, the role of panellists prior to, and on the day of, the interview varied in the extent of involvement in reviewing job descriptions, defining interviewing questions and inclusion in the decision making process. There were divergent views on where the boundary and remit should extend. For example, job descriptions currently go to panellists to comment on content in advance of the interview. There was some disagreement here regarding the value of this process. Employers were clear that the work plan is the remit of the employers. National panellists and Colleges review job descriptions as an important part of maintaining standards, appropriateness of the job responsibilities and standardisation across Scotland. To illustrate this point one stakeholder said, " there had to be a safeguard to stop managers over-ruling clinical safety issues. A few years ago, we were trying to appoint 600 new consultants and national panellists had a major role in ensuring that not just anyone was appointed, even through there was some pressure to do so".

Some believed that given the cost of attendance at the interview, more should be asked of these external reviewers. " We have these highly experienced people in the room and we don't allow them to give their views on the strengths and weaknesses of the candidates. It's a waste of time and expertise them sitting there". Once the panellists have determined whether candidates are appointable, often panellists opinion is sought " off the record". There is no formal channel for this and some suggested that there should be a mechanism for their views to be incorporated into the decision-making process, in a more transparent way. The alternative view was that panellists are not always in the best position to offer a view beyond that of appointable or unappointable, and they shouldn't be providing views of which of the appointable candidates they believe should get the job. Some already felt panellists sometimes go beyond their remit: "one panellist wrote to the Executive complaining that we had appointed the wrong candidate. We had 4 candidates and all were deemed appointable by the national panellists, so it was a local decision which of these would get the job. After ranking all of them, we went for candidate 2 but the panellist argued that candidate 1 was "better" and should have been offered the job. We reviewed it but were happy with our decision. They were all appointable and so it was our decision on who would best compliment the team." Other examples illustrate the boundaries of the panellist role "the right person for the post may not always be the highest achieving candidate in terms of publications and other honours. National panellists are not always best placed to make this judgement on how they would fit in the department."

2.2 External review role provided by National Panellists

This role is viewed by most stakeholders as important for QA. For example, it is perceived as helpful to ensure against 'local bias' in appointments. For example, one stakeholder commented " You need to have a referee at the appointments panels - decisions can't be left down to a local cabal who could fix the appointment towards the local guy. We have a similar referee when we conduct the RITA process and I look to the Colleges to provide that person. There must be examples of weak National Panellists but on the whole the process of having people to scrutinise the process works".

Equally, taking a share of the responsibility for decisions to appoint by panellists was seen as helpful in the support against appeals. "there is a sense of confidence if two or more external panellists are on the appointment committee". This reassurance was based on concerns regarding a future increase in challenges and appeals. Panellists were seen as providing defensible independent opinions. " One example was where a locum was already doing the job for a while and applied for a post when it came up ….this was the same job but a permanent consultant position. They didn't get it and appealed...The Board went back to the national panellists for an independent verdict. The panellists were able to verify the reasons for not appointing and dismiss the claim that it was based on local bias [i.e. interviewers knew the candidate]." While local clinicians could make similar judgements on appointability, the external review by national panellists was seen as a robust endorsement of process and appointment. While most welcomed and valued a need for an external scrutiny role, some felt that this role could be carried out by senior doctors not necessarily on the national panellist list. The need for external scrutiny was supported by all stakeholders. This is illustrated by the comments " I don't understand why we need a heavily standardised process, but I do understand the need to ensure QA and patient safety" and "External input to the process is necessary to make sure the candidate is appropriately qualified… this is done well on the whole and is a strength in the current process. We need this assessment of clinical competence as a safeguard; in certain specialities it's increasingly complex".

2.3 Process for assessing clinical competence (paper-based v's face-to-face).

Some stakeholders felt that the CCT defined clinical competence and that this was a 'given' if candidates got through shortlisting. As such, some believed that further assessment of clinical competence at interview was duplicating effort. "It's very rare for concerns to be raised in the interview. So what is their [ national panellists] contribution? Often national panellists are struggling to find things to say in the interview….particularly when the CV is straightforward in the experience gained. It seems a bit of a waste of time sometimes". It was recognised that, in theory, for UK trained medics a paper-based assessment of the suitability of their experience and training for the particular post may be applicable and appropriate. Judgements can be made on the basis of the UK training experience. However, for training acquired outside of the UK, a paper-based exercise was not seen as appropriate or sufficient to make this judgement. The national panellists and other interviewers would not know what it meant if applicants spent 3 months in a particular place/ hospital/ rotation outside of the UK and how this would relate to the job they were applying for. For most stakeholders, probing this via the interview was seen as the only way to establish the extent of the training experience and in which areas.

National panellist input was viewed by some as important in giving advice on technical aspects particularly in sub-specialist appointments. This was most useful where the local knowledge of board members (e.g. medical director, chairs) was not sufficient to understand the needs of the sub-specialty and where they wanted an opinion beyond the views of their local consultants. Almost all stakeholders questioned suggested that the process could be streamlined by having one national panellist, illustrated by the comments " as a Panellist, my personal opinion is that National Panellists perform an important role although I can't fathom out the need for two at every consultant appointment. I suspect that there may be the odd occasion where the input of two national panellists may be required but suspect these instances are few and far between" and "one panellist is sufficient - two is overkill". However, the College perspective is that they would like to reserve the right to have two panellists in case of dispute or appeal.

2.4 Patient safety and role of CCT.

Patient safety is dependent on fitness to practice and having competent consultants in place. Most of the focus of how this is ensured has been on clinical competence and the role of the CCT and interviewers (including National Panellists/ local clinicians) in assessment. From the stakeholder interviews, it is clear that the CCT provides a different type of assessment of clinical competence than what is assessed at the interview panel (which is often experience and 'fit'). Some stakeholders suggested that there needed to be clarity about the CCT and judgements about a candidate's clinical competence. For example, a national panellists said, "The concept is that possession of a CCST (dental specialties) or CCT (medicine) automatically makes the individual eligible for every consultant post. However, in some dental specialties appointment to a consultant post requires an additional training period post CCST. Also, possession of a CCST/ CCT, whilst confirming clinical competence, may not signify ability to work in particular geographic areas and perhaps the classic example here would be working in remote/rural areas which throw up their own individual challenges". A further illustration of the issues is provided in the box below.

2.5 Selection and training process for national panellists.

Appointment of panellists was perceived as unclear and " opaque". How panellists are selected, and the level of qualifications required for the role were common questions. " The appointments process for NPs is a bit opaque - there is a nomination via the Royal Colleges, but how the University nominees come forward is really opaque. Improving the transparency of the process would help. The management of the process needs improvement … it's unwieldy. "

National panellists themselves consistently discussed the need for a more structured approach to induction and training for the role. While many may well be experienced assessors and interviewers elsewhere, a clearer, shared understanding of their role and expectations would be welcomed. This is illustrated by a comment by a national panellist who said "National Panellists really must have better training. At the moment it's pretty much non-existent". Another commented, "there needs to be more equality and diversity knowledge, and more on training on employment legislation".

2.6 Training of interviewers.

While many of those involved in the interview process were reportedly experienced interviewers, many commented that there was no provision in place to review interviewing skills or review practice in line with changes to employment legislation or QA issues. As one stakeholder put it, "the trouble with recruitment is that everyone thinks they can do it…the reality is that some can do it better than others". Training was perceived to be growing in relevance for the future, especially regarding employment law. Training for Chairs was seen as particularly important given their roles as guardians of the process. Further clarity on the role of lay panel members would be welcomed. Many believed that NES could have a key role in providing appropriate training and development in this area to optimise the effectiveness of panels.

Key Finding:
Verification of clinical competence of candidates' remains paramount, to ensure patient safety. There were mixed views on whether this needed to be carried out by National Panellists as it currently stands and whether the process could be paper-based, or separated from the interview. While the CCT is an important indicator of minimum clinical competence, alone it does not confirm whether a doctor has the experience to safely practice within a particular job context. All stakeholders welcomed a clarification of the role and boundaries of the National Panellists remit, in addition to more structured training for the role. Training of interviewers was also viewed as necessary for the future.

Theme 3. Selection Criteria

The identification of appropriate selection criteria relevant to the consultant role was frequently raised across all stakeholders. What criteria can and should be assessed? The identification and assessment of clinical competence has been the focus of the selection criteria historically. However, the future assessment of non-clinical/'softer' skills was consistently raised.

3.1 Identification of selection criteria, link to interview questions and scoring.

Many reported that the identification of the criteria to be assessed through the process, particularly the non-clinical aspects, was not explicit and that the content and focus of questions was variable and could be improved. Our review showed that there had been little if any formal job analysis to identify the selection criteria. Research consistently shows that job analysis is the cornerstone to best practice selection practices. This analysis allows accurate design of the Person Specification and the related selection criteria, guiding design of interview questions. This issue was raised by many stakeholders illustrated by the following comment, "I'm not sure we often know what we are trying to find out…those applying aren't clear what is expected of them and the questions reflect this lack of focus". The actual methods that were reported to be used to score individuals were variable. Our review suggests that more robust scoring mechanisms should be developed to improve the effectiveness, standardisation and transparency of this process. As a result, there could be significant opportunities for giving feedback to candidates.

3.2 Focus on 'attitude' and communication skills as important selection criteria.

This was consistently raised as an issue by almost all stakeholders interviewed. Specifically comments here related to identifying attitude as a critically important selection criterion - an issue that differentiates successful versus potentially unsuccessful consultants. However, further clarity is needed regarding what is meant by the word 'attitude' (i.e. is it personality, motivation etc,) and how it can be assessed fairly, accurately and appropriately. " We have focused on technical and clinical competence in the past but there is insufficient focus on 'who' we are hiring…especially as modern medicine is more about team based efforts. User/patient expectations are now so much higher…sharing information, acknowledging a team effort, understanding obligations to patient and employees".

Most stakeholders consistently said that communication skills were crucial and if a doctor fails, then often this is an issue, illustrated by the comment "They simply must have the ability to communicate with people at a variety of levels. Obviously the vast majority of consultants do an outstanding job but when there have been problems is usually to do with a break down in relationships…the people side of things". Another stakeholder said " In my experience just about all the complaints I've had to deal with over the years relate to communication. Although it's there in the appointments process there really hasn't been sufficient emphasis on the behavioural and communication aspects when making appointments".

3.3 Focus on assessment of 'organisational fit' as important.

Most stakeholders commented on a key requirement for a candidate is to 'fit' into the local context (both technically and in terms of the people aspects). Specifically, local clinicians and employers perceived the need for those appointed to 'fit' the needs of existing teams. Identification of this organisational 'fit' was mentioned by many stakeholders. Some suggested the use of a presentation to team members, so that the existing local team could have input into the new appointments process. One stakeholder suggested " I think the academic appointments process has some very positive elements that could be considered. A portion of time is set aside where the candidate visits the department and they give a presentation or seminar, and the department can quiz them on their approach and how they would supervise students…just like asking how they might deal with patients…and do they acknowledge the consequences of not doing it right…views from the department are then represented at the interview panel. I'd really like to see some pilots of some more innovative approaches".

3.4 Appropriate identification of otherknowledge, skills and abilities.

Whilst the assessment of clinical competence clearly remains paramount, the majority of stakeholders interviewed would like to see more accurate identification and assessment of other selection criteria such as non-clinical attributes. This is less clearly defined in the job description and there is little agreement on how best to assess this. Common areas identified as important to assess included team working (in particular working in multi-disciplinary teams), clinical leadership, motivation and intention for development of special interests (e.g. clinical and non clinical areas), communication (written and verbal). A common belief expressed was that although it was critical that clinical competence is assured through this process, it is these non-clinical attributes that "make or break successful appointments"; "Consultants we have had in the past that were not successful once appointed didn't fail because of technical or clinical competence …… it was more often than not related to other aspects [of their behaviour or skills] that we hadn't pursued at the interview…whether it's something we could have got to the bottom of in some interview process is another question".

Many stakeholders suggested that the selection criteria should be more accurately documented and then investigated thoroughly at interview. This is illustrated by the comment, "t he selection criteria need looking at…the consultant nowadays must be an innovator, a researcher, a teacher, a leader and a manager of people. Consultants for the future must not only embrace change but must be in a position to lead change. The nature of medicine is changing rapidly and so are public expectations. We can't afford to hold back on progress."

3.5 Integration of selection criteria into job description.

Some stakeholders believed that there will be a requirement to identify criteria and job requirements more accurately to enable more robust assessment of these areas in the future. Specifically, this was needed where the consultant role requires them to be more involved in activities outside the clinical group. This might include administration, management activities such as performance management, waiting times, patient experiences, and broader aspects of the job role e.g. engaging with hospital wide delivery. Currently many of these parts of the job role are not clearly featured in the job description and therefore it was not something that panels focused on or explored in any depth in the interview. Many would like to see more effective ways of exploring this and ensuring applicants are aware of this prior to applying for a post. "When applicants phone me and ask me what is it we are looking for, I say that their technical competence is a given….so long as their experience matches the needs of the department, we are looking for qualities in addition to this that will allow them perform well and get on with the team".

Some stakeholders also stated that they would like to see the job descriptions reflect local needs more accurately. For example, one stakeholder said, " we've just appointed a Chief Exec, we had an external assessor, a chairperson and importantly a job description that although it was set out in a national format, it really did reflect local needs and requirements too". Another stakeholder suggested, "there should be national and local elements to the job descriptions".

3.6 Quality and usefulness of person specification.

Many stakeholders said that the quality of the Person Specifications could be improved. In particular, the content could contain more specifics relating to non-clinical selection criteria and perhaps reflecting more locally driven requirements. On the whole, stakeholders said there could be more information to help guide choice of questions in interviews and how criteria could be weighted. In particular, one stakeholder said, "The behavioural aspects haven't been scoped…the stuff beyond the CCT needs to be documented". A common view is illustrated by, "the person specs for jobs don't have anything on what kind of doctor we want to recruit, it really only states the technical and research aspects"⁴.

Key finding:
Clinical competence is clearly specified on job descriptions. However, all stakeholders believed that improvements could be made to the identification and assessment of other important selection criteria. Common areas include team working, clinical leadership, motivation and intention for development of special interests, communication and organisational fit. Identifying these criteria so they can be effectively assessed was viewed as an area for improvement. There are improvements that can be made to the design and implementation of the panel interview process. However, any improvements must account for the utility and feasibility so as not to reduce the efficiency of the process. Potential improvements have also been identified for the Person Specification.

Theme 4. Selection Methods

Here, views on current and future selection methods depended to some extent on stakeholder experiences and knowledge of the contribution and application of selection methods at this level.

4.1 Current strengths and weaknesses of short-listing.

Currently, it was felt that the number of applicants to posts do not create the need to do 'fine grained' ranking at shortlisting (as the selection ratio is low) and so there is little concern over short listing methods. In general, the majority of eligible applicants are invited to interview unless there are identified concerns. For many stakeholders, attracting a pool of appropriately qualified candidates is the main challenge. As such, the short listing criteria is often CCT acquisition, work experience etc, although practices appeared to vary by area and specialty. In summary, this process is aimed more at selecting out, rather than ranking individuals. However, many stakeholders recognised that if numbers applying for posts increase and there are more eligible applicants than capacity to interview, short listing methods would need to be improved for the future.

4.2 Current strengths and weaknesses of interviews.

All stakeholders on the interview panel have an opportunity to ask the candidate questions in relation to their relevant stakeholder group interests. Therefore the interview it is perceived by some as inclusive, and sharing responsibility on decisions in the interview. This is illustrated by the comment, "the approach we have is that all the panel members are in this together, including the national panellists." For many, they believe there is no documented evidence that this approach to interviewing is not working; on the whole, most feel they select the right people from those in the applicant pool. There was no impetus to remove the use of the interview, particularly without any evidence of alternative methods in this context, but there could be ways in which they are improved as a selection method.

4.3 Potential areas for improving selection methods.

Most stakeholders believed that the selection process using an interview worked sufficiently. However, many also believe that there was significant scope for improvement. This is also supported by our audit of current processes. Improvements can be made to the content (questions), selection criteria, timing, structure, and training of interviewers (in advanced interviewing skills).

From an employer perspective, one person said, "I strongly believe 40 minutes interview is insufficient for the jobs we are appointing to…and often the real selection work goes on outside the interview….either way, we are committing huge resource, millions of pounds…so we need to ensure this process is tight". Although clearly most stakeholders believed the interview process had worked well, many stakeholders believed that there was good reason to review the process in light of likely future changes. This is illustrated by the following comment, "the rate of change in the next 30 years will be substantially higher than the rate of change we have experienced in the last 30 years - to think we can rely in future on a 40 minute interview as it stands is absurd". Some stakeholders suggested the use of personality measures to assess the behavioural and attitudinal aspects, and suggested that these processed could be piloted to see whether they could add useful information.

Many stakeholders believed that that the selection methods could be more closely focused to the clinical and on non-clinical aspects. For example, one suggestion was, " I like the notion of a two-tier system. First there is the clinical competence assessment via CCT/National Panellists etc. Second there is the organisational fit, the behaviours and attitude; these need to be judged by the local team as well as the interview panel".

4.4 Using selection methods to identify development needs of candidates appointed.

Some stakeholders stated that often the interview identifies development needs of candidates that are appointed. However, they believed the extent to which these were followed up was variable, or whether they were pursued in any systematic way at all. Of those stakeholders that raised this issue, most agreed that there is a significant opportunity to identify and address development needs for candidates. Many believed that the requirement to do this in the future may be, and should be, more prominent.

4.5 Information from selection methods to aid feedback to candidates, especially if unsuccessful.

Some stakeholders felt that there was insufficient attention paid to providing feedback to candidates, especially those who were unsuccessful in the selection process. Although some recognised that this would take time to achieve, at this level of appointment feedback should be made available. Some stakeholders suggested a feedback sheet could be completed at the end of the panel interview process to help streamline this process.

4.6 Future selection methods.

Although all stakeholders believed the interview was an important part of the process, many suggested additional methods that might improve the process. Various potential selection methods were suggested by stakeholders, but there were mixed views on how these would work, often depending on stakeholder experience and understanding of the methods. Most stakeholders were unsure of how any alternative methods would be received or the evidence for the validity and reliability within this consultant context ⁵. All stakeholders believed that new methods could not be introduced without appropriate piloting. Many were keen to find out what other methods could add (e.g. via piloting and validation studies) but would not like to see significant changes without appropriate evidence. Methods suggested by various stakeholders included:

• Multiple (smaller) split interview panels focusing on different aspects of performance (e.g. clinical and attitudinal) exploring different aspects of the candidates' achievements, motivations, capability (e.g. clinical, behavioural etc). etc. Use of scenarios specific to the context of the job.
• Presentations to team members and/or the panel. Input from the local team/department was viewed as very important as the newly appointed candidate will be working directly with the 'team' (not the interview panel).
• Written exercises (completed in advance of the interview or during the day): examples of recent achievements, indicating various clinical and non-clinical examples. Submitted prior to the interview and signed off by current employer/referee.
• Personality assessment there was mixed responses to the use of these at this level. Some would not like to see them included "I prefer to speak to people and find out from themselves what they are like. I've taken psychometric tests and I'm not sure what is gained from it". Others could see some benefit where they were completed before to inform the interview. "We have so little to go on in a short space of time. Surely testing can help us focus on key areas". Many stakeholders said they would be interested in piloting the use of such measures to investigate whether they could help in the assessment of attitude, in particular.

4.7 Evaluative standards for judging the quality of selection processes.

As the literature review details, there are various best practice standards by which any selection process should be judged. These include validity, reliability, objective, standardised, fair, cost-effective, feasible and defensible, amongst several others. Many stakeholders recognised that improvements could be made to the current process but that cost-effectiveness was also an issue. On the other hand, many believed that the process could be made more efficient. If new methods were introduced alongside the interview process, this might increase the resources used. On the other hand, by splitting an interview panel there could be gains in efficiencies. Also if a presentation is introduced for example, there would need to be a clear view on how this would be weighted and scores integrated into the selection decision making process. (i.e. one stakeholder suggested that there is no point having a presentation if the information is not used in decision making).

4.8 Role of references/referees.

Our review shows there is significant inconsistency in the way references are used in the current appointments process. For those that were paper-based and open-ended some panels used them in discussions at the beginning of the interview, others used them once the interview was completed. In addition, many reported the extensive used of informal references requested via networks - depending on 'who knows who' - which resulted in low standardisation in the use of references. There was little formal guidance in how reference reports should be best gathered and used.

4.9 Role of visits to departments.

Many stakeholders stated that this process needed to be made more transparent. Specifically, there should be transparency and guidance regarding;

i whether or not applicants can visit the department in advance;
ii what information should be made available;
iii what they should expect to cover in the visit;
iv who they will see (the team members, etc).

In particular, the opportunities available for candidates to visit units varied considerably. Some candidates were unclear as to whether they were 'allowed' to visit and how this might be viewed. By contrast, some stakeholders suggested that some candidates " expect to see the Chief Executive".

Key finding:
The shortlisting methodology is not currently seen as a major issue. Usually, with a relatively small applicant pool, as many eligible applicants as possible tend to be invited to the interview. However, this could change in the future, putting more pressure on shortlisting methodologies. In addition, as described in Theme 1 'The selection process', the logistics of the shortlisting process can incur significant time delays. All stakeholders supported the use of interviews as a method for recruitment at this level. However, many would like to see improvements in the interview (eg number of interviewers, content, criteria, timing, structure, training of interviewers) to help generate more valuable information on candidates, while reducing bureaucracy and workload. Stakeholders would like to see better ways of using information from an improved interview to feed into development activities for appointed candidates, and feedback to unsuccessful ones. There was significant interest in the use of other selection methods (e.g. presentations, work-based exercises, personality measures), but most stakeholders were unsure of the evidence and acceptability for their use in this context. Many would like to see them piloted to generate evidence of the validity, reliability and utility.

Theme 5. Fairness and diversity

This theme relates to fairness and diversity throughout the recruitment process, from advertising through to interviewing and induction.

5.1 Retaining talent and fair recruitment.

It was recognised by many stakeholders that informal identification of future consultants plays a role in the current recruitment practice. This was described by one stakeholder as a "double-edged sword" in balancing the needs of a fair recruitment process, but also in retaining identified talent. As one stakeholder explained, "….we often know who is coming through the system, what they are like and what post they would be well suited to….. When they are good you want to keep them in the system, not lose them elsewhere. Clearly we have to balance this with ensuring a fair and open system". There was some concern regarding how this manifested itself in practice. For example, stakeholders described how it was not unknown for job descriptions to be created to match identified individuals skills and advertising strategies put in place to meet legislation requirements but enhance the likelihood of appointing the identified candidate. There was no formal documented evidence of this but many recognised that these approaches to recruitment may happen at this senior level in medicine. Most stakeholders would like to see a fair and transparent way of retaining this " talent pipeline" (identifying those coming through with potential and aptitude). However, it was recognised that this must be balanced with open and transparent opportunities for all.

5.2 Attracting a pool of applicants.

A key issue for some stakeholders was the lack of applicants applying for posts, and so attraction is a major issue. This clearly has an impact on the diversity of the pool from which selection is taking place, as the pool is relatively small. There are benefits in ensuring advertising of posts is widespread, in that it can attract applicants that may be better for the post. Many stakeholders reported that legislation around advertising and attraction has a minimal impact on increasing this pool. Most argued that benefits to employers and applicants of wider advertising needs considering. The whole issue of advertising was a recurrent theme. One stakeholder suggested " in some cases posts were only advertised when a panel knew who they wanted to appoint…how else you explain the level of vacant posts? I think we need some clearer policy in this". Others were keen to ensure that the job descriptions were clear in identifying the core parts of the consultant job and to be clear about the opportunities for work-life balance. As one stakeholder commented, " Advertising should ensure a diverse pool of applicants, wherever possible".

5.3 Candidate perceptions of fairness.

Recent appointees and future candidates were consistent in wanting more information about posts and the actual appointments process. 'It [ appointment process] seems to be clouded in mystery…it's really not very transparent. I feel I've had to find out by stealth what it is all about and how to beat it. It would help if we were given more information about what to expect". Another potential candidate said, "It would help if we knew when jobs were coming up, some central database or something. I get to hear about the ones local to me through local meetings or colleagues but there could be ones further afield that I'd be interested in but never get to hear about". Some commented that in the future, with the introduction of MMC, the end of training will be at a similar time so new consultant posts could co-incide with this.

5.4 Compliance with employment legislation.

Many recognised that the existing appointment regulations were " out of date" as employment legislation had moved on. Of those stakeholders who were aware, most assumed they would be updated at some point. In addition, some referred to the Board involvement in the appointment process and suggested that the current processes (in terms of Board involvement) was not compliant with the current regulations.

5.5 Standardisation of selection methods to ensure fairness.

There were mixed views about whether increased standardisation of methods/process increases fairness and effectiveness. For example, in the interview, some felt fairness was ensured by asking the same question of all candidates without deviation. Others however felt this was unrealistic and not helpful when recruiting at this level when informed discussion around key areas was required. One stakeholder said "Interviews should have some structure but shouldn't be too onerous, they should be semi-structured". This is in line with best practice. Some believed that further training of interviewers and Chairs would help to understand how fairness in interviews is enhanced. For example, in using job descriptions and selection criteria to identify areas for exploration by interview panel (e.g. communication skills) rather than asking exactly the same question. Appropriately probing of candidates responses was also seen as important at this level and concern that this should not be too formulaic or standardised. Some stakeholders also raised the issue of lay involvement in the interview process. Similarly, many suggested that specific guidance would be helpful in this respect.

5.6 Equal opportunity and diversity training.

Some stakeholders with specific expertise in employment law expressed concerns about the current level of awareness of panels to equal opportunities and employment legislation. Of those with expertise they suggested the need for specific training of those involved in the appointment process. Whilst panel members may have been trained, it was often not clear to what level and how long ago training had occurred. As one stakeholder commented, "As a Chair of a panel, I'm unsure what training the panel has undergone. Some clearly have, others may have but have either forgotten the core elements or need a refresher". Most stakeholders believed that appropriate training in this area would be welcomed and is necessary. Some suggested there were more subtle issues that could be addressed (such as suggesting there may be an indirect gender bias in senior level recruitment). Some stakeholders suggested this issue should be evaluated in particular (perhaps through commissioning primary research) given the increasing numbers of women into the medical workforce.

5.7 Auditing of selection methodology.

Many stakeholders said there should be specific mechanisms in place to audit selection methodologies used for appointments across Scotland. Given that the nature and structure of speciality training has changed significantly, creating processes to plan for the consequential impact on senior level appointments is advisable. Such audit mechanisms could also include processes to monitor information on fairness and diversity.

Key finding:
Attracting an eligible pool of candidates from which to select from was a key issue. Advertising widely clearly helps. However, this strategy alone may be insufficient to ensure an eligible and diverse applicant pool. Tensions between stakeholders wanting to retain local talent with equal ops/fairness/diversity were apparent. Legislation currently requires that there is fairness in the process. However, organisations and departments have developed strategies to meet their own recruitment needs which may not always reflect best practice. In particular, legislation in this area is a necessary but not a sufficient condition to ensure fairness and equal opportunities in the process. Training in equal opportunities, fairness, and interviewing can help stakeholders to better understand the benefits of attracting a wider pool to individuals, departments and organisations.

Theme 6. Future perspectives

All stakeholders were asked to comment on the future of consultant recruitment.

6.1. Applicant perspectives on futures.

There was some concern by applicants that those coming via the new training system based on MMC principles will have their CCT but may have less experience and 'time served' in the specialty. Trainees and other stakeholders felt that through no fault of their own, they may be at a disadvantage to 'experienced' doctors already in the system. Some suggested that there could be posts created that appoint doctors straight from specialty training and then offered structured development to acquire relevant experience. Most of this concern was related to the uncertainty generated by the transitional period and some suggested that those doctors who were less experienced may find it difficult to be appointed until all the "experienced people were out of the system".

6.2. Process improvement (logistics, utility).

A consistent message from all stakeholders was that any change must bring with it less bureaucracy in the process. Anything new introduced must not " increase the burden". As one interviewee put it, "There must be a method of providing a better experience for all parties… interviewers, HR, panellist and candidates...we must not add to the burden". It was clear that stakeholders do not want to see the introduction of a centralised electronic application system, like the MTAS process. Many commented that any improvements to consultant recruitment processes at this time had to managed carefully given the context and experience of selection by junior doctors. One stakeholder said, " following recent changes, MTAS, and the impact on the profession, it would be a very brave decision to significantly overturn any of the current processes in consultant recruitment". In this respect, any proposed improvements need appropriate consultation with stakeholders.

6.3. Potential impact of 'oversupply' in the future.

Some stakeholders believed that there will be a 'bulge' of qualified specialists exiting training at the same time in the future. One stakeholder commented, " I'm not convinced there will be a sub-consultant grade as the skill level required should be no different. However, I think we can expect a 'big bulge', and the financial implications need to be thought through". Many other stakeholders said that there are likely to be financial and practical implications in the future that need to be addressed now. Specifically, an oversupply of trained specialists would lead to an increase in applications to consultant posts. While this may fit well with current aims for attraction to these posts within Scotland, increased numbers of applications would bring a need for changes to selection practices and processes, in order to deal effectively with the process.

6.4. Opportunities to further improve the appointments system.

Most stakeholders welcomed this review and whilst there were concerns regarding change in the current climate, all interviewees believed that there were opportunities to improve the system in various areas, from advertising through to interviewing methods and provision of development plans to candidates.

6.5. Feedback to unsuccessful candidates.

Many stakeholders said they would like to see improvements that lead to enhancing feedback to candidates (especially those that are unsuccessful in the process). Currently, interviewers felt that they often struggled to give meaningful feedback to unsuccessful candidates, while others were concerned about legal implications of how the content of feedback is used. Improvements in identifying the selection criteria and interview methodology could aid and enhance this process.

6.6. Review of guidance to candidates, interviewers, HR.

Guidance is available on the appointments process, but many felt that this could be more practical, accessible and available for all stakeholders. Provisions of 'top tips' and guidance on best practice for multiple stakeholders was suggested by many stakeholders.

6.7. Evaluation of validity, reliability and utility of selection methods.

While many stakeholders were interested in using other selection methods, perhaps alongside the interview, there was a lack of specific knowledge regarding the evidence for their usefulness (validity, reliability and utility) in this context. Given there is limited evidence in the literature regarding use in this context, there are clearly opportunities to pilot methods in localities or specialties that wish to explore this. In particular, some stakeholders suggested that primary research should be conducted to fully establish areas for ongoing improvements.

6.8. Impact of new training pathway ( MMC) on the next generation of consultants & continuous development of consultants in post

There were concerns expressed by various stakeholders that the new training pathway means that new appointees (in the future) will necessarily have had less experience in their specialty and in hospitals. However, the MMC pathway suggests that trainees will not progress unless they have acquired the necessary competences. Some believe there is also a need to focus on the development of consultants in post, both in clinical areas but also non-clinical areas. For example, working effectively in teams, attitudes and management development. One stakeholder that illustrates this issue said, " I'm a senior medical figure and there is a need for 'management development'; it's about attitude and how to behave in organisations. It's an issue that needs addressing".

6.9. Future identification of selection criteria.

There were consistent messages from stakeholders regarding the selection criteria that are important for current consultant appointments. However, some would like to see further work carried out to specifically identify the criteria for the future consultant role. Given that many stakeholders suggested that the role is changing rapidly, many suggested that it would be apposite to address this specific issue.

Key Finding:
It was clear across all stakeholder groups that improvement is required to the appointments process. Stakeholders differed to the extent of improvements required, and had some concerns about what was possible in the context of changes to junior doctors selection, but most felt that clearly opportunities exist to improve the selection process and pilot selection methods. Given the output from specialty training in the near future, there is a need to plan for this now to ensure a robust system is in place in time.

« Previous | Contents | Next »