« Previous | Contents | Next »
Listen
Group Discussions
Following the presentations, delegates broke into four groups for discussion. Each group considered five questions about POE, over two sessions, before feeding back to a plenary session. The following summary captures the main points made.
Question 1: What type of POE, if any, is being carried out at present or is planned in your Council?
Most authorities have done/are doing some form of evaluation
Most local authorities have done/are doing some form of evaluation on completed school building projects, although very few are conducting what they would term formal POE exercises. However, several of these authorities indicated that they would plan to take POE on board after the workshop.
Most authorities which are currently involved in their second PPP programmes had received feedback from their first project to inform their second. However, it appeared that POE was not being included as a contractual requirement in Public Private Partnership ( PPP) projects.
One authority was in the process of conducting a POE exercise in a non- PPP primary school. The first phase, which involved an experts' walkthrough and debrief, was complete. The next stage will be to interview parents and children. Thereafter, there are plans to evaluate a joint campus school and then use these experiences of POE to inform their PPP Invitation to Negotiate ( ITN).
Evaluation exercises are generally being conducted 'in-house'
Most authorities are commissioning, designing and conducting the evaluation exercises 'in-house' using council Education and Property staff. However, a few authorities have engaged external architects to help assess technical outcomes, and one has considered using a university research department.
Most evaluation exercises involve consultation with stakeholders
Most evaluation approaches have involved consultation with stakeholders - in many cases involving end users (staff, pupils and parents). Some have restricted their consultations to council officials (Education and Property) and headteachers.
Question 2: What do you see as the barriers/challenges to making POE a regular activity?
Lack of resources
The lack of time was considered a major factor in restricting POE activity. It is necessary to ensure that staff are allocated dedicated time to participate, and this can be difficult to arrange/fund during school hours.
While all authorities agreed that there are benefits arising as a result of carrying out POE, the lack and uncertainty of funding available to undertake the studies and implement their recommendations was also seen as an issue restricting POE activity to date.
Lack of experience
It was noted that that few 'in-house' local authority staff had POE experience, and few external organisations either had, or were actively marketing, Post Project or Post Occupancy Evaluation expertise and services.
For some local authorities it was unclear who was responsible for carrying out POE, as it was not identified in any job description. One authority noted that the work was given to staff who were already facing existing workload challenges relating to school building issues - such as preparing School Estate Management Plans.
Fear of criticism
Many authorities (particularly elected members, but also senior officials) are concerned that evaluation exercises may lead to public criticism for any (even relatively minor) design failures in new buildings. There is also a risk of creating unrealistic expectations about what action will be taken to follow up recommendations.
Fear of blame or contractual and insurance concerns may also limit the willingness of design teams and contractors to participate in formal evaluation exercises. Private sector design professionals may find that participating in POEs may affect their professional indemnity insurance premiums. Legal and contractual issues may also inhibit a contractor's ability to participate in evaluation exercises, particularly in PPP projects.
Question 3: What do you see as the benefits of undertaking POE?
Allows 'fitness for purpose' to be assessed
A principal benefit of POE is the opportunity to assess 'fitness for purpose' of the building. It can demonstrate how well the building is being used and can identify adjustments to both the building and to management practices which can optimise its use. The results can also be used to rank 'suitability' in School Estate Management Plans/Core Facts.
Provides a good platform for stakeholders to air their views
Irrespective of the outcomes of POE exercises, the very act of conducting them sends a positive message that authorities are seen as acting responsibly, being accountable and carrying out genuine consultation. In addition, the participatory approach can have educational value for pupils and staff. It can help building-users to understand project constraints and thereby lead to increased user satisfaction and co-operation.
Will help improve design quality
POE has a role in improving design quality, as it provides an opportunity to feedback good and bad design features for incorporation/omission into future briefing and design. It provides a way of researching innovative design or policy areas, and the outcomes can be used to develop good practice guidelines. Furthermore, it can allow detailed feedback on parts of buildings which may not otherwise receive a lot of design time on new projects ( e.g. toilet and cloakroom layouts, lockers, furniture, etc.).
Assists in streamlining future procurement processes
POE of a completed project helps to promote continuity within a programme, as it can be seen as both the end of consultation on one project and the starting point of the next. It can assist in achieving a more streamlined briefing and design process, and be seen as a valuable tool for future projects as less 'first principles' research and consultation is required. POE might also help set more realistic budgets for future projects which would lead to potential savings.
Question 4: What needs to happen for POE to be carried out successfully?
Commitment, clarity and leadership
There needs to be a greater clarity about the purpose of POE, and this requires greater leadership by local authorities, the Scottish Executive and building professionals. The process of POE must be seen as being genuine and not a token gesture. One authority noted that, from the start of any building project, everyone involved should be made aware of the intention to carry out POE. They should also be made aware of the procedure and the time commitments required, and be willing to participate. In terms of policy drivers, it was suggested that POE is relevant in terms of continuous improvement of service delivery, best value, and sustainability.
Clients and participants must be prepared to accept constructive criticism
There needs to be a commitment and attitude of openness and 'bravery' on the part of the client and participants. One authority noted that clients and participants have to prepare themselves to receive constructive criticism and admit mistakes. Participants should avoid using the exercise to defend reasons for previous decisions, and a blame culture should be discouraged. There is a need to dispel the myth that 'we (the council/building professionals) know best'. Equally, there must be council 'buy-in' and commitment to dealing with the results which must be seen as reasonable and fair. The evaluation itself must be conducted within a culture of honesty, and both positive and negative opinions should be welcomed.
Facilitator needs to be neutral
The need to have a neutral facilitator was considered important. Although this may cost more than using 'in-house' personnel, one authority noted that this was often necessary in order to prevent participants from following personal agendas.
Expectations of stakeholders need to be managed
One authority emphasised the need for expectations and aspirations raised by POE to be managed. It was argued that it was of prime importance to educate stakeholders on the reality of project costs, value for money, etc. If not, they may complain about things that could only be achieved in an ideal scenario.
Process should be comprehensive
The evaluation process needs to be comprehensive in terms of the range of stakeholders to be consulted. A wide range of views should be sought, and the opinions of pupils should include those from individuals who may be regarded as disruptive.
There was general support for the view that POE should be seen in the context of a full stakeholder consultation. It was also felt that any programme of evaluation should be estate-wide, and conducted regularly in order to engender widespread support. A number of authorities also felt that POE should be employed beyond the school estate, and that all buildings across the public sector should be included (referring to local authorities' role in Community Planning under the Local Government in Scotland 2003 Act). However, a caveat was mentioned regarding the raising of expectations and the limited budgets available to deal with the results.
There was also concern about the term POE, as it implied a 'once-only' exercise when a more ongoing system of evaluation is perhaps more relevant. For this reason, one delegate suggested the renaming of Post Occupancy Evaluation to Periodic Occupancy Evaluation.
Question 5: What needs to happen for POE to become commonplace?
Initial POEs need to be seen in a positive light
It was agreed that initial POEs must be seen in a positive light. For this reason, one authority had chosen to evaluate a school that was keen to participate in POE. It is also important for evaluation outcomes to be balanced, and not simply critical. One authority had tried to encourage participants in POE to accentuate the positive aspects of the building. For the report, they asked users to identify three positive aspects and three areas for improvement. POE in this authority has been well-received and a valuable attempt to build the evidence base for school estate investment.
POE must be made an essential requirement of school estate activity
A number of authorities felt that POE needs to be made mandatory for it to become commonplace. POE should become an integrated part of council procedure, or alternatively there should be a 'trigger point' around which to centre POE, such as the lead up to budget development. It was also suggested that it should be a standard item in the client's brief. Alternatively, it could be an integrated part of the School Estate Management Plan process to reflect on progress over the past year. However, it was also noted that if POE was made mandatory, it would risk turning into simply a
'box-ticking' exercise.
Benefits need to be demonstrated through sharing of good practice
POE outcomes must be made accessible to engage stakeholders, and be presented in such a way that they can influence decisions by other council departments such as Finance/Corporate Services, and by elected members.
There was also discussion around sharing the outcomes of POEs more widely, which could be promoted/facilitated by the Scottish Executive through benchmarking. In addition, there should be agreement on a 'fitness for purpose' standard in consultation with teachers and other stakeholders. One authority suggested that the Scottish Executive should set out a national policy which local authorities could adopt. Another authority suggested that the Scottish Executive issues 10 key points stating best practice for POE.
Funding has to be available
It was noted that POE activity was limited by current budget levels and that funding systems have to be flexible enough to allow for spending on POE. Funding needs to be included in project budgets, since these may be trimmed at the end of projects and POE may be dropped. One authority noted the importance of ring-fencing funding for an external party to undertake POE. It was suggested that such funding should be available from the Scottish Executive to include POE in the School Estate Management Plan.
It was noted that funding is also required to take action on issues raised in the POE exercise. If no consideration is given to the recommendations, stakeholders will lose confidence in the process.
For this reason it was suggested that there may also be a need to set aside contingency funding in project budgets to deal with POE issues.
POE should be included in the training of building professionals
It was proposed that POE should be included in the training of building professionals to enable them to add value and to encourage standard practice. At present, there appears to be limited expertise and resources to conduct evaluations on a comprehensive basis.
« Previous | Contents | Next »