Monitoring and Quality Enhancement Sub-group Meeting
27th September 2006
The purpose of the meeting was to discuss the taking forward and co-ordination of the Monitoring and Quality Enhancement paper, in relation to those institutions validated through the Toolkit of Quality Indicators, to carry out assessments for the Disabled Students Allowance.
Moving forward
The validation panel currently meet twice a year in May and November to consider new applications.
The Monitoring and Quality Enhancement (M&QE) paper outlines the validation process, in the future those institutions already validated, will be asked to submit yearly confirmation to SAAS, that they still intend to carry out their own needs assessments and that resources/skill base remains the same. Institutions then prepare a self evaluation in the third year, which will be submitted to the validation panel, in order to be revalidated for the next 3 years.
At the moment we are still receiving new applications, and also further information for applications received in previous rounds, which have not yet been validated, in the future we will be dealing not only with new applications, but also those for re-validation.
It has been suggested that the validation panel should increase the number of times it meets, from twice yearly, to 3/4 times a year, in order to deal with the new applications, and re validations, and also to pull together all the further information received from institutions still on going in the original validation process, in order to make this a less ad-hoc process.
Example timetable:
- End April for new submissions, 3 yearly resubmissions and annual confirmations
- Mid June for consideration of additional evidence,
- End October for new submissions and 3 yearly resubmission
- Mid December for consideration of additional evidence
The issue of support for those institutions currently preparing for the validation panel was raised, as several members of the validation panel had been approached for assistance, but due to conflict in interests, had to signpost the institutions to alternative sources of help.
It was suggested that contacts from other willing institutions could be included on the invitation letter, as a point of contact for assistance.
As previously mentioned, institutions will have to indicate to SAAS each year that they wish to continue as an accredited centre for carrying out assessments. The group discussed making this a more comprehensive document, in order to encourage quality enhancement and monitoring throughout the year, as well as notifying of any change in personnel.
* HELS to take forward to establish dates and invitations.
Data Monitoring
SAAS provided a draft spreadsheet for the group to discuss (annex A). The spreadsheet will be for institutions to complete in order for SAAS to monitor the impact of the toolkit. The group were concerned at the column entitled 'Date of initial contact', as this could be well in advance of any official acceptance on to the course, and could highlight a false delay before the assessment taking place. The group suggested inserting a column entitled 'interim arrangements', which would take into account legitimate delays, such as taking advantage of loan equipment supplied by the university.
* SAAS to take this forward.
A student questionnaire will also be sent to each student, the M&QE paper advises that this will be sent out with each students DSA letter. The group discussed this and came to the conclusion that a more accurate reflection of the impact of DSA on the student's learning experience would gathered if the questionnaire was sent out 3 months after the student had received their DSA. This is to allow for the student to have had time to make use of their DSA in their learning environment, and more accurately reflect their opinion, and in the future for reapplying students, include questionnaire along with their letter detailing their DSA entitlement .
The group discussed the format of the questionnaire and all agreed that any opinions received would be very subjective, and therefore to focus less on the mechanics of the assessment, (the who, what, when and where), and focus more on the impact the process had on the student.
The group came up with a few questions to be included in the questionnaire, and will ask SKILL and NUS for their input.
Q1, What impact did the assessment have on your experience as a student.?
Q2, Did the assessment process help you to recognise your needs better in a higher education environment.?
Q3, If you received the equipment allowance as part of your DSA, were you offered any training on your recommended hardware/ software, and did you accept it?
*Further questions to be included, along with general questions on SAAS' processes. All group to take forward
Networking and Sharing of Good Practice
The group discussed the possibility of holding workshops, like ones previously held, as the institutions had found them useful. The idea is to have the workshops in an institutional led format, by inviting the institutions to submit examples of good practice, what works, what doesn't. The group discussed whether these would be beneficial for those institutions who have not yet been validated, possibility of splitting the day into sessions for just those that had been validated, and then opening up to others who are intending to apply, to organise 'critical friend' sessions, as an aid to what the intending applicants should be focusing on, without talking away from the point of the day, which would be to share good practice for those already validated.
It was suggested that these workshops should be held at the end of January.
* propose monitoring and evaluation workshops for end of January, to next meeting of the DSSSG. HELS to co-ordinate all group to contribute
The M&QE paper suggest the use of visiting panels, comprising of members of the SE, SAAS, and other members of the DSSG.
The group discussed the possibility of inviting a representative of HMIE on to the panel, but thought that this would be inappropriate , as the visit is to analyse the systems in place for assessments, and not to monitor the institution as a whole.
Due to the complexities involved in arranging visiting panels, suitable dates etc, other ways of providing support and advice were discussed.
The group concluded that the reviewing of an anonymised assessment, would be the best way to evaluate the institutions, it would enable the panel to identify the strengths and weaknesses within the institutions process, and would help to quality improve the reporting. The panel would be looking to break down the case and focus on the quality of evidence, how it was gathered, and evaluated. What provisions the institutions made, what recommendations were made to the student, both IT, and non IT based. These reports could feed into the annual confirmation and be considered by the validation panel sessions.
The next validation panel is to be scheduled for late November early December, with the first set of revalidation taking place in April/May.
HELS to find date and confirm
Action Points
- HELS to take forward the approach of yearly revalidation, and to issue invitations for revalidation
- SAAS to take forward spreadsheet for institutional monitoring
- All group to take forward questions to be included on the student questionnaire issued by SAAS
- HELS to co-ordinate, all group to contribute, the proposal of monitoring and evaluation workshops, at the next meeting of DSSG
- HELS to find date and confirm for round 4 of validation