«Using EFQM in higher education: Ten years of experience with programme auditing at Hanzehogeschool Groningen Evert Bisschop Boele, Hiltje Burgler, ...»
Improvement begins with constructive co-operation between people. A consensus meeting is an occasion par excellence for staff members of a particular programme to consult one another about the quality of education and its assurance. The interviews which take place as part of the audit process provide a picture of communication, development, engagement, and policy support. In the post-audit feedback, the audit team discusses points of departure for improvement, and the auditors can express their appreciation of the stronger aspects of the programme. This may then result in improvement strategies Beiträge zur Hochschulforschung, Heft 1, 30. Jahrgang, 2008 Evert Bisschop Boele, Hiltje Burgler, Henriette Kuiper regarding the processes in the programme, leading to better results and better quality and therefore to better accreditation results. It is this balance between focus on processes in the internal audit system and focus on content and outcomes in the external accreditation system that gives added value to both.
Looking back at ten years of experience with EFQM auditing in higher education, we can identify five main success factors.
7.1 Facilitation and management support Auditing all programmes every three years costs time and money. For the work of an audit team of three members, hundred hours have to be budgeted (40 for the chair, 30 for each of the other members). On top of that comes the time for preparing and holding the consensus meeting and the hours members of the programme team spend on preparing the audit (putting together the documentation, inviting students, staff and external stakeholders and the like). Furthermore, time is spent on training new auditors, on the yearly meeting of auditors, and on the co-ordination and quality assurance of the auditing programme as a whole.
This massive exercise cannot work on a purely voluntary basis. Contributing to quality assurance processes and products are therefore considered to be an integral part of normal daily work for staff, additional work such as e. g. the membership of an audit team is remunerated separately. Also, institutional management must see the benefits of such an extensive auditing programme, which is the case at Hanzehogeschool Groningen. The board stresses its importance, facilitates the process and takes the outcomes very seriously.
7.2 A bottom-up approach
For many people, internal quality assurance is not the most attractive subject. To keep quality assurance continuously on the agenda, it helps enormously if the exercise is not exclusively owned by a small group but is the concern of many people throughout the institution. The way the audit system has been set up takes this into account. Two important features stand out. First, auditors are ordinary staff members – in daily life they teach, lead (part of) a programme or work in a support role. As there are currently eighty 104 Beiträge zur Hochschulforschung, Heft 1, 30. Jahrgang, 2008 Hanzehogeschool Groningen auditors, and the number is still growing (there is even more interest in becoming an auditor than can possibly be accommodated), expertise is widely spread throughout the institution. Second, a wide group of people participates in the consensus meetings and the following audits. Overall, at least half of teachers at Hanzehogeschool Groningen have at some point been in touch with the EFQM model, thereby learning how to think in quality assurance terms. Recently, half a day of EFQM training was included in the introductory programme for new teachers.
7.3 Improvement orientation
While the EFQM audit is an internal process from the perspective of the institution as a whole, it is perceived as “external” by the programme teams. People not connected to the programme come to scrutinize it, which is potentially threatening. At the same time, internal quality assurance only works if ownership is put in the hands of the professionals who finally have to deliver quality in their daily work: teachers and support staff.
They will only assume ownership if they clearly see the benefits, which mainly lie in positive advice on how to (further) improve matters. At Hanzehogeschool Groningen, we therefore try to avoid an atmosphere of judging and to focus on feedback and dialogue instead. It is for this reason that in principle, the follow-up of the auditing process is put in the hand of the programme, that is the programme management. After the audit procedure, they possess a consensus report (expressing their own view) and an audit report (expressing an external view). Relating the two, formulating possible improvements, and putting them in the hands of staff, is done by the programme management itself and fed back into the planning and control cycle of the university. Mostly, audit reports will be an item of discussion between dean and executive board; if necessary they can for example agree upon including one or more improvement actions in their management agreement.
7.4 Using a higher education-specific model
A university of applied sciences is a community of professionals who are highly focused on their principal task: teaching and research in their specific area. Translating the general EFQM/INK model into a specific model for higher education has proven to be one of the main success factors. This holds in particular for the way the fifth – and core – aspect of EFQM, “management of processes”, has been specified to refer to the primary processes of a university of applied sciences. Because the descriptions refer directly to teaching (and, still to a lesser extent, to research) and not to services in the labour
market as in the EFQM model, it becomes easier for people to relate to the rather abstract EFQM model.
7.5 Linking internal quality assurance to the steering system An explicit link between the result-oriented steering mechanisms the institution uses and the audit system is a key success factor of the quality assurance system. The audit system itself can only measure whether or not the institution’s processes will lead to completion of the PDCA cycle and therefore to better results, to an external focus etc.
In reality, however, better results, external focus etc. are not a result of the audit system but of what people actually do in their daily work. The result-oriented steering philosophy provides the focus on results, the auditing system measures whether the processes work and where they can be improved. Auditing is a meta-process and only becomes useful if the processes under the meta-level receive attention. It is for that reason that e. g. most of the management contract is dedicated to actual results, and not for example on scores on separate criteria or aspects of the EFQM model.
8 Points for further discussion On the basis of its 10-year experience, Hanzehogeschool Groningen will further develop its system of internal quality assurance based on EFQM. To complete this article, we would like to highlight six of the many open questions ahead.
8.1 The relationship between internal and external quality assurance The balance between internal and external quality assurance is delicate. To many, one of the main purposes of the internal EFQM quality audits is to prepare for accreditation.
We are, however, hesitant to stress this factor. Firstly, EFQM may be used for that end, but as other universities of applied science show, other instruments may be used just as well, for example a system of “test site visits” more directly linked to the external accreditation framework. Secondly, and more important, we believe that internal quality assurance is actually largely independent from external quality assurance. External quality assurance should in essence only confirm that the internal quality processes are sufficiently developed. Nevertheless, we acknowledge that there is a relationship. When we adapted the EFQM/INK model to higher education, we took into account the Dutch accreditation framework, for example by inserting an addendum at the end of the book describing in detail the relation between EFQM and the accreditation framework. Also,
we adapted the internal quality assurance cycle to the external quality assurance cycle of six years, not to our internal planning and control cycle of four years.
Nevertheless, internal quality assurance should not be reduced to an instrument for passing external accreditation. Recently, Hanzehogeschool Groningen discussed whether we should slightly adapt our internal quality assurance cycle, for instance by maintaining the audit preceding accreditation as it is, but by downscaling the intermediate audit to a lower level. We took the stance that the audits stand in their own right and therefore should stay as they are – if any change was to be made, we would rather lighten the preparatory audit and maintain a full-fledged intermediate audit to keep quality awareness between two accreditation processes on a high level.
8.2 The independence of auditing from management
An issue of continuous discussion is the relation between self improvement by the programme teams and management interference. Our auditing system primarily serves to heighten quality awareness among academic and administrative staff, challenging programme teams to look closely at themselves and confront their own ideas with the views of external auditors. However, as quality is of course a matter of primary concern for the higher levels of management (deans, executive board) as well, it is tempting to use the audit reports not only as an improvement catalyst but also as a control instrument. In Hanzehogeschool Groningen, this issue has never become really critical, but continuous awareness from both sides on potential goal confusion is required to ensure this.
8.3 The internal quality assurance of auxiliary processes
The audit system of Hanzehogeschool Groningen focuses on the primary process of teaching (and, lesser so but in a growing amount, on research), but takes into account all relevant support processes needed for education, such as student support, administration, media facilities etc. As a consequence, the auxiliary processes are often discussed in the context of education and less often in their own right. This may not give a complete picture of the quality of auxiliary processes and may be disappointing especially for those working in auxiliary support departments. Although the business of for example the financial department eventually is of course to support the education Hanzehogeschool Groningen offers, financial staff will often, and at least partly rightfully, consider financial processes, and not teaching processes, as their primary process. The quality assurance team of Hanzehogeschool Groningen has therefore started to experiBeiträge zur Hochschulforschung, Heft 1, 30. Jahrgang, 2008 Evert Bisschop Boele, Hiltje Burgler, Henriette Kuiper ment with carrying out extra audits in support departments. Recently, the central offices for financial affairs and educational affairs were audited. An open challenge is to find the right audit model, as the EFQM/INK model for higher education is too education centred for this purpose. In the latest version of the model, an alternative description of criterion 5 has been included in an appendix. It seems however that it needs further refinement.
8.4 The limiting effect of the five-stage model
Hanzehogeschool Groningen states (Hanzehogeschool Groningen 2007, p. 5) that generally it is expected from programmes to function at stage 3, i.e. to be engaged in systematic and continuous improvement on the basis of evaluation (see section 4). One effect of this goal is that some programmes focus on attaining stage 3 and do not aim higher.
This may at some point become counterproductive. The audit system should not be the quality benchmark for programmes. Rather, programmes should continuously try to improve their quality – and end up at stage 3 or higher at some point.
8.5 The pitfalls of systematic repetition