Evaluations of quality in education
NOKUT carries out evaluations of quality in education as part of its external quality assurance. This task is defined in the Act relating to universities and university colleges § 2-1 (4), which states that “NOKUT must conduct evaluations of significance for assessing the quality of higher education.” NOKUT conducts comparative evaluations. Each evaluation looks at a set of study programmes, usually within a particular subject area. However, they can also include thematic evaluations focusing on particular aspects of quality in study programmes across subject areas.
Providing knowledge and facilitating quality enhancement
NOKUT’s evaluations are designed to provide the HE institutions and other stakeholders with information about the quality of education, and to facilitate quality enhancement. The evaluation design is comparative and combines formative and summative elements.
Importantly, these evaluations do not examine compliance with legal requirements and, as such, final reports include recommendations for quality enhancement rather than formal decisions on compliance.
To facilitate quality enhancements, NOKUT also strives to develop evaluation activities and processes that include rich opportunities for dialogue and reflection among evaluation participants and other key stakeholders. This can for example include:
- seminars and workshops exploring topics pertinent to the evaluation
- the publication of interim reports that share new knowledge at different stages of the evaluation
- the inclusion of peer observers in site visits to facilitate peer learning and to aid transparency and legitimacy in the evaluation processes
Fit for purpose – stakeholder involvement and tailored evaluation criteria
To ensure that evaluations are fit for purpose, each evaluation is designed in dialogue with key stakeholders and has its own set of evaluation questions and criteria tailored to the education context that is being examined. These are published on the NOKUT website and communicated to evaluation participants before the evaluation is initiated.
Evaluations are carried out by groups of external peer-review experts. These groups include a student representative. The experts review self-assessments submitted by the participating institutions, as well as other quantitative and qualitative data (including data from NOKUT’s annual student survey). The experts conduct a site visit, and they produce a final evaluation report.
Feedback and complaint procedures
Participating HE institutions are invited to submit feedback on proposals and draft text at key stages of the evaluations. This includes:
- An invitation to review the proposed composition of the expert group and submit any concerns.
- An invitation to review and provide feedback on proposed evaluation questions and criteria. The expert group will examine the feedback and consider any proposed changes.
- An invitation to review the final report chapter about the institution and submit any concerns about the representation of the institution or about recommendations made. The expert group and NOKUT will examine any submitted concerns in light of the evaluation data, and concerns that are deemed to be valid are addressed in the redrafting of the report text.
- An invitation to submit a position statement that responds to the evaluation report. The position statements are published with the report.
After the publication of the report, institutions are also invited to complete a survey on how they have experienced the evaluation process and how the evaluation report has been received.
Appeals can be made on the grounds of procedural error in the conduct of the evaluation or on the grounds of perceived bias. Appeals are reviewed by an Appeal Panel that includes three representatives from NOKUT’s board, including the HE student member. They are:
- Lise Iversen Kulbrandstad (head of the NOKUT board and professor at Inland Norway University of Applied Sciences), appointed until 31.12.24
- Jan I. Haaland (deputy head of the NOKUT board and professor at NHH Norwegian School of Economics), appointed until 31.12.22
- Aleksander Thiren Sriskantharajah (student member of the NOKUT board), appointed until 31.07.22
The panel may request that the evaluation report is withdrawn or redrafted.
Each evaluation has a two-part follow-up:
- The first follow-up activity is a seminar held within 6 months of the report publication. For this event, NOKUT invites the participating institutions to reflect on recommendations made in the evaluation report, and their plans for quality enhancement.
- The plan for the second follow-up activity is made within 8 months of the publication of the evaluation report, and the activity itself takes place within 1–2 years. The follow-up is designed in dialogue with the participating institutions to ensure that it is fit for purpose. It revisits the discussion from the initial seminar and addresses the recommendations from the report and quality enhancements that have taken place since the evaluation.