National study programme evaluations

National study programme evaluations are part of NOKUT’s work to assure and enhance quality in education. NOKUT conducts comparative, thematic evaluations of selected groups of study programmes. These will often be study programmes within a particular academic subject.

Objectives for NOKUT’s national study programme evaluations

NOKUT’s evaluations should generate knowledge about quality in education and facilitate quality enhancement. Evaluations are not designed to investigate compliance, but to identify strengths and weaknesses, highlight examples of good practice and give recommendations for quality enhancement. The evaluation process should facilitate learning through the exchange of experience and knowledge, and the evaluation results are published in a final report. 

Framework for NOKUT’s evaluations of quality in education

The Act relating to universities and university colleges § 16-2 establishes that NOKUT’s work on evaluations is professionally independent. § 16-3 states that the  Ministry of Education and Research can instruct NOKUT to carry out such evaluations pursuant to §16-2.

NOKUT's evaluations fall under the national framework for the evaluation of Norwegian research and higher education and should therefore contribute to quality work at the institutions, knowledge-based policy development, a knowledge base for study applicants and external partners, public trust in higher education, and transparency in the use of public funds.

NOKUT’s evaluations are designed and carried out in line with the European Standards and Guidelines for Quality Assurance in the Higher Education Area (ESG 2015).

Based on this framework, NOKUT has developed a set of principles for our external evaluations.

The evaluation process

Below is an overview of key elements in NOKUT’s evaluation process.

• Preparations

  NOKUT selects evaluation themes in dialogue with stakeholders, in order to ensure that the evaluation is relevant and useful, The themes selected should take into account the particular characteristics of the academic discipline, as well as other key aspects of the educational context, and they should target areas with significant potential for quality enhancement. Stakeholders may, for example, be involved through:

  • Input meetings
  • Reference group meetings
  • Hearings

  The evaluation terms of reference, including the evaluation themes, the data collection plan, and the milestones, is  published on NOKUT’s website and are communicated to all participating institutions before the evaluation is initiated.

• Expert panel

  In evaluations assessments are carried out by a panel of peer experts appointed by NOKUT, while NOKUT is responsible for the evaluation methodology and administrative support.

  NOKUT must ensure that the panel composition is fit for purpose, and that it includes at least one student and at least one employer or professional practitioner. It will usually also include international experts. NOKUT invites stakeholder input on the possible composition of the committee and each participating HEI is given the opportunity to comment on the proposed expert panel before the experts are appointed.

• Data

  Data used in the evaluation includes the self-assessment, digital or physical site visits, and registry data from DBH. It may also include other forms of quantitative or qualitative data.

  NOKUT develops the self-assessment form in dialogue with the expert panel and key stakeholders. NOKUT encourages HEIs to involve staff, students, alumni and relevant external partners in the self-assessment process. The expert panel examines the self-assessments and other data as part of their assessment.

  The self-assessments and other data are complemented by physical or digital site visits to each participating HEI. The experts will then interview programme stakeholders. This may for example include programme heads, departmental or faculty leadership, teaching staff,  students and employers or professional practitioners.

• The evaluation report

  The expert panel write an evaluation report, supported by NOKUT. The draft report is sent to the participating HEIs for review,  and they are invited to report any factual errors that should be corrected before publication. The HEIs are also invited to write a public statement about the evaluation report. Such statements are presented to the NOKUT board and published with the evaluation report.

• Stakeholder dialogue with NOKUT during the evaluation period

  In line with NOKUT’s principles for evaluations (section 3e)the participating HEIs may submit concerns and complaints related to the evaluation process. They may do so by sending an email to NOKUT’s project manager, with a copy to stein.erik.lid@nokut.no and postmottak@nokut.no.

  Each HEI can also submit an appeal regarding the evaluation report if it contains errors related to NOKUT’s principles for evaluations. A HEI cannot appeal NOKUT’s professional judgement. The appeal must be sent to postmottak@nokut.no, with a copy to NOKUT’s project manager, within four weeks of the publication of the evaluation report. The appeal is reviewed by an appeals panel consisting of three members of NOKUT’s board (the chair, the deputy chair and the student representative). The appeals panel may request that the evaluation report is revised or withdrawn.

Illustration of the process

Flexibility for institutions to align local periodic evaluations with NOKUT’s national study programme evaluations

According to Section 1–8, sixth paragraph of the Act relating to universities and university colleges, institutions must carry out periodic evaluations of all study programmes as part of their continuous improvement of the programmes. NOKUT is aware that some institutions may have planned to conduct a periodic evaluation during the same period as the study programme is scheduled to participate in a national evaluation, and that this may result in an increased workload for the institution. NOKUT emphasizes that the institutions themselves are responsible for conducting periodic evaluations of all their study programmes, and that the national evaluation cannot replace the institutions’ own periodic evaluations. At the same time, institutions have the flexibility to adapt their periodic evaluations to the national study programme evaluations conducted by NOKUT, for example by adjusting the timing of the local periodic evaluation, or by coordinating or supplementing the local evaluation of the relevant study programme with the national evaluation.

Overview of all external NOKUT evaluations

• National programme evaluations
  • Evaluation of post-experience master’s programmes in management
  • EUROMA – Benchmarking of master programmes across European countries
  • Evaluering av arbeidslivsrelevansen i disiplinfag (in Norwegian)
  • Evaluation of Integrated secondary teacher education
    
  • Evaluation of primary and lower secondary teacher education
  • Evaluation of the Professional programme in Medicine
  • Joint Academic Evaluations in Research and Education
    
  • Evaluation of Preschool Teacher Education in Norway 2010 – Summary
  • Engineering degree programmes 2008 – Summary
  • Allmennlærerutdanning 2006 – Del 1: Hovedrapport (in Norwegian)
  • Allmennlærerutdanning 2006 – Del 2: Institusjonsrapport  (in Norwegian)
• Other evaluations
  • Evaluering av implementeringen av RETHOS (in Norwegian)
  • Evaluering av Nasjonalt kvalifikasjonsrammeverk for livslang læring (in Norwegian)