Artificial intelligence as a tool in analytical work
A new report from NOKUT explores how artificial intelligence (AI) can be used as support in analytical work related to external quality assurance in higher education. The report builds on experiences from NOKUT’s thematic analyses of the third round of periodic reviews.
The report forms part of NOKUT’s overall thematic analysis and has a methodological perspective. The aim is not to replace professional judgement, but to examine how large language models can support work with large volumes of text-based data.
– Artificial intelligence can be a useful support tool in analytical work, particularly when working with large amounts of text. At the same time, our experience clearly shows that AI cannot replace professional judgement and expertise in analytical work, says Philipp Emanuel Friedrich, Senior Adviser at NOKUT.
The dataset for the analysis consists of 48 supervision reports from the period 2017–2024. The report compares experiences from AI-assisted analysis with traditional manual analysis.
Strengths and weaknesses of AI in analytical work
The analysis shows that language models are particularly strong in handling large data volumes, structuring information and identifying overarching patterns. This can make it easier to gain an overview and support work on thematic analyses.
At the same time, the report clearly shows that manual analysis remains essential for understanding context, interpreting nuances and making judgment-based assessments. Human quality assurance is therefore necessary at all stages of the analytical process.
– It is the combination of AI-assisted analysis and manual professional assessment that produces the best results. AI can contribute structure and overview, but it is humans who must interpret, assess and quality assure the findings, says Friedrich.
The report also highlights important conditions for the effective use of AI in analytical work, including the need for well-structured data, clear instructions, and sufficient subject expertise and AI competence among those using the tools.
Read the full report here (pdf)