National guidelines and regulations are needed on the use of generative AI (GenAI) by universities in the United Kingdom to assess the quality of research for the allocation of public funding, found a study published on 1 December.
Measures must also be taken to prevent the tools being used for research assessment from falling into the hands of external parties, including ‘tech bros’, who might capitalise on the information and data contained in academic research, the report’s author said.
The study, led by the University of Bristol and funded by Research England, is the first to examine how higher education institutions are using AI to evaluate research quality and how AI might be used to save universities time and money.
It focused on the Research Excellence Framework (REF), whose outcomes influence how £2 billion (US$2.7 billion) in public funding is allocated for research. The REF is criticised for being overly burdensome and costly for universities.
Lead author Richard Watermeyer, professor of higher education at the University of Bristol, said: “GenAI could be a game-changer for national-level research assessment, helping to create a more efficient and equitable playing field.
“Although there is a lot of vocal opposition to the incorporation of it into the REF, our report uncovers how GenAI tools are nevertheless being widely if currently, quietly used, and that expectation of their use by REF panellists is high.”
The last REF happened in 2021, and, following a review, changes to guidance for the next one, REF2029, are expected to be announced shortly. The total costs of REF2021 were estimated to be around £471 million (US$628), with an average spend of £3 million in each participating higher education institution. The costs for REF2029 are expected to be much higher.
Watermeyer told University World News that there is never a time in higher education institutions “when academics are not, in some shape or form, thinking about the next exercise and the review process.
“Universities need to consider how AI can alleviate some of the work and what job it can do,” he said. “Can it help to review, or at least help to score research against the evaluation criteria set out with the REF guidelines for output and impact, for example?
“Another dimension is how we can use the tools to do a more accurate job in helping to predict how the panellists, who do the formal assessments, will judge our research submissions.
“I don’t think there was a university that we consulted whose staff didn’t think that the panellists themselves wouldn’t be utilising the [AI] tools because of the sheer enormity of the task and because it was good value for money and saved time.
