REF, the Research Excellence Framework, is the new system for assessing research in higher education institutions in the UK. It is the successor to the RAE, the Research Assessment Exercise, which was held in 1992, 1996, 2001, and 2008. The School of Informatics will be one of the units assessed, evaluated by panel B, sub-panel 11, Computer Science and informatics. Alan Bundy sat on the sub-panels for RAE 2001 and 2008, and Bonnie Webber sits on the sub-panel for REF 2014. As for the RAE, doing well in the REF is essential to maintaining our budget, our reputation, and our position as a great place to do research.
REF assessment is based on five components:
Unlike RAE 2008, indications of esteem will not be collected. Every member of staff will want to ensure they have four excellent publications for REF 2, and the School will need to ensure we have a strong portfolio of impact case studies for REF 3 (we will require about a dozen case studies).
In forming their overall quality judgements, the sub-panel will assess three distinct elements of each submission - outputs, impact and environment - against the following generic criteria, with the following weights:
Each research output and case study will be given a star level:
The sub-panel will award an overall quality profile to each submission, for publication in December 2014. The profiles will show the proportion of each submission that is judged to meet each of four 'starred' quality levels and 'unclassified', in steps of 1 per cent. See the figures below for sample profiles, and an example of how the profiles are derived. (Click to enlarge.)
In the 2008 RAE, our profile was
**** 35%, *** 50%, ** 15%, * 0%, unclassified 0%
In order to retain our rank, we are likely to have to better that this year. Everyone should aim to submit papers of four star or three star quality.
The REF will list profiles in steps of one per cent, whereas the 2008 RAE listed profiles in steps of five per cent; so relatively small differences may significantly impact our ranking. Further, the REF rankings will include impact and environment, as well as outputs.
Most sub-panels will use citation information provided by the REF team. The Computer Science and Informatics sub-panel will, exceptionally, also use citation information from Google Scholar. The sub-panel will continue to rely on expert review as the primary means of assessing outputs.
Each output may be accompanied by a description of at most 100 words, providing factual information about the output that is not evident within the output itself. This could include, for example, additional evidence about how an output has gained recognition, led to further developments, or has been applied. You should aim to provide such a description for each output you submit.
The evidence must be succinct, verifiable, and externally referenced where appropriate. Where claims are made relating to the industrial significance of the output, the name and contact details of a senior industrialist must be given to allow verification of claims. Information provided should not comprise a synopsis of the output or a volunteered opinion as to the quality of the output, and information provided that is of this nature will be disregarded. Information provided must not include citation data: any panels that make use of citation data will be provided with the data by the REF team.
The sub-panel will assess all outputs on an equal basis, regardless of whether or not citation data is available for them. It will recognise the limited value of citation data for recently published outputs, the variable citation patterns for different fields of research, the possibility of 'negative citations', and the limitations of such data for outputs in languages other than English.
Impact case studies should include up to six key references to research that underpins the impact described in the case study, and identify up to three of these references that best indicate the quality of the underpinning research. The sub-panel will use this information to determine whether the quality threshold, defined as research that is predominantly at least equivalent to two star, has been met.
The School ran a competition to encourage the preparation of excellent impact case studies.
Keep in mind that you will need to submit four excellent outputs to the REF, published within the period 1 January 2008 to 31 December 2013. Maintaining excellent links to your published work on the web is a simple step to take that may maximise your citation count in Google Scholar.
The School will need to submit about a dozen cases of impact based on research conducted within the school and/or promoted by activities of the School. If you are involved with a case of significant impact, please contact the REF managers, Stephen Gilmore and Philip Wadler.