We welcome that REF2028, overall, is taking strides towards measuring the capability of institutions to develop researchers and recognising collective endeavours, which should encourage strategic, longer-term approaches to investing in the research base. However, there is a significant amount of change (between REF2021/REF2028) and limited time to implement this, particularly given the impacts of the disruption of/since Covid. There is a danger this will become even more complex for departments/institutions and increase unit burden. Also, we do have some concerns (outlined below) around the unintended consequences across disciplines or institutions for perverse impacts on stated goals for equity, diversity and inclusion.
5. What practical challenges may institutions face in implementing these changes?
Some concerns have been raised by the geographical community, particularly from smaller units and institutions, about the challenges around harmonising local and central data processes. There are particular concerns about the timeframe.
6. How might the funding bodies mitigate against these challenges?
Greater, and earlier, clarity on volume at a point that allows institutions to plan their submission outputs and impact case studies. Particular concerns expressed by departments that may increase, or decrease in size over this period, particularly smaller units where there may be the greatest unintended burden.
7. What would be the impact of these changes on individual researchers and particularly those with protected characteristics or other
underrepresented groups?
Equality impact assessments have been an important mechanism for ensuring there is a fair and representative submission to REF in terms of people and outputs and outcomes. It is important the proposed process enhances this agenda and does not erode it. There should be some consideration of the composition of the staff who are contributing to the volume measure in terms of protected statistics.
Output Submission
8. What would be the impact of these changes on individual researchers and particularly those with protected characteristics or other underrepresented groups?
The logic of de-coupling researchers from their outputs supports the aim of the assessment to assess institutions rather than individuals. It supports the aim of mobility between sectors (academic, professional practice, for example and reduces disincentives to hire ECRs. Removing the need for the individual staff circumstances process will simplify matters for institutions and also have a positive impact on staff affected (this was a very sensitive and difficult process for many of them).
However, there are potential adverse consequences:
- Heavy reliance on a small number of researchers whose output is prolific. It will be important that other elements of the submissions act as a counter-balance to this risk so that institutions do not concentrate resources in a way that is detrimental to nurturing a consistent and equitable research culture that develops ALL research-staff.
- An incentive for institutions to engage in REF poaching key individuals for short term engagements. The indicators of substantive connection need to be robust.
- The increased burden on institutions in terms of selection of outputs. This could reinforce the likelihood that under-represented groups will remain so.
- In theory, there could be an even greater amplification of high volume of outputs from a small number of prolific individuals if they are transdisciplinary and can be submitted in multiple Units of Assessment i.e. the potential for double-counting outputs across multiple institutions rather than attribution to the environment(s) that enabled the research. This could lead to inflation of score and differential impact across disciplines (UoA).
- A risk that non-traditional publications and interdisciplinary research are deprioritised due to risk aversion and the time and resources needed todevelop and curate such output types.
Concerns expressed by some around the reduction in the weighting of outputs from 60% to 45% - with expressions that the most accurate way of judging the quality of research is to assess the quality of the outputs.
Suggestions by some of mission creep - of increasingly trying to create change in the sector and moving from metrics of judging the quality of what has been done to incorporating greater steers of what should be done. This is evident in the idea of judging the research environment and culture and giving so much weight to it. This is not, of course, in any way arguing against more inclusive research cultures, but the role of REF to change this.
9. What impact would these changes have on institutions in preparing output submissions? For example, what may be the unintended consequences of allowing the submission of outputs produced by those on non-academic or teaching-only contracts?
There will be a great burden on institutions in preparing output submissions given there will be a larger potential field of outputs and fewer limiting options. This will have differential effects across institutions and panels. There is an ethical issue in terms of including output from staff that falls outside their explicit job purpose and for which they have not been allocated time.
There is potential for staff on teaching only/non-academic contracts to feel pressured to publish when they don't have protected time to do this. Members of the community applaud the aim to include technical staff. There is the potential for there to be even more intense competition/comparison between staff due to the prestige of being submitted to REF which could negatively impact on the institutional environment.
10. Should outputs sole-authored by postgraduate research students be eligible for submission? If so, should this include PhD theses?
Universal agreement that sole-authored papers from postgraduates should not be eligible for submission. There is already substantial pressure on PGR students. This could introduce extraneous considerations in the selection of PGRs such as ability to contribute to REF rather than the merits of the project and the imperative to ensure a diverse and representative PGR body. This would also require careful thinking about the contractual and relational status of PGRs.
11. What would be appropriate indicators of a demonstrable and substantive link to the submitting institution?
- A legal contract of employment (including secondments) is the most appropriate basis for determining a substantive link to a submitting institution. However, concern was expressed that a six-month contract is not long enough to demonstrate a substantive link.
- Evidence of the post-holder supporting wider institutional initiatives as an institutional citizen and colleague.
12. Do the proposed arrangements for co-authored outputs strike the right balance between supporting collaboration and ensuring that assessment focuses on the work of the unit?
Significant questions have been raised about multi-authored outputs and contribution. The consensus is that this requires more thought.
The proposed approach will have quite different impacts across the discipline of geography - with those in some parts of physical geography (STEM) far more likely to have multiple authors on a research paper, other parts single authored papers.
Those multi-authored papers have the risk of being counted multiple times. There also a high chance that research assessed in previous REF cycles (e.g. underpinning new model evaluations, meta analyses) will be counted again.
There are real questions of how authorship will be attributed in a robust way in light of (1) decoupling of outputs from individuals and the (2) UoAs not having a bounded identity around a specific group of staff.
13. Are there any further considerations around co-authored outputs that need be taken into account?
This needs more consideration
Impact Case Studies
14. What will be the impact of reducing the minimum number to one?
Reducing the minimum number of impact cases to one for small UoAs does reduce burden for smaller unit submissions, and may be an important step in growing a relevant discipline area within an institution, but it also runs the risk of removing anonymity around scoring and narrowing the unit's opportunity to describe a comprehensive set of benefits. Welcome removal of the requirement for underpinning research 2* threshold. This discriminated against research with impact built in from the start. The addition of rigour may have implications in terms of the types of impact/activity it encourages. In a discipline like Geography many different types of impact (social, economic, environmental etc.) are achieved by a huge variety of methods and working arrangements.
15. What will be the impact of revising the thresholds between case study requirements?
Concerns expressed that this will disproportionately benefit large institutions.
16. To what extent do you support weighting the impact statement on a sliding scale in proportion to the number of case studies submitted?
General support
Unit of Assessment
17. If the UOA structure is relevant to you/your organisation, please indicate clearly any changes that you propose to the UOA structure and
provide your rationale and any evidence to support your proposal.
Support retention of the Geography and Environmental Studies panel. However, we do note that the real issue is the pre-allocation of funding to the four panels. Last time this led to significant negative financial outcomes, particularly for Panel C. This may encourage gaming where institutions make strategic submissions to units of assessment (favouring Panels A and B). For geography this has the implication that submissions may well be split - with physical geography being submitted to Panel B and human geography to Panel C. This could have have negative implications for departments and the health of the discipline more widely.
Read the consultation response