Resources for reviewers

Good practice for reviewing

High quality reviews are invaluable in helping a funding body select the best and most appropriate applications to support. They also provide constructive feedback to applicants to help them develop and further their research plans and improve future grant applications. 

A high-quality review should:

• Provide clear and concise comments and recommendations that clearly identify strengths and weaknesses against the criteria for the funding call. 

• Provide justification for your comments and the overall evaluation. 

• Raise concerns in the form of questions for the applicant.

• Provide a balanced assessment of the proposal.

• Provide constructive criticism and feedback, that expresses negative comments tactfully.

• Be open to new approaches and perspectives.

To help provide a high-quality review, a reviewer should:

• Read the assessment criteria and scoring matrix.

• Be objective and professional.

• Be aware of unconscious bias.

A reviewer should not:

• Make the review personal.

• Exceed word limits/space restrictions.

• Include anything to identify the reviewer, including references to the reviewer’s work, who they worked with or where they have worked.

Questions to consider when reviewing an application

The funding body will provide specific assessment criteria and questions to ask when reviewing applications, however generally speaking reviewers should consider:

• What is the project’s contribution to the discipline?

• Is the project innovative and original?

• How important are the research questions, or gaps in knowledge, that would be addressed?

• Are the researchers capable of undertaking the work and to do it in the time and budget proposed? Do they have the right team, experience, skills, and support? Do they have the appropriate training? 

• Have any impacts of disruption described by the applicant been appropriately considered?

• What are the strengths and weaknesses? Are there major flaws?

• Is the methodology (and/or experimental design) clearly set out and justified? Are the methods appropriate? Are there alternative approaches that should be considered? What could they do better?

• Is there evidence of sufficient planning in terms of health and safety?

• Are there ethical issues to be addressed? Permissions? Are there any issues here? Is the project working appropriately with local partners and collaborators? See our list of resources on research conduct and ethics in the field.

• Is the budget and request for resources sufficiently detailed and appropriate? Does the proposal represent good value for money?

• Is there awareness shown of the relevant environmental, social and cultural responsibilities and impacts of the project, and strategies to minimise and mitigate these?

• Are there plans to disseminate findings locally as well as through academic channels?

This guidence draws on resources from the UKRI Medical Research Council, UKRI Engineering and Physical Sciences Research Council and Reviewing a Manuscript for Publication - Allen S. Lee.

Conflict of interest

The integrity of peer review is of paramount importance. This means that any personal or professional interests as a reviewer must never influence the outcome. A conflict of interest may occur where:

• The applicant is a close friend or relative.

• The reviewer is directly involved in the work the applicant proposes to carry out.

• The reviewer works in the same research organisation/department as the applicant(s), co-applicant(s) or project partners.

• The reviewer works closely with the applicant(s) (e.g. as a co-author or PhD supervisor) or has done within the last five years.

• The reviewer may benefit financially from the work.

If you consider that you have a conflict of interest (or you are unsure) you should declare this to the funder. Depending on your declaration of interest, the funder may still ask for your review of the proposal, taking note of your declaration, providing you feel capable of reviewing the proposal objectively. 

Unconscious bias

In order to identify the highest-quality applications and foster excellence in the discipline, it is essential that funding decisions are made purely based on the quality of the proposed research and merit of the applicant. All applicants should be treated equally and applications assessed on equal terms. 

The information below serves to prompt thinking on the topic of unconscious bias. It is strongly suggested that you explore the online resources listed. These provide both comprehensive summary pieces and more detailed information on this topic. We also encourage you to engage in appropriate training in your institution.

What is unconscious bias?

Unconscious bias is when we make judgments or decisions based on our prior experience, personal thought patterns, assumptions or interpretations, and we are not aware that we are doing it. This arises from a feature of the human brain that allows us to automatically make quick judgements. Unconscious bias can be triggered by many things, not just gender, race, ethnicity or other visible diversity characteristics, but also age, gender identity, physical abilities, religion, sexual orientation, weight, name, language and many other characteristics. 

Unconscious bias in peer review

Alongside bias associated with personal characteristics, within the peer review process unconscious bias may occur in the form of:

• Confirmation bias (e.g. knowing of the excellent work an individual has done in the past and assuming that the application under review is equally exceptional, resulting in less critical evaluation).

• Bias for a particular research area (e.g. having more enthusiasm for applications addressing your own area of research).

• Institutional bias (e.g. focusing unduly on the institution’s reputation, size, type, location, or prior research).

• Affinity or similarity bias (e.g. favouring applications from those similar to the reviewer in some way).

• The halo effect (e.g. one perceived positive trait of the applicant makes a reviewer view everything about the application in a positive way. The opposite of this is the ‘horn effect’).