REVIEWER ORIENTATION

The Reviewer of the SC CTSI Pilot Funding Review Committees plays a critical role in the functioning and success of the peer review process. This orientation is designed to provide an overview of the review process and information about Reviewer responsibilities in order to ensure effective leadership.

CONTENTS REVIEW PROCESS OVERVIEW..2

Proposal Assignment and Process Overview2 Reviewing the Proposals. 2 o Written Critique o Scoring o Review Criteria and Considerations Review Committee Meeting Activities..3 o Summary o Presentation and Discussion o Final Score and Voting

REVIEWER RESPONSIBILITIES.5
Pre-Meeting Responsibilities. 4 Meeting Responsibilities. 4 o General o For each assigned Application Post-Meeting Responsibilities4

ETHICAL CONDUCT...5
Conflict of Interest. 5 Confidentiality.5 Research Misconduct... 5

1
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

REVIEW PROCESS OVERVIEW

PROPOSAL ASSIGNMENT AND PROCESS OVERVIEW
Proposals are reviewed using a dual review system. Submitted applications will be assigned to a review committee organized by area. All proposals will be reviewed by at least two individuals with expertise in a closely related, appropriate field, and discussed by the entire review committee. Proposals will be scored according to the criteria listed in the descriptions for each award. Reviews and feedback will be returned to the proposal PI. Proposals will be funded based on their scores, and their alignment with programmatic goals ORD assigns proposals to Review Committees Review Committee Chair/Co-Chair assign proposals to reviewers within the Review Committee ORD distributes reviewer assignments Reviewers examine review assignments, review materials and instructions Reviewers review assignments for conflict of interest or concerns related to content match with expertise Reviewers submit reviews and scores to ORD ORD compiles and distributes scores to Review Committee Chair/Co-Chair Chair/Co-Chair and reviewers prepare for discussion at the Review Committee Meeting Review Committee Meeting takes place, final scores are determined and compiled

REVIEWING THE PROPOSALS

Written Critique Written Critique is a valuable resource to the PI. If the proposal is not successful in securing funding, reading reviewer comments will assist the PI/team in developing strategies to address the reviewers feedback Reviewers will use bullets to note strengths and weaknesses for each of the review criteria Reviewers will write a paragraph summarizing the factors that informed their Overall Impact score

Scoring

The scoring system utilizes the NIH 9-point rating scale (1 = exceptional; 9 = poor) Each reviewer assigned to an application gives a preliminary overall impact/priority score for that application using the customized scoring form provided for each individual funding opportunity. The scoring criterion will vary slightly depending on the specific requirements of the funding opportunity Reviewers weigh the different criteria as they see fit in deriving their overall score Preliminary scores are used to determine which applications will be discussed in full at the Review Committee Meeting Final scores are decided at the meeting

Review Criteria and Considerations Overall Impact Reviewers will provide an overall impact/priority score and critique Reviewers will provide a paragraph summarizing the factors that informed their Overall Impact score Note that an application does not need to be strong in all categories to be judged likely to have major scientific impact and thus deserve a high impact/priority score Review Criteria

Reviewers will consider each of the scored review criteria as presented in the scoring form in the determination of scientific and technical merit and give a separate score for each The major criteria (scored and un-scored) for each SC CTSI funding opportunity include: Significance, Approach, Innovation, Translational Potential, Investigator(s), Environment, Budget, Feasibility and Future Potential Additional Review Criteria As applicable for the project proposed, reviewers will evaluate additional items while determining scientific and technical merit and in providing an overall impact/priority score, but will not give separate scores for these items. o Protection for Human Subject o Inclusions of Women, Minorities, and Children o Vertebrate Animals

2
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

o o o o o

Biohazards Resubmissions Renewal Revision Budget concerns

Additional Comments to Applicant Reviewers may provide additional guidance and feedback to the applicant

REVIEW COMMITTEE MEETING ACTIVITIES

Summary ORD will begin the meeting by reviewing policies and describing meeting procedures As decided by the Chair/Co-Chair applications may be reviewed based on the preliminary, overall impact/priority score (beginning with the best scores) Applications will be grouped together when feasible (e.g., same mechanisms, new investigators, or clinical applications) In most cases, only the more meritorious applications (based on preliminary scores) will be discussed at the meeting Applications that are discussed at the meeting will receive a final impact/priority score, individually assigned reviewer criterion scores, a summary statement with critiques, and a summary of the discussion Applications that are not discussed will receive summary statements containing written critiques and individual criterion scores from assigned reviewers

Presentation and Discussion Applications will be introduced by the Chair of the Review Committee Assigned reviewers will share their preliminary overall impact/priority score and should be prepared to explain the significance of the problem and the overall impact the research will have on the field Group discussion follows assigned reviewer presentations Open discussion of scientific merit may result in disparate levels of enthusiasm The reasons for any disparities should be made clear to allow for both an informed vote by all panel members, and also a high quality summary statement Because consideration of human subject protections, inclusion plans, vertebrate animals or biohazards can reflect scientific and technical merit, these elements are discussed before moving to a final scoring

Final Score and Voting Based on the presentation and discussion, and the preliminary overall impact/priority score from each assigned reviewer, each discussed application is given a score by all reviewers who are eligible to vote on that application Members vote on each application for which they do not have a conflict of interest Only reviewers who are in attendance (in person or by phone) vote on final, overall impact/priority scores Reviewers may use non-numeric impact/priority scores, as appropriate: o AB abstain from voting o CF conflict of interest; did not participate in the discussion and scoring o NP not present during discussion The scores from all eligible reviewers for a given application are collected and averaged (calculated to one decimal point) to determine the final overall impact/priority score Once scoring is complete for all proposals, the Chair/Co-Chair may decide to view anonoymized, averaged scores (and distribution) for Review Committee assigned proposals and re-visit discussion of proposal scores with disparate distribution

3
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

REVIEWER RESPONSIBILITIES
PRE-MEETING RESPONSIBILITIES
Receive and review proposals as assigned by the Review Committee Chair Examine review assignments, review materials and instructions Review assignments for conflict of interest or the appearance of conflict of interest and notify ORD of issues Review assignments for concerns related to content match with expertise and notify ORD of issues Review the Reviewer Orientation Materials Read and evaluate assigned proposals Complete review forms (addressing all requested criteria) for each assigned proposal Prepare for discussions at the Review Committee Meeting

MEETING RESPONSIBILITIES
General Responsibilities at the Meeting

Operate within review regulations, policies, and procedures Set a professional tone for the discussion Engage with other Review Committee members during discussions Ensure review is fair, equitable and free of bias Speak to Review Committee Chair about any concerns Beware of personal biases Consult with Review Committee Chair on inadequate reviews or irregularities Follow the order of review, as communicated by the Chair

Responsibilities for Each Assigned Application

Provide initial level of enthusiasm, preliminary score and critique for each assigned proposal Provide concise review of assigned applications with emphasis on its impact, strengths, and weaknesses Open the review for further discussion by all members of the panel

POST-MEETING RESPONSIBILITIES
Provide feedback on the performance of Chairs/Co-Chairs Identify and discuss issues or problems that arose during the meeting and potential solutions with ORD

4
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

ETHICAL CONDUCT
The peer review system relies on the professionalism of each reviewer to identify any conflict of interest (COI) or apparent COI that may affect or appear to affect the integrity of the peer review process. The SC CTSI Pilot Funding COI rules for initial peer review are derived from federal regulations and policy used by NIH governing the Scientific Peer Review of Research Grant Applications and Research and Development Contract Projects.

CONFLICT OF INTEREST
Situations that create conflicts of interest or the appearance of conflict of interest should be reported to ORD Check for potential conflicts of interest (or appearances of conflicts) and alert ORD immediately of any conflicts of which you are aware All participants should recuse themselves from the review of any application where their participation constitutes a real or apparent COI During the meeting, if a reviewer has a real or apparent conflict of interest with any application, the reviewer must leave the room during evaluation and scoring of that application In accepting the review assignment and signing the post-review certification forms, each reviewer certifies that he/she did not participate in an evaluation of any application or proposal with which he/she knowingly had a conflict of interest

Considered conflict of interest: Individuals o Individuals participating in the proposed project with major roles contributing to the scientific development or execution of the project in a substantive, measureable way, whether or not compensation is requested. This includes but is not limited to proposal: PI, Co-PI, Co-investigator Senior/Key Personnel Other significant contributors Collaborators Consultants Mentors o Individuals with a special interest in the project (financial or managerial) who stand to gain resources (staff, funding, etc) and or publications, etc directly as a result of the project Professional Relationships o Reviewer who is currently or has within the 3 preceding years, collaborated with, prepared a joint funding application with, co-authored a publication with, and/or mentored or trained the PI, CO-PI, Co-investigator or an individual named in the application o Reviewer who has written a letter of general support for the proposed project o Reviewer named as a speaker/participant in a conference/seminar/workshop proposed

Not considered conflict of interest: Working in the same School/Department/Unit

CONFIDENTIALITY
Respect for the privacy of the investigators' ideas is important; all applications and related materials are privileged communications that cannot be shown to or discussed with unauthorized individuals In accepting the review assignment and signing the post-review certification forms, each reviewer certifies that he/she fully understands the confidential nature of the review process and agrees to confidentiality and nondisclosure

RESEARCH MISCONDUCT
Research misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results, but not honest error or differences of opinion It is vital that you do not make allegations of potential misconduct in the critique

5
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm

Instead, such concerns must be brought to the attention of ORD in a confidential manner, preferably before the study section meets

6
Note: NIH Peer-Review Guidelines content used with modification http://grants.nih.gov/grants/peer/reviewer_guidelines.htm