Accept, reject, or revise?

Improving scholarship by
improving peer review

[This content was created for Wolters Kluwer and has been republished with permission. It has
been authored by Elizabeth S. Karlin, MA, staff editor; Jennifer Campi, senior staff editor; and
Mary Beth DeVilbiss, managing editor, Academic Medicine.]
Peer review is the assessment (or review) of research by a scholar (a peer) who has expertise in
the research topic or possibly the methodology applied. The purpose of peer reviewthe focus of
this articleis to help ensure that the published literature is of the highest possible quality.
Reviewers act as a sort of jury, determining if the research is of a quality that deserves
dissemination.
For centuries, the peer-review process has remained relatively unchanged: Researchers prepare a
manuscript, a report of the study or investigation they have conducted. Next, the researchers
send the manuscript to an appropriate journal. Usually, the manuscript undergoes an internal
review. A decision-making editor (an editor-in-chief or associate editor) decides whether to reject
the manuscript before review or to send it out for peer review. Decision-making editors, journal
board members, and even the manuscript authors themselves may suggest reviewers. On average,
two or three reviewers are selected from a pool of experts who are often authors themselves.
This next step in the processreview by a peeris an essential element of scholarship. Although
editors need not (and do not) always agree with or abide by reviewers recommendations, they
rely on reviewers as a vital source of information about manuscripts. Editors and readers, other
scholars, and the people who benefit from advances in sciencethat is, all of usrely on peer
reviewers to provide insights into research, to outline strengths and weaknesses, to uncover
critical flaws, and to illuminate new discoveries.
The most important questions peer reviewers can answer are:

Will this manuscript advance the literature? (Is the topic important? Will it change how we think
about an issue? Does the research contribute to the field?)
Is there a fatal flaw in the design (the methodology) or the argument (the logic)?
How can the author/s improve this manuscript?

To answer the questions above, peer reviewers should examine the reports introduction and
reference list, the methodology, the results or findings, and the discussion and conclusion. We
have provided a few questions and suggestions about each of these immediately below, followed
by a description of other resources to help reviewers.
Introduction and references: Do the authors provide enough background information, based in
the literature, for readers to understand the nature of the topic, the context, and the need for
the current report? Is the literature the authors cite comprehensiveincluding both the most
current articles as well as foundational research? Is the literature balancedpresenting different
perspectives? Do the authors state their purpose, question, and/or hypotheses clearly?
Methodology: Is the method appropriate for the problem or question the authors hope to study?
Have the authors provided enough details to allow future researchers to replicate their study? Is
the setting clear? Where and when did the research take place? Who are the research subjects or
participants? Did the authors procure all necessary ethical approvals, especially if working with
human or animal subjects? Have the authors described their tools and their analyses? Have the
authors considered all aspects of the problem?
Results: Do the results reflect the methodology? Do the authors report the findings of all their
tools and analyses? Have the authors shown their results to be significantstatistically and
clinically?
Discussion and conclusions: Have the authors considered whether their findings are generalizable
to other settings, other subjects? Have the authors considered all of the implications of their
findings? Have they shown how their findings will affect science? Have they considered avenues
for, or questions to address in, future research? Have they discussed any limitations and the
effects of those limitations on their conclusions?
Although the questions in the preceding paragraphs will help guide reviewers, here are some
additional resources:
Review Criteria for Research Manuscripts, 2nd Edition (www.aamc.org/reviewcriteria) is a
comprehensive guide for reviewers that outlines the entire review process and systematically
examines each aspect of research that reviewers should assess, from title and byline, through
abstract and intro, to conclusions and references. The resource was edited by two of Academic
Medicines associate editors and written by experienced scholars and reviewers. It includes a
useful checklist for reviewers; discusses new types of manuscripts such as reports of qualitative
research, systematic reviews, and descriptions of innovations; and it even covers reviewer
etiquette. The manual can serve as a useful tool for novice and experienced reviewers alike. It
will help any reviewer struggling with a difficult manuscript, will assist writers who want to
prepare the strongest possible manuscript, and will support trainees (and their mentors) who are
learning how to conduct, assess, apply, disseminate, and discuss research.
What Editors Want: An Overview for Reviewers (https://vimeo.com/academicmedicine) answers
a common reviewer question: What do editors want? This short video presentation by the editors

and staff of Academic Medicine covers reviewer etiquette, review format, and review content. It
is part of the journals expanding collection of reviewer resources, which include a downloadable
guide to reviewer recommendations and are available through the Academic Medicines For
Reviewers
page
athttp://journals.lww.com/academicmedicine/Pages/ForReviewers.aspx.
Additional videos and practice review exercises are in development.
Reviewer
workshops
During these dynamic, interactive workshops, Academic Medicines editors and staff present basic
information on peer review and then turn the session over to the participants. Session attendees
work in small groups to review an actual manuscript submitted to Academic Medicine. At the end,
the presenters and attendees discuss the manuscript, the process of reviewing it, and the insights
gained through the process. Contact the journal at acadmed_online@aamc.org to learn about
upcoming workshops and events.
Reviewer
resources
series
on
Academic
Medicines
blog,
AM
Rounds
Perhaps one of the best ways to learn about and improve peer review is to read what the peer
reviewers
themselves
have
to
say.
In
this
series
of
11
blog
posts
(http://academicmedicineblog.org/category/peer-reviewer-resources), reviewers who have won
Academic Medicines Excellence in Reviewing Award share their tips on how to conduct and write
quality peer reviews, their thoughts on the benefits and purpose of peer review, and even their
love of the process.
And hereenjoying peer review, the benefits of peer review for the reviewers and authorsis
where we want to end. In addition to the pragmatic benefits of review, including letters
recognizing service, lines on a CV, and sometimes rewards or CME credit, scholars note other
reasons for continuing to review one anothers research. Despite the time and effort that quality
reviews require, scholars assess manuscripts because doing so allows them the opportunity to
learn about the newest discoveries in their field. They are participating in the scholarly process,
giving back to the community that has helped them, and advancing science for the benefit of us
all.

Survey on employment decisions of U.S. and foreign

doctoral graduates: An overview

The U.S. is home to one of the largest scientific workforces in the world and thus holds an appeal for
those pursuing higher education and research. In the past decade, international mobility among
researchers has increased greatly, which has aroused the interest of policymakers in understanding the
factors that influence the researchers employment decisions. The National Science Foundation (NSF)
recently published areport that reveals the number of U.S. citizen doctoral graduates in science,
engineering, and health fields within the US. The survey report is interesting because it also tracks the
number of graduates who intend stay in the U.S. at the time of graduation, the reasons underlying their
decision, and their work conditions. Lets look at some of the survey findings to understand current
research employment trends in the US.
Scope of the survey
The NSF survey takes into consideration U.S. doctoral graduates from academic years 200109,
including U.S. citizens living in the U.S. (sample size 6085), U.S. citizens living abroad (sample size
315), temporary U.S. visa holders living in the U.S. (sample size 2660), temporary U.S. visa holders
living abroad (sample size 2123), and foreign doctorates living in the U.S. (207). It examines their
employment outcomes, working conditions, and the associations between their employment
characteristics and ratings of job factors to shed light on issues that potentially influence their
employment decisions.
Survey findings
According to the report, the factors influencing employment decisions vary among doctoral graduates
who were temporary visa holders and those who had U.S. citizenship. However, differences can be
observed across other groups as well in aspects such as job satisfaction and benefits extended by
employers. Here are some of the most remarkable trends observed in the report:
1. Expected stay rate
The expected stay rate (percentage of of doctoral graduates who intend to live in the U.S.) and the
actual stay rate (percentage of those who end up living in the U.S.) changed over time. 96.4% of U.S.
citizen doctoral graduates from academic years 2001-2009 reported that they intended to live in the
U.S., and in 2010, all of 96.2% were still living in the U.S. Although the stay rate seems fairly even,
when the total numbers were analyzed based on other factors, striking differences were observed.
Among doctoral graduates who were temporary visa holders, the expected stay rate was 76.4%.
However, the actual and expected stay rates differed as time since graduation elapsed. By 2010, only
68.5% of these respondents remained in the U.S. This difference seems to have its roots in various
factors, including the employment preferences, opportunities, and job satisfaction.

2. Employment preferences
Researchers career preferences depend largely on their interest and the field they are pursuing.
However, another important factor that influences their choice is their residence. As per the NSF
survey, foreign doctorates living in the U.S. and temporary visa holders were more likely to work in
private, for-profit sectors compared to U.S. citizens living in the U.S. and those living abroad. Another
notable trend was that graduates who were U.S. citizens living in the U.S. were the least likely group
to work primarily in R&D (47.4%), compared to temporary visa holders who were the most likely group
to work in R&D (67.5%). When it comes to field-specific preferences, in engineering, temporary visa
holders were more likely to work in the private, for-profit industry than were those living abroad.
Whereas, in the fields of computer and mathematical sciences, physical and related sciences
engineering, and health, temporary visa holders living abroad were more likely than those living in the
U.S. to work in the academic sector. Remarkably, of the doctoral graduates who were employed full
time, only 15.9% were working in postdoctoral positions.
3. The level of job satisfaction
All the analysis groups were asked to rate their level of job satisfaction based on nine job-related
aspects: salary, benefits, job security, job location, opportunities for advancement, intellectual
challenge, level of responsibility, degree of independence, and contribution to society. Temporary visa
holders living in the U.S. tended to be least satisfied with their principal jobs, whereas those residing
in the U.S. and abroad (48.3% and 51.9%, respectively) reported to be highly satisfied with their jobs.
However, between the two groups of temporary visa holders, those living abroad were more likely to
be very satisfied with their job benefits but have more issues regarding salary, job security, location,
and opportunities for advancement. Overall, all of those employed in non-academic sectors were less
satisfied with their jobs intellectual challenges and degree of independence, but were very satisfied
with salaries. On the other hand, those working in the private and government sectors were less
satisfied with their jobs level of responsibility than those in academia.
4. Job benefits extended by employers
All of the analysis groups were compared based on the benefits offered to them by the employers such
as, health insurance, employer contributed pension or retirement plan, profit-sharing plan, and paid
vacation or sick days. U.S. citizens residing in the U.S., temporary visa holders in the U.S., and foreign
doctorates in the U.S. were most likely to have been offered health insurance. When analyzed on their
attendance at professional meetings and conferences in the past one year, it was found that temporary
visa holders living in the U.S. had the lowest rate of attendance (64.8%); whereas U.S. citizens living
abroad had the highest rate of attendance (81.5%).
5. Reasons for job change
Many employed graduates opted to change their jobs. The group that was most likely to changed jobs
within two years of graduation was the U.S. citizens living abroad (44.7%) compared with the rest of
the groups (about 30%). When they were asked to choose among nine reasons for job change, pay,
promotion opportunities (58.1%), working conditions (32.3%), change in career or professional
interests (32.3%), job location (29.0%), and laid off or job terminated (22.6%) were the five most
common choices. Most U.S. citizens living in the U.S. reported pay, promotion opportunities as a
reason to change the job whereas change in career or professional interests was a less common
reason as compared to the two temporary visa holder groups. The two groups living abroad were more
likely to report job location as a reason (42.9% and 35.3% for U.S. citizens and temporary visa
holders, respectively) when compared to the two U.S.-residing groups (28.4% and 25.6% for U.S.
citizens and temporary visa holders, respectively).
Mobility in pursuit of a better future is the reality for most researchers. However, after attaining the
postdoctoral degree, graduates face uncertainty due to the dearth of career opportunities suitable for

their field and personal interests. This report is crucial from the perspective of policymakers,
academia, researchers, and institutions as it throws light on some of the most vital aspects of a
postdocs life right from deciding on the location where they wish to establish their future to selecting
the employment sector to changing jobs. Policymakers and other major decision makers in academia
and industry can use the findings of this report to provide researchers with guidance and support to
lead them on a successful career path.