research study

Turnitin Effectiveness in U.S. Colleges and Universities

Promote Academic Integrity

Evaluate Student Learning
Monitor Student Progress
turnitin.com

Table of Contents
1.0 Introduction ................................................................................................................... 3
2.0 Study Results ................................................................................................................ 4
3.0 Conclusion .................................................................................................................... 9
4.0 Methodology ............................................................................................................... 10

About Turnitin ..................................................................................................................... 12

research study | Turnitin Effectiveness in U.S. Colleges and Universities

1.0 Introduction
What impact is digital technology having on higher education? As states, systems,
schools and entrepreneurs rush to deliver a digital educational experience, the great
debate regarding the efficacy of technology in the classroom continues. Todays questions
will be answered years from now after the technology becomes more widely dispersed
and long-term studies have run their course.
Meanwhile, this report shares data and insights into how one digital technology, Turnitin, impacts education. Specifically,
this report examines Turnitins efficacy in helping to reduce plagiarism in student writing, facilitate paperless submission of
documents and allow instructors to provide feedback to students digitally.
Why does this matter? The corollary of reducing unoriginal writing is that a student who builds critical thinking, communication
and digital literacy skills as a result becomes a more informed person, productive employee and responsible citizen.
Transitioning to paperless assignment submissions opens a range of possibilities for furthering education by including
new insights into how to educate, measure and engage students in a rapidly globalizing environment. Finally, the ability for
instructors to offer digital feedback delivers a three-fold benefit: saving instructors grading time; increasing the depth of
instructor feedback; and engaging students more thoroughly than red ink in the margin ever has.
The sample for this study is large. Nearly one-third of all U.S. higher education institutions are included, and the five-year
timeframe provides an unprecedented look into the long-term impact of using a specific technology in the classroom. A
detailed methodology at the end of this report details how the study was conducted.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

2.0 Study Results

Key Findings

Colleges and universities using Turnitin reduced unoriginal writing by 39 percent over the course of the study.
The median reduction across the various types of institutions is 44 percent.

Institutions in the study submitted almost 55 million documents electronically, laying the foundation for
improvements to student teaching and learning.

Instructors are adopting online grading en masse because of the time saved, depth of feedback possible and
increased student engagement.

Reduction in Unoriginal Writing

In a 2011 survey by Pew Research of 1,055 college presidents, 55 percent said that plagiarism has increased. Of the college
presidents who believed plagiarism to be on the rise, 89 percent attributed the cause to computers and the Internet.1 There is
little debate that the plagiarism is a growing problem in classrooms. The bigger question is: what can be done about it?
Turnitin analyzed how unoriginal writing changed in over 1,000 colleges and universities that used Turnitin over three to five
years. This was a comprehensive study; we did not select institutions with only good news to share. Instead, we included all
Turnitin customers who met the following criteria:

U.S. non-profit, higher education institutions;

Began using Turnitin before January 1, 2011;
Continued to use Turnitin through January 1, 2013.

For each institution a baseline was established for the level of unoriginal content, which is defined as papers with more than
50 percent of its content matching content in Turnitins database of Web pages, scholarly publications and archived student
papers. To calculate the percentage of papers with unoriginal content, we divided each institutions number of unoriginal
papers by the number of their submissions within each academic year (July 1 to June 30), allowing us to compare the levels
of unoriginal content over time. More information on the methodology can be found at the end of this report.
Based on the 55 million papers analyzed over a five-year period, institutions using Turnitin experienced the aggregate change
in unoriginal writing shown in Chart 1.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

Chart 1: Levels of Unoriginal Content, All U.S. Higher Education

+1.8%

-10%

-13.2%

-20%

-25.5%

-30%
-40%

-39.1%

Year 1

Year 2

Year 3

Year 4

Year 5

Year 1 in Chart 1 reflects the baseline level of unoriginal content after the first year of Turnitin use across all 1,003 colleges and
universities analyzed. Year 2 depicts a slight increase (just 1.8 percent) in the level of unoriginal content, a development that
is not surprising. In fact, this slight increase occurs across nearly all levels of institutions in the U.S. and internationally. The
increase reflects the adoption lifecycle that institutions see when implementing Turnitin. Once the software has been tested by
students and instructors and proven to be effective, levels of unoriginal writing begin to decline.
By Year 3, there is a significant reduction in unoriginal writing (13.2 percent) against the baseline of Year 1. By the final year of
the study, colleges and universities using Turnitin experience an aggregate reduction in unoriginal writing of 39.1 percent.
The study also found that results varied based on the type of schooltwo-year or four-yearas well as the size of the school.
Table 1 displays how each group fared over the course of the study.
Table 1: Change in Level of Unoriginal Content by School Type and Size
Type of School

Population

2-Year College

<1,000

Count

Year 2

Year 3

Year 4

Year 5

44

2.40%

-48.00%

-45.50%

-64.90%

2-Year College

1,000-2,000

48

8.30%

7.80%

-23.10%

-43.20%

2-Year College

2,000-3,000

41

23.80%

17.90%

-20.90%

-41.80%

2-Year College

3,000-5,000

52

-67.20%

-67.60%

-67.30%

-77.90%

2-Year College

5,000-10,000

51

-9.60%

-13.70%

-57.30%

-68.10%

2-Year College

10,000+

37

6.10%

-35.00%

-36.30%

-60.00%

<1,000

76

18.90%

12.40%

-4.40%

-19.10%

4-Year College or University

4-Year College or University

1,000-2,000

197

3.10%

-20.20%

-37.00%

-43.80%

4-Year College or University

2,000-3,000

111

19.80%

2.20%

-4.20%

-45.90%

4-Year College or University

3,000-5,000

105

9.00%

8.20%

-25.00%

-33.30%

4-Year College or University

5,000-10,000

119

-0.10%

-12.00%

-17.80%

-25.70%

4-Year College or University

10,000+

122

8.90%

-0.40%

-10.60%

-23.00%

All Schools

All

1003

1.80%

-13.20%

-25.50%

-39.10%

research study | Turnitin Effectiveness in U.S. Colleges and Universities

Nine of the twelve cohorts initially experienced an increase in unoriginal writing, a Year 2 finding that is consistent with other
studies. By Year 3, however, five showed increases with six cohorts experiencing a significant reduction and one with a marginal
decrease in unoriginal writing. By Year 4, ten of the 12 cohorts experienced decreases of 10 percent or more. And by the final
year, all twelve categories reduced unoriginal content by at least 9 percent. The greatest reduction of 77.9 percent occurred in
2-year colleges with a population of 3,000-5,000 students. The median reduction across all cohorts was 43.5 percent.

The Move to E-Submission

An overlooked but important effect of technology on education is the move to digital submission of student work. Assignments,
due dates and post dates have migrated from being a manual, error-prone process to one that is transparent and untethered
to the physical location of the classroom or office. The resulting benefits explain the dramatic growth of online submissions
shown in Chart 2. In less than 10 years, electronic submissions increased from just over 500,000 papers to over 45 million
papers.
Chart 2: Papers Submitted to Turnitin by U.S. Higher Education Customers, 2004-2013

60M
50M

# of Papers

40M
30M
20M
10M
0

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

The benefits of curtailing the paper chase are numerous but they overlook other benefits afforded by online submission. First,
electronic submission allows for students to submit from their dorm room across campus or even from their home thousands
of miles away from the instructor. Indeed, online submission of written work is one of the foundational pillars of the move to
e-learning that is embodied in Massively Open Online Courses (MOOCs), virtual schools and e-learning programs that are
teaching students who, a few years before, were beyond reach because of distance or cost.
Digital submission holds promise for how we measure and improve student performance. For the first time, institutions can
maintain an online portfolio of their students work to help tailor learning, uncover trends and improve the performance of both
instructors and students. Terms like differentiated and one-to-one are surfacing from practices made possible by online
submission and technologies that can distill meaning from all this informationoften called Big Data.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

There is an environmental benefit to digital grading as well. Based on estimates from Conservatree.org, a single tree provides
8,333 sheets of papers. And in the last five years, higher education institutions included in this study submitted close to 55
million assignments. The average Turnitin submission is three pages, so nearly 20,000 trees were potentially saved over the last
five years from the use of Turnitin.

Reducing Time Spent Grading

In 2012, The Higher Education Research Institute at UCLA published a national survey of undergraduate teaching faculty. The
report shows that instructors spend similar amounts of time on preparing to teachincluding reading papers and gradingas
they spend on actually teaching students.
Table 2: 2010-2011 HERI Faculty Survey2
Percentage of Time Devoted to Teaching Activities
Preparing for teaching (including
reading student papers and grading)

Scheduled teaching
(actual not credit hours)

4.9%

5.8%

1-4 hours

11.5%

15.8%

5-8 hours

24.4%

34.6%

9-12 hours

22.4%

28.6%

13-16 hours

13.9%

9.2%

17-20 hours

12.1%

3.7%

21-34 hours

8.0%

1.7%

35-44 hours

1.8%

0.2%

45+ hours

0.9%

0.2%

Time
0 hours

While these numbers come as no surprise to instructors, they highlight the grading burden that instructors face relative to their
overall workload. Moreover, many instructors express concern that their feedback is largely ignored by students who have one
primary interest: the grade on the paper. In an analog world, this problem is compounded by the fact that students often have
trouble deciphering feedback their instructor scribbled in the margins or wrote over existing text.
In order to provide richer feedback, save time and engage students more in the learning process, instructors have rapidly
adopted Turnitin for online grading. Chart 3 depicts an average annual growth of 120 percent in papers graded digitally by
Turnitin users since 2004.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

Chart 3: Papers Digitally Graded in Turnitin by U.S. Higher Education Instructors, 2004-2013

4M

# of Papers

3M

2M

1M

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

In 2012, Turnitin asked 350 instructors why they use Turnitin for online grading.3 The survey revealed three key findings:
1. Respondents reported reducing their grading time by 31 percent on average compared to traditional pen and paper
methods of grading. If a typical paper traditionally takes 20 minutes to grade, instructors using Turnitin could grade it in 14
minutes, saving 6 minutes per paper. Projecting this by the eight million digitally-graded papers in this study would equate to
48 million minutes. Thats 800,000 hours or 33,333 days or 91 years!
2. Instructors noted that the quality of their feedback improved by 52 percent when using digital grading. The ability to use
electronic rubrics, QuickMark comments (customizable, template marks that are dragged onto the paper), voice comments
and general comments allows instructors to provide more thorough, personalized feedback in a shorter amount of time.
3. Instructors reported a 46 percent increase in student engagement with online graded papers. With their marks more legible
and the feedback more thorough, students spend more time reading and digesting instructor feedback and responding to
it. A Turnitin feature that indicates whether or not a student has reviewed the instructor feedback on the paper strengthens
student engagement.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

3.0 Conclusion
This report provides evidence that technology delivers significant educational benefits
by encouraging students to become more original writers, facilitating electronic
submission and helping instructors reduce the amount of time spent grading, while
increasing the quality of feedback they give and the level of student engagement. It also
shows that these areas are serving as foundational technologies that help institutions
improve academic performance. We can easily imagine that in a very short time period,
institutionsenabled by the technologies described in this reportwill provide a truly
differentiated learning experience that raises student outcomes.
Technologys impact on education is already happening and its acceleration is inevitable.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

4.0 Methodology
This section describes the methodology for this report, including the users studied, the
data that was analyzed and how the findings are calculated and presented.

Criteria for Inclusion

This report analyzes the full population of Turnitin customers at non-profit, higher education institutions in the United States that
began using the service before January 1, 2011 and remained a customer through January 1, 2013.

Time-Aligning Accounts Based on Usage

To provide comparable measurement between schools and to produce accurate results, this study aligned customers based
on years of use rather than the year in which they began using the service. For example, if Institution A began using Turnitin in
2010, it would have two years of usage (2011, 2012) and if Institution B began using Turnitin in 2008, it would have four years
of usage. Customers were aligned based on their years of usage (First Year, Second Year, Third Year, etc.) to study the levels
of unoriginal content after identical years of usage.

Calculating the First True Usage Start Year

Institutions that subscribe to Turnitin in a given year do not necessarily generate meaningful usage information in the first
year under license. In particular, Turnitin deployments at larger institutions are often part of larger initiatives to change
academic integrity policies, adopt a new learning management system or support professional development and training.
For example, an institution of 4,000 students might only submit 40 papers to Turnitin as they test the service in the first
year, then increase submission volume to 3,000 in the next year. Using the 40-submission year as a baseline measurement
would skew the analysis because it does not reflect a sufficient sample size.
To control for this, the study calculates the true usage start year based on the following guidelines:

A customer must have used the product for a full calendar year.
The first year is defined as the first year of usage that reflects a minimum of 10 percent of total lifetime submissions
by the institution.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

10

Unoriginal Content and Writing

The Turnitin service identifies matches in copy between papers submitted to the service and content in the Turnitin databases.
Each paper that is submitted receives an Overall Similarity Index (OSI) that shows the percentage of content in the submitted
paper that matches to content in the Turnitin databases. This study only considered papers with OSI scores between 50-100
percent of matched content. While 50-100 percent matched content represents a significant amount of unoriginal writing, it
does not necessarily equate to plagiarism.

Calculating Effectiveness
After accounts were aligned based on years of true usage, a sum of originality submissions for all institutions was generated,
as well as a sum of all papers with levels of unoriginal content above 50 percent for the first year of usage. By dividing the
sum of 50-100 percent unoriginal papers into the total submissions, we arrive at a percentage of unoriginal papers. We used
this same formula to calculate the level of unoriginal writing for each year of usage and compared it to the first year of usage
to measure the change in highly unoriginal papers over time.

Comparisons Between Groups

This study examines the changing levels of unoriginal content by type of institution (two-year or four-year) and student
population. It does not provide a basis of comparison for levels of unoriginal writing between groups. Such a comparison
would lead to improper conclusions, because each group is measured against a different baseline of unoriginal papers.
Therefore, it is not appropriate to conclude that two-year colleges with populations of 3,000-5,000 students (with a percent
reduction in unoriginal papers of 78 percent) have a lower rate of unoriginal content than four-year schools with populations of
5,000-10,000 (26 percent reduction) because the study (intentionally) does not provide the baseline percentage of unoriginal
content per group.

Pew Research Center. The Digital Revolution and Higher Education, Pew Research Center, 2011. Web. 9 September 2013. <http://pewinternet.org/Reports/2011/Collegepresidents/Summary/Findings.aspx>

Hurtado, S., Eagan, M. K., Pryor, J. H., Whang, H., & Tran, S. (2012). Undergraduate teaching faculty: The 20102011 HERI Faculty Survey. Los Angeles: Higher Education
Research Institute, UCLA.

Survey Finds Turnitin Saves Time and Improves Feedback. Turnitin, 2012. Web. 4 Jan 2014. <http://turnitin.com/en_us/resources/blog/421-general/2034-survey-finds-turnitinsaves-time-and-improves-feedback>.

All products and services mentioned in this document are trademarks or registered trademarks of their respective owners.
2014 iParadigms, LLC. All rights reserved. Version 0214.

research study | Turnitin Effectiveness in U.S. Colleges and Universities

11

About Turnitin
Turnitin is the global leader in evaluating and improving student work. The companys cloudbased service for originality checking, online grading and peer review saves instructors time
and provides rich feedback to students. One of the most widely distributed educational
applications in the world, Turnitin is used by more than 10,000 institutions in 126 countries to
manage the submission, tracking and evaluation of student work online. Turnitin also offers
iThenticate, a plagiarism detection service for commercial markets, and WriteCheck, a suite
of formative tools for writers. Turnitin is backed by Warburg Pincus and is headquartered in
Oakland, Calif., with an international office in Newcastle, U.K.
Turnitin is a service of iParadigms, LLC
1111 Broadway, 3rd Floor
Oakland CA, 94607 USA
USA/Canada: 866-816-5046
International: +1 510-764-7600
sales@turnitin.com
turnitin.com

research study | Turnitin Effectiveness in U.S. Colleges and Universities

12