MEMO

To: Harvey Link, Vice President

Academic and Student Affairs

From: Ken Kompelien, Dean

Arts, Science & Business Division
Kyle Davison, Interim Administrator
Outreach Division

Date: January 27, 2010

Subject: Related Studies Grading Review Summary Report

Over the past several weeks, we have reviewed aspects of the Pharmacist-Assisted Technician Self-
Instructional Modules (PATSIM) Program in relation to allegations made by Mr. Larry Shereck. Mr.
Shereck, a former NDSCS College Outreach employee has made several allegations concerning “grading
inconsistencies” and has questioned the grading and record keeping practices of the PATSIM program.

As instructed by you in your memo to us dated November 24, 2009, the following is a summary report of
our review of the allegations made regarding the PATSIM program.

A brief description of the role Related Studies plays in the administration of the PATSIM program:
The Related Study Program offers over 100 non-credit correspondence courses/modules for individuals
in a vocational field who cannot attend regular college learning activities. The Related Studies Program
facilitates the enrollment, module distribution and referrals for students in the PATSIM Program, which
utilizes 10 of these modules. The Related Studies office currently grades 8 of the 10 PATSIM modules.
The other two are graded in the Pharmacy Technician Program office.

The Related Study Program houses the student PATSIM files which include each individual test taken by
every student in the program. The Related Study office mails out the PATSIM module materials and
books as orders come in. The Related Study office also maintains electronic and hardcopy files of
student records. Related Study works with the Pharmacy Tech Program to relay any student concerns,
questions, or any other problems that may come up for the student or the Pharmacist that is assisting
the PATSIM student.

All test material is kept in the student’s hardcopy file. The tests are kept for a minimum of 6 years but
generally held for 8-10 years or more. The test results are kept “forever”, however the tests may be
shredded as storage space requires after the 6-8 year period.
Harvey Link
Page 2
January 27, 2010

A brief description of Mr. Shereck’s responsibilities while temporarily assigned to the Related Studies
Department. Also include the time frame he was assigned to this department:
Mr. Shereck was temporarily assigned to Related Study to cover a two month medical leave of a Related
Study employee beginning January 24, 2009. He was to fill in where needed but his primary duties were
to package and send out Apprenticeship and Pharmacy Technician courses. Related Study funded half of
his salary and he was to be developing training in other curriculum areas as a part of his position. During
this time Related Study went through a database conversion to utilize “Student Manager” which was
used by the main Outreach office. Mr. Shereck elected to spend time helping with the database
conversion after the individual on medical leave came back. This conversion didn’t happen as quickly as
planned so Related Study continued to fund half of Mr. Shereck’s salary until the end of the fiscal year.

A summary of how the allegations related to the grading of PATSIM instructional material have been
investigated:
In July of 2009, Mr. Shereck reported concerns regarding the grading and recording practices taking
place in the PATSIM program which is administered by the Related Studies Department. Mr. Shereck
asserted that grading practices resulted in students being required to unfairly retake (and pay for)
exams, or in students failing. Consistent with college practices, Vice President Link requested former
Dean Dale Knutson work with Related Studies Department Head Russ Karlgaard to investigate these
concerns.

During the course of this present investigation, it is our opinion that the informal initial review that Mr.
Knutson directed and Mr. Karlgaard conducted was thorough. The concerns that Mr. Shereck raised
initially were reviewed and if a change was required it was implemented immediately. This review
included checking all answer keys, evaluating how grading was handled, and asking for answer
clarification from Pharmacy Technician Program faculty. Mr. Karlgaard also re-corrected a random
sample of tests from the last several years. Less than 5 changes were made and none had a material
difference on the final grade of any student. Following that first review, Mr. Knutson reported to Vice
President Link that no significant issues were found.

On October 25th and again on November 19th, Mr. Shereck expressed the same concerns as he earlier
raised in a letter to NDUS officials, SBHE members, the ND State Board of Pharmacy and other public
officials. Following the October 25th letter, Vice President Link requested that we re-investigate and
review the allegations related to the PATSIM Program.

As a part of our investigation, all answer keys in the PATSIM Program modules have been examined for
accuracy. Two errors on one exam were found and immediately corrected. To ensure these discovered
errors did not negatively affect student success, Related Study office and Pharmacy Tech Program staff
physically examined all student tests in question for the past 3 years. No changes in student pass/fail
standings resulted from this review.

It was brought to our attention that being held in the custody of Campus Police were certain records
(original and photo-copied Related Study documents and confidential student records) that had been
both removed from campus and subsequently returned by Mr. Shereck, and which apparently form or
contribute to the basis of his allegations. It was, therefore, determined to be a necessary part of this
investigation that we review those records also. These materials continue to be held in the possession
of NDSCS Campus Police.
Harvey Link
Page 3
January 27, 2010

On December 14, 2009, Russ Karlgaard, Barb Lacher, Sergeant Berg and Ken Kompelien examined the
above mentioned materials. We physically reviewed all materials present. Copies of student tests were
among the items in this collection of materials. Barb Lacher, representing the Pharmacy Tech Program
examined all tests at hand for grading errors or inconsistencies. None were found in this examination –
test grading was determined to be highly consistent and highly accurate.

A summary of how tests and answer keys are constructed, administered, graded and recorded. An
analysis of the accuracy of grading practices:
All module material is prepared by the Pharmacy Technician Program faculty. Instructors (usually State
Pharmacy Board Licensed Pharmacists) put together the material for the modules, including exams and
answer keys. Many of the text books provide test banks with answers and those are used whenever
possible. Given the highly technical nature of these courses it is not unheard of to find occasional errors
in these answer keys. When errors are identified, they are corrected immediately.

The department attempts to review and update course material for the modules on a 3 year cycle. As
previously mentioned, the Related Studies office currently grades 8 of the 10 PATSIM modules. The
other two are graded in the Pharmacy Technician Program office.

Coursework is graded by office staff and the grades are reviewed and entered by a different office
member to ensure the integrity of the coursework grading and to ensure that the grades are correct.
Grades are kept electronically as well as a hardcopy control sheet for redundant backup. The electronic
grades are also entered by a different individual than the staff member who does the hardcopy. After
examination of these practices, it is our conclusion that PATSIM grading practices are accurate and
appropriate.

A brief explanation of how exam failures and retakes are handled:

If a student fails a module one time they are given the opportunity to retake the exams or exam
depending on the module. The fee for the retake is less than the total cost per module. When a student
fails a module twice, all exams are sent to the Assistant Program Director of the NDSCS Pharmacy
Technician Program for review. The accuracy of grading of the exams is then reviewed. Next, the
students are called and the exams are verbally reviewed with them. They are then given the option to
take the material a third time in the online program. To date all the students that have taken this route
have passed.

A summary of what practices were used to support students that fail modules:
The Related Studies office makes every attempt to contact students if they are doing poorly prior to the
final exam of the module. Students are encouraged to contact the Pharmacy Technician office if they
have questions at any time.

If a student is not passing a module but their score is close to the passing level, the Assistant Program
Director of the NDSCS Pharmacy Technician Program routinely calls the student and verbally asks them
questions, rather than have them take the module online (which is more expensive).
Harvey Link
Page 4
January 27, 2010

When questionable patterns appear with students getting the same test item wrong, or students have
questions about why they got something in particular wrong, Related Studies staff contact the Assistant
Program Director of the NDSCS Pharmacy Technician Program. She then reviews the question/answer
and calls the student. As a result, changes in the question may be made or errors in the answer key will
be discovered.

A summary of the number of students that have attempted the PATSIM Program and have not
completed due to low scores on module exams during recent years:

Summary of PATSIM Students

1. Currently 115 students are enrolled and in various stages of completion
o 2 failed law exam—1 yet to retake , other passed on retake --- 1.7% failed on first
attempt

2. 2009 Completers = 38
o 4 failed law exam on first try = 10%
 Only 1 failed on second try—passed online

3. 2008 Completers = 31
o 2 failed the law exam on first try = 6.4%
 1 of these was close to passing so she was quizzed verbally and then passed

4. 2007 Completers = 19
o None failed

Summary of PATSIM Students that have withdrawn since 1998:

1. Of the 437 students that have enrolled in the PATSIM Program, 192 have withdrawn from
the program after enrolling – most withdrew prior to starting any coursework or completing
one module.

2. Only 5 out of the 192 have withdrawn because of failures = 2.6%. These withdrawals
occurred, one each in the years 1998, 1999, 2000, 2004 and 2009.

3. 97.3% of those that withdrew did so for reasons other than academic.
Harvey Link
Page 5
January 27, 2010

A brief explanation of records retention practices for tests and other records:
The American Society of Health System Pharmacists (ASHP) is our accrediting body. ASHP requires that
we keep samples of student tests and all records for 6 years (our current accreditation cycle). Records
of grades are electronically stored in the Related Studies office and the Pharmacy Technician office
currently houses the “hard card” copy of each student’s grades since the program started.

Coursework is graded by office staff and the grades are reviewed and entered by a different office
member to ensure the integrity of the coursework grading and to ensure that the grades are correct.
Grades are kept electronically as well as a hardcopy control sheet for redundant backup. The electronic
grades are also entered by a different individual than the staff member who does the hardcopy.

All test material is kept in the student’s hardcopy file. The tests are kept for a minimum of 6 years but
generally held for 8-10 years or more. The test results are kept “forever”, however the tests may be
shredded as storage space requires after the 6-8 year period.

Other relevant information:

The PATSIM Program is an ASHP accredited program, offered as a non-credit program or degree
granting option. To obtain a degree, the same program of study is followed that the traditional on-
campus students take.

The Pharmacy Technician field is one in which nontraditional delivery methods must sometimes be
used. Meeting the required standardized outcomes training is sometimes best done on-the-job. This is
especially important to meet the needs of nontraditional students or pharmacies in rural areas. NDSCS
has developed a program of study that responds to these needs.

The student-technician trains on-the-job while taking didactic courses from the North Dakota State
College of Science with the participating pharmacist serving as the instructor on-site. States vary in their
technician training requirements.

This program is accredited by ASHP and meets the North Dakota State Board of Pharmacy education
standards that must be met prior to registration. The program requires all students to take the
Pharmacy Technician Board Certification Exam, so they are also meeting those certification
requirements set forth by many states and practice sites.

A summary of actions undertaken and/or changes instituted as a result of the investigation:

Specific answer keys in question have been reviewed for accuracy by Pharmacy Technician Program
faculty this fall. One key that was being used in Related Studies did have one error and has since been
corrected. All other keys were found to be 100% accurate.

During this review process, one error was discovered on an exam in question. This too has been
corrected.

As a result of this investigation, it has been decided that even one failure merits further review, and
thus, in the event of any failure of a module, the exams themselves will be sent to the Pharmacy
Technician office for that review.
Harvey Link
Page 6
January 27, 2010

Summary Findings and Recommendations

Program practices occurring when a student fails an exam or module are consistent and fair.
Related Study staff and Pharmacy Technician Program faculty initiate contact with students as
soon as test or module failures occur. A thorough system of student academic support is in
place.

During this investigation one error was discovered on an exam in question. This has been
corrected. Regular test and answer key reviews are done to ensure accuracy (test materials are
on a 3 year development cycle). Tests and answer keys are developed by the pharmacists hired
to teach the classes and have been found to be accurate within the expected campus norms.

A physical review of all materials that were in Mr. Shereck’s possession but returned by him to
Campus Police was completed under the oversight of NDSCS Campus Police. These materials
included tests and related student material, which were reviewed by program faculty for
grading errors or inconsistencies. None were found.

It has been determined that in the event of any failure of a module, the exams themselves will
be sent to the Pharmacy Technician office for review.

Conclusion
After a thorough investigation of the allegations made by Mr. Shereck, it is our conclusion that the
testing and grading practices and procedures used in the PATSIM Program have been historically and
continue to be appropriate, consistent and accurate. We find no factual basis which even remotely
supports those allegations made by Mr. Shereck, relative to grading inconsistencies and grading errors
leading to student failures and unnecessary exam retakes.