Master of Arts in Liberal Studies

IESC presentation of TracDat Results

October 21, 2013
One of the first online MLS programs in the
country
Began as an exam based program
Todays MLS program has a 3 Tier design
Tier I core courses, 12 of 15 credits taken at EC
Tier II courses taken from EC or transferred
Tier III MLS697 Methodology and Thesis (taken
from EC)

Tier I courses
Tier III independent
study MLS697and Thesis
Initial results from
Thesis review below
expectations for
certain outcomes
Changes to Tier III
MLS697 and thesis
Changes to core
courses in Tier I
Results from core
courses and thesis
review
The program will be considered to have Exceeded
Expectations if 95% of the student's work submitted is
evaluated as at least Meets Expectations and 25% of this
work is evaluated as Exceeds Expectations while less than
5% of all the work submitted is evaluated as Does Not Meet
Expectations.
The program will be considered to have met the Meets
Expectations level if at least 90% of the students
completing the thesis first formal review process score at
or above the Meets Expectations level for this outcome.
Also if 10% or less of the work submitted is scored at the
Does Not Meet Expectations level.
The program will be evaluated as Does Not Meet
Expectations if fewer than 90% of the students
completing the thesis first formal review process score at
or above the Meets Expectations level for this outcome,
and/or if more than 10% of all the work submitted is
scored at the Does Not Meet Expectations level.

Critical Thought-
Students who
successfully
complete the MALS
program will be able
to:
No clear
evidence of
criterion =0
Criterion
attempted at an
elementary level
=1
Criterion is
adequately
displayed =2
Criterion is
completely,
thoroughly
displayed =3
Exceptional,
unique/superior
demonstration =4
Demonstrate their
ability to engage in
critical though
processes.
The work fails to
display the elements
of critical thought,
even at a minimal
level.
The work has weak,
sporadic display of
attempts at critical
thought at a nave, basic
level.
The work displays well
articulated critical
thought processes that
are integrated into the
thesis.
The work displays
significant unique critical
thought processes that
are a significant
component of the thesis.
The work displays
exceptional, thoroughly
examined and articulated
critical thoughts that
demonstrate a grasp of
complex and unique concepts
well beyond the scope of the
thesis requirements.
Demonstrate their
ability to engage in
analytical thought
processes.
The work fails to
demonstrate even
rudimentary
analysis.
The work has weak,
sporadic demonstration
of analytic thought at a
nave, basic level.
The work displays well
articulated analytical
thought processes that
are integrated into the
thesis.
The work displays
significant unique
analytic thought
processes that are a
significant component of
the thesis.
The work displays
exceptional, thoroughly
examined and articulated
analytical thoughts that
demonstrate a grasp of
complex and unique concepts
well beyond the scope of the
thesis requirements.
Examine a range of
issues within the
context of designing a
substantive research
project.
The work fails to
display the
examination of
appropriate issues,
even at a minimal
level. No
appropriate
research design is
provided.
The work has some
display of evidence for
the examination of
appropriate issues, at a
minimal level. A
simplistic design is
provided.
The work displays a well
articulated range of
appropriate issues
which are integrated
through the research
design into the thesis.
The work displays
significant unique critical
use of appropriate issues
that are a significant
component of the
thoroughly developed
research design for the
thesis.
The work displays
exceptional, thoroughly
examined and articulated
critically important issues
that demonstrate a grasp of a
complex and unique research
perspective and design well
beyond the scope of the
thesis requirements.
Examine a range of
issues by implementing
a substantive research
project.
The work fails to
display the
examination of
appropriate issues,
even at a minimal
level. No
implementation is
apparent.
The work has some
evidence for the
examination of
appropriate issues, at a
minimal level. Little
implementation is
attempted, minimal.
The work displays a well
articulated range of
appropriate issues
which are integrated
through the research
implementation plan
into the thesis.
The work displays
significant unique critical
use of appropriate issues
that are a significant
component of the
research implementation
plan for the thesis.
The work displays
exceptional, thoroughly
examined and articulated
critically important ideas that
demonstrate a grasp of
complex and unique research
perspective and is
implemented well beyond the
scope of the thesis
requirements.
Thesis
Review
RESULTS

Program
scores
Outcome #1.
integration
&
application
Outcome #
2.
hypothesis
Outcome
#3. issues
& context
Outcome #
4. cultural
differences
Outcome
#5. critical/
analytical
thought
Exceeds 17% 0 39% 28% 23%
Meets 83% 44.4% 61% 66.6% 77%
Does not
meet
0 55.6% 0 5.5% 0
Program Performance
Report
Meets Does Not
Meet
Expectations
Exceeds Exceeds Meets
Step I
Thesis writing stage is organized as a course
MLS697 from independent study to a course
(premiered 2012)
Instructors, reviewers and thesis mentors were
all given training on using the rubrics
FPD held discussions with instructors and mentors
on how to assist students with demonstrating
mastery of all outcomes within their thesis work
Step II
Core courses revised (premiered 2012)
Increased focus on program level learning
outcomes
Added rubrics to evaluate SLO
FPD worked with instructors to understand and
appreciate the importance of these changes
Instructor training on using the rubrics
The program will be considered to have successfully met the
Exceeds Expectations level if 85% or more of the students
completing the course receive a score of Meets Expectations
for this outcome and 15% or more of these are at the achieve
Exceeds Expectations level. 15% or less of all the student work
submitted will score at the Does Not Meet Expectations level.
The program will be considered to have met the Meets
Expectations level if at least 80% of the students completing
the course score at or above the Meets Expectations level for
this outcome, and 20% or less of all the student work
submitted is scored as Does Not Meet Expectations.
The program will be evaluated as Does Not Meet
Expectations if fewer than 80% of the students completing
the course score at or above the Meets Expectations level for
this outcome and more than 20% of the student work
submitted is scored as Does Not Meet Expectations.

Results
Core courses
Outcome #1.
(integration &
application)
Outcome # 2.
(hypothesis)
Outcome #3.
(issues &
context)
Outcome # 4.
(cultural
differences)
Outcome #5.
(critical/analytic
al thought)
N= 25 72 17 44 59
Meets+ 96% 81.9% 94.1% 54.4% 74.6%
Program
Performance
Exceeds Does Not Meet
(18.1% scored
as DNM)
Meets Does Not Meet
(45.4% scored
as DNM)
Does Not Meet
(25.4% scored
as DNM)
Thesis Review
N= 27 27 27 32 33
Meets+ 100% 96.3% 92.6% 90.6% 97.0%
Program
Performance
Meets Meets Meets Meets Meets
Continue to work with instructors on
improving the reliability and consistency
student scores in the core courses
Continue working with students to improve
their demonstration mastery of the learning
outcomes through out the program
Focus on SLO # 4 (cultural differences) at the
next SLA-FAC
Continue to collect data
Questions?