PROFILES OF

BLENDED MOOCS AT
THE UNIVERSITY
SYSTEM OF
MARYLAND
Rebecca Griffiths
Ithaka S+R
February 23, 2015

AGENDA

1. Background
2. Lessons and testimonials from

selected tests
3. Summary of findings and
implications
4. Q&A

Poll
I have :
a) Taught a MOOC
b) Taught a blended MOOC with students enrolled at my institution
c) Worked on a MOOC production team
d) I have no experience with MOOCs
e) Other

Background
Collaboration with University
System of Maryland and
Coursera to examine benefits
and challenges of embedding
existing MOOCs in campusbased courses
Supported by Bill & Melinda
Gates Foundation

Study included:
4 side-by-side comparisons

10 case studies

Can instructors use MOOCs

RESEARCH
QUESTIONS

created by other faculty members

to address instructional goals?
What is the blended MOOC

experience like for instructors and

students?
What implementation challenges

did instructors encounter, and how

might these be overcome?

Presented study at seven

institutions, with support from

USM chancellors office

RECRUITMENT
PROCESS

Solicited statements of interest

from faculty
Selected 16 out of around 30

submissions (ended up with 14)

Looked for:
Clear description of educational
goals or problem to be solved
Substantial use of MOOC materials
Feasibility

Questions?

PROFILE #1:
PRECALCULUS

Research university setting

Conducted pilot during summer

Description

followed by implementation in two

sections of ~35 students each
during the fall term
Compared outcomes for students

in traditionally taught sections

with those in hybrid sections

Goals
Replace lecture format with more active

learning

Goals &
Design

Design
Students met once per week for 2 hours
Students were assigned to watch videos and

do quizzes and homework in the MOOC

Class time used to review video material,

answer student questions, practice problem

solving, take quiz at end of each session.
Supported by undergraduate teaching

assistant

What worked & didnt work

Worked well:

Room for Improvement:

Student outcomes better than

other sections

Students liked blended MOOC

sections less

Videos provided alternate ways

of explaining concepts; students
could watch them multiple times
(or not at all)

Some students found it confusing to

have multiple instructors

Having TA was priceless

Quizzes in MOOC were good,
but needed more

TA thought students were less

engaged with online content
Platform did not allow monitoring of
individual progress

Student Outcomes Comparison

1.00
0.80

4.00

0.79
0.78

0.74
0.67

3.00

2.42
2.24

0.60
Hybrid
Control

0.40

Hybrid
Control

2.00
1.00

0.20
-

Pass rates

Final exam***

Course grade

Notes: *** p<0.01, ** p<0.05, * p<0.1; Robust standard errors appear in parentheses. All models include controls for SAT
math and verbal scores, cumulative GPA at the beginning of the semester, race/ethnicity, gender, parents' income, and age.

Precalculus Average Survey Responses

5
4.4

4.5
4

Control

Treatment

3.7
3.4

3.5

3.6

3.4
3.1

3.0

2.6

2.5

2.6

2.4

2
1.5
1
0.5
0
Weekly hours
outside class

Increased interest
in pre-calc

Amount learned

Control N = 56 Treatment N = 36

Course rating

Course difficulty

Video clip:
Tatyana 7:20-8:25
https://vimeo.com/album/31
23302/video/110863058

Questions?

PROFILE #2:
INTRODUCTION TO
PROGRAMMING

Goals
Explore alternative to YouTube videos

(Bucky) and commercial products

Goals &
Design

Design
Students met 80-100 minutes per week
Students assigned to watch videos and do

quizzes in the MOOC

Used class time for practice assignments
In 1st iteration MOOC module was

delivered during the first half of the

semester; reversed in 2nd iteration

Student Survey Responses

8
7.1

7
6

Control

5.3

Round 2

4.7
4.0

3.9

4
3

Treatment

3.4

3.4

3.6

3.3

3.6

3.6

2.8
2.5
2.1

2.0

2
1
0
Weekly hours outside Increased interest in
class
comp sci

Amount learned

Course rating

Note: N for control = 50, N for treatment = 74, N for Round 2 = 9

Course difficulty

Student Outcomes Comparison

1.00
0.80

4.00
0.660.73

0.680.71

3.00

0.60
Hybrid
Control

0.40

2.00

1.73 1.98

Hybrid
Control

1.00
0.20
-

Pass rates

Final exam

Course grade

Notes: *** p<0.01, ** p<0.05, * p<0.1; Robust standard errors appear in parentheses. All models include controls for SAT
math and verbal scores, cumulative GPA at the beginning of the semester, race/ethnicity, gender, parents' income, and age.

What worked & didnt work

Worked well:

Room for Improvement:

Coherency of MOOC compared

to YouTube videos

First iteration started course

with MOOC, students not
adequately prepared

Access to professor 24/7

Saved time for instructors

Alternative to textbooks

Desire for more dynamic video

presentation
Instructors not well prepared
for first iteration
Technology integration

Video clip: Students

0:18-1:42
https://vimeo.com/album/31
23302/video/116671864

Questions?

PROFILE #3:
MODERN
POETRY

Goals
Provide model for intellectual discourse

Goals &
Design

Enhance use of technology

Shared learning experience

Design
Students met 3 times per week for 50 minutes
Students enrolled in live version of MOOC
Students had to do assignments in MOOC

and contribute to discussion boards

Created videos of student-led seminar

discussions

Student Survey Responses - Rating on a 5 point Scale

5
5
4

4.0

3.8

3.7

3.5

3.4

3.3

2.9

3
3

3.1

2.0

2
2
1
1
0
Amount
learned

Course
difficulty

Lecture
videos

N=6

Online
Online
discussion problem
forums
sets

Online
quizzes

Learned a Materials
great deal
were
from the engaging
MOOC

High
quality

What worked / didnt work

Worked well:

Room for Improvement:

Modeling of class discussion

Students started to feel that a

poem was done after
watching seminar videos

Eliminating need to cover content

in class

Creating videos of student seminar

discussion modeled after MOOC
Exposure to diverse student body

Discussion boards could be

overwhelming

Video clip: Amy

1:14-2:13
https://vimeo.com/album/
3123302/video/11086304
7

Questions?

PROFILE #4: ORAL

COMMUNICATIONS

Goals
Increase emphasis on critical thinking and

communication skills

Goals &
Design

Exposure to different teaching style

Design
Students met 80 minutes per week
Students were assigned to watch videos and

complete a quiz before class

4 sections used the MOOC

Average Survey Responses for Oral Communications

6

5.7
Control

5
4.3
3.7

3.9

Treatment

3.9 4.0

4.2

4.4

2.4

2.2

2
1
0
Weekly hours
outside class

Increased interest Amount learned

in comm

Course rating

Control N = 80 Treatment N=93

Course difficulty

Student Outcomes Comparison

1.00

0.93
0.89

4.00

0.890.89

3.27

0.80

3.11

3.00

Hybrid
Control

0.60

Hybrid
Control

2.00

0.40
1.00
0.20
-

Pass rates

Final exam

Course grade

Notes: *** p<0.01, ** p<0.05, * p<0.1; Robust standard errors appear in parentheses. All models include controls for SAT
math and verbal scores, cumulative GPA at the beginning of the semester, race/ethnicity, gender, parents' income, and age.

What worked / didnt work

Worked well:

Room for Improvement:

Hybrid format

Technical issues with

integration

Higher level instruction than

textbook materials
Coherency of online materials works
well for first generation students
Exposure/diversity
Could track whether students were
watching videos (unlike reading)

Video clip: Pam

1:46-2:31
https://vimeo.com/album/31
23302/video/110863048

Questions?

SUMMARY OF
FINDINGS &
IMPLICATIONS

Summary of benefits
Exposure to different perspectives, models of intellectual discourse, teaching

styles
Gain flexibility in use of face-to-face time (without creating online materials
from scratch)
Augment course with topics outside instructors own expertise
Replace textbook; more accessible & engaging

Professional development for instructors

36

Summary of challenges
Fit between online materials and program needs
Degree of difficulty too high or low
Intellectual property concerns
Concerns about student engagement with online materials
Technical issues, especially integration with LMS
Very time consuming to redesign courses with MOOCs

Some implications
Potential for use of external online course materials to enhance courses and achieve
variety of instructional goals.
Technology platforms and content will continue to improve. Question is whether they

strive to serve institutions or go directly to students. Will depend in part on where they
believe there is demand.
Overcoming internal challenges requires strategic plan to incentivize, reward, and

support innovation.

38

Questions?

ITHAKA uses digital technologies to preserve the scholarly record and to

advance research and teaching in sustainable ways.

JSTOR helps people discover, use,

and build upon a wide range of
content through a powerful research
and teaching platform, and preserves
this content for future generations.

ITHAKA S+R helps academic,

cultural, and publishing
communities thrive in the digital
environment.

PORTICO to help the academic

community use digital technologies
to preserve the scholarly record and
to advance research and teaching
sustainable ways.