Houston, we have a problem:
There’s a leak in the CS1 affective oxygen tank
Dawn McKinney and Leo F. Denton
School of Computer and Information Sciences
University of South Alabama
Mobile, Alabama 36688
(251) 460-6390
{dmckinney, ldenton}@usouthal.edu
ABSTRACT
The affective domain can be used to support the internalization of
cognitive content and foster the development of curriculum and
industry-related interests, attitudes, values, and practices. This
study investigated correlations between affective factors and
course grade. Interest, perceived competence, effort, lack of
pressure, and value correlated significantly with CS1 course
grades. Moreover, this study investigated the levels of these
factors over the course of CS1. Almost all of the levels of these
significant factors decreased significantly during the CS1 course
as measured by pretests and posttests. Results of this study
further indicated that the use of specific affective objectives and
instructional strategies lessened these decreases.
Categories and Subject Descriptors
K.3.2 [Computer and Information Science Education]: Computer
Science Education
General Terms
Design, Experimentation, Management, Measurement
Keywords
Affective objectives, affective assessment, interest, competence,
effort, pressure, value, belonging, motivation, Intrinsic Motivation
Inventory, Institutional Integration scale, student achievement,
retention, CS1, CS education research
1. INTRODUCTION
Elements of the affective domain include interests, attitudes, and
values. These affective elements support the internalization of
cognitive content. Like Bloom’s cognitive domain [4],
Krathwohl’s affective domain consists of levels of achievement.
These levels are: 1.0 receiving, 2.0 responding, 3.0 valuing, 4.0
organization, and 5.0 characterization [13]. Each level represents
a higher level of learner internalization. Professional
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
SIGCSE’04, March 3–7, 2004, Norfolk, Virginia, USA.
Copyright 2004 ACM 1-58113-798-2/04/0003…$5.00
organizations such as the National Association of Colleges and
Employers (NACE) have identified characteristics needed in
college graduates. Many of these characteristics are affective.
NACE’s top ten list is: communication skills (verbal and written),
honesty/integrity, teamwork skills, interpersonal skills,
motivation/initiative, strong work ethic, analytical skills,
flexibility/adaptability, computer skills, and self-confidence [1].
The effects of student affect extend in and beyond the classroom.
The cognitive and affective domains are interrelated. There is no
clear division between these two educational domains; growth in
one domain tends to contribute to growth in the other [13, 15].
Low levels of interest, relevance, and the sense of belonging, are
among the top reasons for early departure (i.e. first year) from
colleges and universities [11, 14, 17, 18, 20]. Student persistence
is particularly important in the SMET (Science, Mathematics,
Engineering, and Technology) disciplines which are experiencing
shortages in college graduates especially in underrepresented
groups [11, 19]. These disciplines are not only losing low-
achieving students but also considerable numbers of high quality,
talented students [11, 14]. Moreover, even when students do
persist and become employed, many have not internalized needed
professional practices and skills [6, 7, 12].
In order to determine whether students possess NACE
characteristics and value professional practices, affective
assessment must be conducted. In this context, affective
assessment can contribute to fuller understanding of course
outcomes, can lead to the improvement of courses, and can assist
in the fulfillment of accreditation mandates from organizations
such as ABET-CAC. The American Association for Higher
Education (AAHE) supports this attention to affective assessment
in it’s Nine Principles of Good Practice for Assessing Student
Learning where the point is made that learning “involves not only
knowledge and abilities but values, attitudes, and habits of mind
that affect both academic success and performance beyond the
classroom” [2].
In CIS disciplines, it is generally reported that course-completion
rates in CS1 are low. Often CS1 is viewed as a “weed-out”
course. CIS classrooms are characterized by a “defensive
climate” [3] not conducive to student achievement. Studies also
indicate that high quality students are also leaving [11, 14, 17, 18,
20]. These studies are by and large qualitative in nature. In our
study, we have sought to provide quantitative confirmation of
these findings.
Our results indicate that interest, perceived competence, effort,
pressure, and value are significantly correlated with course grade
in CS1. Furthermore, using pretests and posttests, our results
show significant drops in these affective factors during the
semester in students who completed the course. In other words,
there is a significant leak in the CS1 affective oxygen tank.
Houston, we have a problem.
2. PRIOR WORK
In 1994, our school established a Bloom-based cognitive
framework [10]. This work generated materials which were
transferable to other instructors and ensured that a common set of
objectives were being met. Students reported that they perceived
this approach to be beneficial. Yet despite continuing efforts,
course-completion rates usually remained in the 30 to 50% range.
During 2001, two CS1 instructors, the authors, began to address
these low course-completion rates by enhancing CS1’s cognitive
framework with two affective initiatives [8]: (1) the discussion
approach, and (2) the self-reflection approach. The discussion
approach involved the students’ in-class discussions about the
students’ cognitive-affective potential. The self-reflection
approach used a tool called the BAM chart (Bloom-Affective
Maslow) which was designed to promote self-regulated learning.
When piloted these efforts resulted in significantly higher course-
completion rates. However, without clearly defined affective
objectives, the goals of these initiatives were unclear and difficult
to assess. Moreover, these initiatives did not transfer to other
instructors who were uncomfortable with our methodologies.
In 2002, the authors introduced specific affective objectives to the
CS1 course. We conducted affective assessment using validated
instruments to determine which affective factors contributed to
higher achievement as measured through course-completion rates
and grades. This effort [9] identified perceived competence,
effort, and interest/enjoyment as factors which were significantly
correlated with course grade. Student belonging with peers (i.e.
student-peer belonging), student belonging with faculty (i.e.
student-faculty belonging), and lack of pressure were found to be
significantly correlated with grade factors for specific ethnic
groups.
3. METHODOLOGY
The goals of our study were threefold: (1) to further identify
specific affective factors which promote student achievement as
measured by course grade, (2) to determine significant differences
in the levels of these factors at the beginning and end of CS1, and
(3) to identify any significant differences between the affective
levels at the end of the semester between CS1 sections using our
affective instructional methodology and those sections which did
not use a specific affective methodology. To accomplish these
goals, we administered pretests and posttests versions of the
Intrinsic Motivation Inventory (IMI) [16] and pretests and
posttests versions of the Institutional Integration scale [17] to all
CS1 sections during Spring 2003. There were four course
sections and three instructors. Each instructor had an individual
teaching style but the same cognitive content was covered in each
section in a synchronous manner.
3.1 Assessment Instruments
The Intrinsic Motivation Inventory (IMI) is a validated instrument
which has been used in studies for measuring intrinsic motivation,
internalization, and self-regulation. The scales we used measured
interest-enjoyment, perceived competence, effort, value-
usefulness, and felt pressure-tension regarding a particular
activity. Students rated items based on a seven-point scale
ranging from not at all true to somewhat true to very true. The
IMI was designed to be tailorable to the specific activity which is
being measured [16] and our IMI instrument was tailored for CS1.
Figure 1 shows sample questions from this IMI.
I think problem-solving strategies are important skills.
I think I will enjoy writing programs for business applications.
I think I will be pretty good at programming.
Figure 1: Items from our IMI Pretest
The Institutional Integration scale is a validated instrument
designed to measure the degree of peer-group interaction and
student-faculty interaction. A Likert-scale was used: strongly
disagree, disagree, don’t know, agree, and strongly agree. These
factors were used to measure the students’ sense of belonging
with other students and faculty. Samples of Institutional
Integration questions for CIS are shown in Figure 2.
It has been difficult for me to meet and make friends with
other students in CIS.
My non-classroom interactions with at least one member of
the CIS faculty has had a positive influence on my career goals
and aspirations.
My interpersonal relationships with other students in CIS
have had a positive influence on my personal growth, attitudes,
and values.
My non-classroom interactions with at least one member of
the CIS faculty has had a positive influence on my intellectual
growth and interest in ideas.
Figure 2: Sample items from the Institutional Integration
Scale
3.2 Affective Instructional Methodology
Although the specific instructional methodology used in the
sections using affective objectives is not the focus of this paper,
we present a brief description of that methodology here. Two of
the four sections in this study used affective objectives (see Figure
3 for examples). These sections were characterized by active and
cooperative learning activities. Team work was emphasized.
These sections implemented the discussion approach and self-
reflection approach described in the prior work [8, 9]. The laptop statistically significant. The student-peer belonging factor was the
policy of our School of CIS (i.e. students must bring personal lone factor which showed an improvement. This factor is largely
laptops to each CS1 class) provided classroom opportunities for under the students’ control and is supported by our university’s
hands-on programming activities during class times. The active Supplemental Instruction program, the student ACM chapter, and
and cooperative components of the course facilitated affective departmental activities. This improvement was statistically
assessment. Course adjustments to enhance student achievement significant.
are made based upon in-class affective assessments, cognitive
assessment, and student feedback.
Table 1: Correlations of Factors with Course Grade
Pearson Correlations
1.0 Receiving
Factors with Course Grade N p-value
Students become aware that testing is a part of the software
development life cycle during the initial programming labs. Pretest Value 0.215* 86 0.047
2.0 Responding Posttest Interest 0.640** 56 0.000
Students use coding standards as taught in the lectures with Posttest Competence 0.588* 56 0.001
minimal prompting in the lab. Posttest Effort 0.445** 56 0.001
3.0 Valuing
Posttest Lack of Pressure 0.614** 56 0.004
Students consistently show strong work ethic, cooperation, and
Posttest Value 0.614 56 0.000
initiative while working in groups.
4.0 Organization ** Significant at the 0.01 level (two-tailed)

Students discuss a wide variety of useful personal initiatives * Significant at the 0.05 level (two-tailed)
indicative of a growing commitment to life-long learning.
Figure 3: Examples of Affective Objectives in CS1 Table 2: Pretest/Posttest Paired-Samples Comparisons
Degrees Significance
Pretest/Posttest of (two-tailed
4. RESULTS Factors t-value Freedom p-value)
Several analyses were conducted on the pretest and posttest scores
Interest -2.811** 52 0.007
and course grades to investigate correlations between affective
factors and grade. First, the pretests and posttests scores for each Competence -4.106** 52 0.000
factor were correlated with the course grade. Second, paired- Effort -3.706** 52 0.001
samples t tests were conducted to determine whether there were
significant differences between the pretest and posttest scores for Lack of Pressure -2.721** 52 0.009
each affective factor. Third, independent-samples t tests were Value -4.124** 52 0.000
conducted to determine if there were any significant differences in
the posttest affective factors between the two sections using Faculty Belonging -1.004 52 0.320
affective objectives and the two sections not using affective Peer Belonging 4.592** 52 0.000
objectives.
** Significant at the 0.01 level (two-tailed)
* Significant at the 0.05 level (two-tailed)
4.1 Course grade and affective factors
Significant correlations between course grades and the affective
factors measured in the pretest and posttest for all students in the
sample are shown in Table 1. All IMI posttest factors showed a 4.3 Lessening the leak
significant correlation with course grade. The only pretest factor The sections using specific affective objectives showed higher
to correlate significantly with course grade was perceived value. posttest scores for each affective factor when compared with
The affective factors measured in the Institutional Integration sections not using affective objectives. These differences are
scale, student-peer belonging and student-faculty belonging, did shown in Figure 4. To investigate these differences, we
not indicate any significant correlations. conducted an independent-samples t test to see if any of these
individual differences were significant. The sections using our
4.2 Comparison of pretests and posttests affective methodology showed significantly higher posttest scores
All IMI affective factors showed negative, significant shifts from for effort, t(51) = 6.099, p = .017, and student-peer belonging
pretest to posttest in the paired samples t tests. These significant t(51) = 4.377, p = .041.
differences between pretest scores and posttest scores for all It is important to note that all sections showed a decrease in all
students in the sample demonstrate the affective leak in CS1 at factors except student-peer belonging from the beginning to the
our university. The results of these t tests are shown in Table 2. end of the semester. Figure 5 shows these decreases averaged
The student-faculty belonging factor, from the Institutional across sections.
Integration scale, also dropped but the decrease was not
 significant. Additionally, results indicated that the use of specific
Comparison of Affective and Nonaffective Sections Posttest
affective objectives and affective instructional strategies lessened
Scores
these decreases. Some of these lessening effects were significant.
1.00 Affective objectives, like cognitive objectives, provide specific
0.80 disciplined-based goals which can be used to support and measure
student achievement. These affective goals, like their cognitive
0.60 counterparts, can be pursued by different instructors using their
0.40 own instructional strategies. This is a salient advantage of
Nonaffective
Affective
objectives-based education. Anticipated benefits of a broader
0.20
implementation of this integrated cognitive-affective approach are
0.00 higher cognitive achievement, higher course-completion rates,
higher retention rates, more diverse student representation, higher
ue

ng
rt
st

re
e

in
al
nc

fo
re

su

gi
ng
V
Ef
te
te

on
es

lo
In

pe

student enthusiasm, and more complete fulfillment of industry

Pr

el
Be
m

B
of
Co

lt y

er
ck

Pe
cu

demands for qualified and motivated CIS professionals.

La

Fa

Figure 4: Comparison of posttest scores in affective and 7. ACKNOWLEDGMENTS

nonaffective CS1 sections This work was enhanced by the contributions and cooperation of
our colleagues, Debra Chapman and Richard Johnson; and by the
support and advice from to our CIS Coordinator, Michael Doran,
Decreases in Affective Factors in First Semester of CS1 and our Dean, David Feinstein.

1.00
8. REFERENCES
0.80 [1] ACM/IEEE Computing Curricula 2001, Computer Science
Volume, Chapter 10: Professional Practice. Available at
0.60 http://www.acm.org/sigcse/cc2001/cs-professional-
0.40
practice.html
Pretest
Posttest [2] Astin, Alexander, et.al. W. Nine Principles of Good Practice
0.20
for Assessing Student Learning. American Association for
0.00 Higher Education, 2003. Available at
http://www.aahe.org/assessment/principl.htm
ue

ng
st

rt

re
e

in
al
nc

fo
re

su

gi
ng
V
Ef
te
te

on
es

lo
In

pe

Pr

el
Be
m

[3] Barker, Lecia Jane, Kathy Garvin-Doxas, and Michele

of
Co

lt y

er
ck

Pe
cu
La

Fa

Jackson. “Defensive Climate in the Computer Science

Classroom.” Proceedings of the 33rd SISCSE Technical
Figure 5: The leak in the affective oxygen tank Symposium On Computer Science Education, March 2002,
Cincinnati OH, pp. 43-47.
[4] Bloom, B. S., et al. Taxonomy of Educational Objectives:
5. FUTURE WORK The Classification of Educational Goals. Handbook I: The
The affective framework, which we are integrating into our Cognitive Domain. McKay Press, 1956.
school’s cognitive framework, is still in its beginning phases of [5] Bloom, B. S., G. F. Madaus, and J. T. Hastings. Evaluation
development. An important area for future development is the to Improve Learning. McGraw-Hill, Inc., 1981.
systematic incorporation of professional practices drawn from the
priorities of ACM, ABET, NACE, NSF, and other appropriate [6] Dahlbom, Bo, and Lars Mathiassen. The Future of Our
organizations. We have also begun to extend this work to other Profession. Communications of the ACM, Vol. 40, no. 6,
courses in our CIS curriculum. June, 1997.
[7] Denning, Peter J. A New Social Contract for Research.
6. CONCLUSIONS Communications of the ACM, vol. 40, no. 2, February, 1997.
The affective domain supports the internalization of cognitive [8] Denton, Leo F., Michael V. Doran, and Dawn McKinney.
concepts and fosters the development of curriculum and industry- Integrated Use of Bloom and Maslow for Instructional
related interests, attitudes, values, and practices. This study Success in Technical and Scientific Fields, In the
provides quantitative results that confirm that interest, perceived Proceedings of the 2002 American Society for Engineering
competence, effort, lack of pressure, and value are significantly Education Annual Conference & Exposition, Montreal,
correlated with CS1 course grades. Because these factors are Canada (2002). Available at
significantly correlated with course grade, attention should be http://www.asee.org/conferences/caps/document/2002-
paid to their development or lack of development during CS1. 675_Paper.pdf or at
This study provides quantitative data showing that the levels of http://www.cis.usouthal.edu/~mckinney/
these significant affective factors decrease during CS1. These
decreases were consistent across sections and statistically
[9] Denton, Leo F., Dawn McKinney, and Michael V. Doran.
Promoting Student Achievement With Integrated Affective
Objectives, In the Proceedings of the 2003 American
Society for Engineering Education Annual Conference &
Exposition, Nashville, Tennessee, USA (2003). Available at
http://www.asee.org/conferences/caps/document/2003-
2391_Final.pdf or at
http://www.cis.usouthal.edu/~mckinney/ASEE3530.htm
[10] Doran, M. V. and D. D. Langan. A Cognitive-Based
Approach to Introductory Computer Science Courses:
Lessons Learned. In the Proceedings of the 26th SISCSE
Technical Symposium On Computer Science Education,
March 1995, Nashville, TN, pp. 218-222.
[11] Grandy, J. (1998). Persistence in science of high-ability
minority students. Journal of Higher Education, 69(6), 589-
620.
[12] Hoffman, Thomas. Preparing Generation Z. Computerworld.
Vol. 37, Issue 34, August 2003, p. 41.
[13] Krathwohl, D. R., B. S. Bloom, and B. B. Masia. Taxonomy
of Educational Objectives: The Classification of Educational
Goals. Handbook II: Affective Domain, David McKay
Company, Inc., 1964.
[14] Light, Richard. Making the Most of College. Harvard
University Press, 2001.
[15] Martin, B. L., & Briggs, L. J. (1986). The Affective and
cognitive domains: Integration for instruction and research.
Inglewood Cliffs, NJ: Educational Technology Publications.
[16] McAuley, E., Duncan, T., & Tammen, V.V. “Psychometric
properties of the Intrinsic Motivation Inventory in a
competitive sport setting: A confirmatory factor analysis.”
Research Quarterly for Exercise and Sport, 60, 48-58, 1987.
Available at
http://www.psych.rochester.edu/SDT/measures/intrins.html
[17] Pascarella, E. T., and P. T. Terenzini. “Predicting freshman
persistence and voluntary dropout decisions from a
theoretical model.” Journal of Higher Education, 51:60-75,
January/February 1980.
[18] Seymour, Elaine, and Nancy M. Hewitt. Talking About
Leaving: Why Undergraduates Leave the Sciences.
Westview Press, 1997.
[19] Smith, Theresa Y. Center for Institutional Data Exchange
and Analysis, The University of Oklahoma, Volume 2,
Number 2, April 2000. Available at
http://ehrweb.aaas.org/mge/Archives/5/smet.html
[20] Tinto, Vincent. Leaving College: Rethinking the Causes and
Cures of Student Attrition. Second Edition, The University of
Chicago Press, 1993.