Validating a Writing Test

for Graduate Admissions

Donald E. Powers
Mary E. Fowles
Kelli Boyles

GRE Board Report No. 93-26bP

November 1996

This report presents the findings of a

research project funded by and carried
out under the auspices of the Graduate
Record Examinations Board.

Educational Testing Service, Princeton, NJ 08541

********************

Researchers are encouraged to express freely their professional

judgment. Therefore, points of view or opinions stated in Graduate
Record Examinations Board Reports do not necessarily represent official
Graduate Record Examinations Board position or policy.
********************

The Graduate Record Examinations Board and Educational Testing Service are
dedicated to the principle of equal opportumty, and their programs,
services, and employment policies are guided by that principle.
EDUCATIGNAL TESTING SERVICE, ETS, the ETS logo, GMDUATE RECORD EXAMINATIONS,
and GRE are registered trademarks of Educational Testing Service.
Copyright 0 1996 by Educational Testing Service. All rights reserved.

Abstract
Proficiency in writing is important to success in many fields of graduate study.
Measuring this construct, however, involves facing difficult challenges and criticisms from various
perspectives. Among them is a concern that the writing skill that students demonstrate in an
assessment situation may not relate fully enough to the writing competencies used in other
contexts. This study was designed to reveal some of the facets of writing proficiency that
different kinds of timed writing exercises might reflect. The approach taken was to examine
how performance on these exercises correlates with performance on the following nontest
indicators of writing skill: (a) self-assessments of writing skills, (b) self-reports of writing-related
accomplishments and relevant undergraduate grades, and (c) independently graded samples of
writing, such as course-related writing assignments and personal statements written in
conjunction with application to graduate school. The writing exercises investigated in this study
were of several kinds: composing a relatively traditional expository essay, writing somewhat
briefer responses involving either explaining the meaning of a quotation or presenting ideas in a
specified format, and revising a poorly written composition. These exercises were selected by
university faculty members serving on a Gm Writing Committee.
A geographically diverse sample of 470 undergraduate students, all of whom had taken
the GRE General Test (and who represented the GF2E test-taking population reasonably well),
were recruited to participate in the study. Subgroups of the study sample tried various
combinations of the writing tasks.
The assorted nontest indicators of writing proficiency correlated with one another in
differing degrees. None, however, appeared to be (statistically speaking) completely redundant
with the others, suggesting that, in all likelihood, each reflected a somewhat different aspect of
writing skill. The alternative writing exercises, in turn, exhibited distinctly different patterns of
correlations with the individual indicators and with several composite indices based on logically
determined subsets of indicators. By virtue of these distinct correlational patterns, the exercises
appeared to have differential utility for a GRE writing test. A traditional issue-based expositoty
essay appeared to be the most promising candidate. An exercise involving the revision of a
composition also appeared to have potential. Two other experimental tasks, however, did not,
from their relationships with the writing indicators, seem as defensible.
Limitations of the study are discussed -- principally the shortcomings of the nontest
indicators as criteria for validating the assessment of writing proficiency. Despite the
limitations, the results of the study are believed to have provided a reasonably good beginning to
accumulating the kind of evidence required by professional standards for educational and
psychological testing. Additional information wiU be required, however, about the validity of the
inferences to be drawn and the decisions to be made about the examinees who will eventually
take the GRE writing test.

Acknowledgments
The authors are grateful to the following people who contributed

to the study reported

here:
Phyllis Murphy, Patricia Bakr, Elizabeth Finotti, Dorice Hartman, Debera Hill-Walker,
Susan Krupka, Rose Marie Meath, and Robert Wilmot from ETS Field Service Offices,
who helped arrange for and carry out data collection
Robin Pergament and Lou Woodruff, who helped us recruit additional subjects and
arrange for data collection
Kathy Carbery and Diane Thomas who matched files for us and retrieved GRE records
Elana Broth for consulting on project design
Seth Weiner for help in developing scoring methods
Faculty members John Collins, Marilyn Collins, Noreen Duncan, Abigail Harris, Ron
Kostar, RoseAnn Morgan, and Marti Pahnere for scoring the essays and the writing
samples
Peggy Redman for preparing the data for analyses
Laura Jerry for analyzing the data
Ruth Yoder for carrying out numerous valuable administrative

tasks

Hunter Breland, Brent Bridgeman, and Ann Willard for thoughtful reviews of an earlier
draft
Members of both the GRE Research Committee
their input to the study.

and the GRE Writing Committee

for

Validating a Writing Test for Graduate Admissions

The ability to communicate, that is, to share ones ideas, knowledge, and insights with
others (both orally and in writing), (Enright & Gitomer, 1989, p. 10) is, according to eminent
psychologists and distinguished scholars, a trait that is critical to successful performance in
graduate school (Enright & Gitomer, 1989; Reilly, 1976), and undoubtedly in many professions
as well. Communication skills are sometimes assessed quite informally, for example, during
unstructured interviews or by casually skimming samples of applicants writing. Often, however,
more rigorous procedures are called for.
Strategies for testing writing skills have traditionally been of two general types: (1)
indirect assessment, for which examinees typically answer multiple-choice questions about the
conventions of writing, and (2) direct assessment, for which examinees are required to produce
actual samples of writing as a demonstration of their skills. The merits of these approaches
have been discussed in detail elsewhere (e.g., Breland & Gaynor, 1979; Cooper, 1984).
Regardless of the particular strategy taken, however, measuring writing proficiency in any
reasonably comprehensive manner poses a significant challenge for large-scale testing programs
such as the Graduate Record Examinations (GRE). As cognitive psychologists have cautioned,
... the writing process is not a neat, additive sequence of stages (Flower & Hayes, 1980, p. 44).
It is not, therefore, amenable to easy capture, especially within the constraints of a typically
short, one-time testing session.
Opinions differ widely about the feasibility of testing writing skills. Some (e.g., Wiggins,
1993) believe that, at least within very short time limits, the task is not possible at all:
Whatever assessors are testing in a twenty-minute
most certainly is m the ability to write (p. 235).

essay [such as found in some tests] ...

Others, who have advocated the use of collections, or portfolios, of students writing for both
instruction and evaluation, have disputed the view that writing is a testable skill at all or that
...we can expect students or anyone else to write on demand (Hout, 1994, p. 3).
The use of general all-purpose writing measures is also often grounds for criticism.
Some faculty are often more interested in assessing discipline-specific skills than generic
competencies. Moreover, the expected style of writing may differ among specialties according to
the context and purpose for writing. Creative writing, for example, most assuredly invites the
use of different skills than does a step-by-step demonstration of mathematical proofs. These
observations were reinforced by the fmdings of a 1992 GRE survey of graduate deans and
faculty, who often noted the discipline-specific nature of the writing required of graduate
students.
Perhaps the best that can be hoped for as a result of giving a relatively brief
standardized measure is a reasonably clear snapshot of an examinees skills, rather than a fulllength video. When properly taken, however, even a simple photograph can often be
remarkably revealing of its subject, provided that the photographers aim is true and that
extraneous influences do not unduly distort the image. In fact, faculty advisoty committees and
others involved in the development of writing assessments have found that many important
aspects of writing skill can be captured with a test that is properly designed and administered.

2
O

t
c
e
j T
bm af j o r o h
o t
se t u d yr hoe
hp wo e t efrr
t esa re fovdt o ef se
oama h c l e e
i ur
fb t w
e
ra
i at
a c hr d
ae ws n ti t gm ni e f e r ad sea O f a tlyc p e uwp ct r t o r. a a
tl
y
gsg n ub oa sn i i ds uort i e t heepr dehi nt sa p ia c
ro g t c
s o ao
i w h i c h ttmg a u g e t i
po
i s
f t ua ,
t o i s e nes fc fe ar om m
r s r eeo woe t r ps m
ra fo ft uTi cs i fheuo n a c piy o . pc l sl
i
e
e c
nt
emoc
ge o
ut p rn n
s e
r ri g
r s eaa
ed
au H a d tad eW
nu ( i
1i
d 9c
s y h
t nt h g n o t ne ad ie t hefe i rdv r i
toe t
si r
ha f w a s
nef o a v e f cdo c
r o ais u tu
ec
rc
s
B
a W u r a( o t i1h ts n9t , f a ec9at cl ru3 u o i a r)ssha d cti nc g fsne i ngk o ool ftg ce
nho
e
fi
i
n yi (r t t ec a s lernw
,hi e t p s ir eginot f dssi cnsioi n oe pn coeom
sey ) ot c sf atoc h e
o la s
ts
e
r c
i o e o w nl nb ii nr fop lo Trd moa l tsxar hp ti hhw
i s vniti eo ei t. ea ealtr emfr c e shl o
a e
es s
ii g s c e
srr n t ae tn w
dx ioa n re nd i tz aef aod
i o n s f ri
o w
s
n
o
d
p r o f i c i e n c y .
n

o
e

c c
te
e i
o

e ge jA s b n
oo
iw t a
tsa c s f hosei v
s a mof laesr s m ia l pls d o wi a t e y r
t a sgtkn sb e i iyn otgc
n ii
br m
s t oGm
iu
Wo l h R
Ct c n
n e nf Eii
s io t r n
G ir hw e
R
e dw
nt e s t B
. r
a e
a
e f
asxv
s o s a h p my o e r, x o tp oe e trfar s rik swm, aee nos t ala l
s
i
n t tt
Ce
r he
s o t e
m
m
i
R

ig
v
k ny r 0
a e
e

dn
eI ui f n r
t t fo v f a o s il
oi r d df rmiim et r oay w
ae s
huc t br t aep i esr
vov ev
c ab B 9
ya ( l
19a
br 9M
e
18na i C 3 lr
()dnl
eB
dr r
e
sv
it
ode o z e nws wmet
du e
, y nb al ss e d-gm t t oa l nh ns i s e uni caod fil hug y noct
th os
dl t i r
eh f cp r o oar r r o e wt tl ae t di o
a
s w
c s i r me t eo oas wh s m su ei n r t a km
ee nts s ut echor l
ei T
na sm m t mu t e ehe
des r
, e g r
s uc e s o fe a c l re
od m r l o m ar e r hoo s l ns r i co t ca h gl g cot y wh o . i O
l u as2ot u d y n 3n s
e
n n i r n
i c et lB u r ehd
ee( Hd vw
ue d i1d9 5 l4 )er et se wt fapoc neo , ro o .t b er
d 4 r
e1
t t s i
t i a ni t n f es lt r dr i u ouc h it sto r r i css b h gw o a h o a lt h e i prn enn r f oo r ad me s saa y n c e
e
r
twe s t B
.
at
f o u n dl t e c o sh r r b e oa l ae s st a tyit os n n esca
s o s co nrt os te
r dsha e
ee
t
mr ,
o
b p s6 t ema
- ir ol ea tne r( n1e
tddf 2 th ns s ato wwu i dt rer Si o eL a trheO
srg ) e S e
s
u
u
n boa s e d o n o c n
i E l nn c gy l i i t
s. o ah l h2, . 3 O sn.r O o e
sd wr
i
s-i

ai
nd
rwM eu a i Co l
et (nl
h1cei t e w
ds 9dr
s9
t f0 c o o) r4 r e o e . l t a 6 t b i 4 o
t ) fe irm
t e lE oa wl s pi faW
u t r g T i e ronS sS he ntt
W
a dTs nel E d t a( u r
ad m
f
ce
u
rt
s t e
s tn
t a e t i nf c c n eho r o s dnt ewa
nv E dy n eaAgs t ru dl ybnd B
i
sr t
eo
kd e z nai r - R
e ect c ( a srn oa 1-h yc,a 9 c d 8 br kM7
sa i) C l
anl
m
e
t dr
J
oM
n e h re r p m o c u
cs
]
g a vr n a s ti ul dey[ i ita d df i m
ii o tr tr oy w e -f
(c e2 t g
yn
n r e ti mps i oo t cdirt o n et rt rerewe laadli d anti fi tsue it ri or t t nu r osc s t bh o rw s o a
(
t
a
r
a
w
ce
noi og
vur( t dt ar 5 e .tthf esad us d ae3et 2r ga. oursssne odd8 in ut) eu tr in t s t n i s
M
S

e
ie
e

nd s i 7
r
t
o
e
Camp,
he
.l i a
o
h
s
p9
r
3
5 .
47

d
e S t ut dsyp ai e r tu w
e i cr4 ni p epc iva ne7am
r t osr a ie 5 xl f
de ur
w no t
t tm
ho
eh o
e t a Gw G
t n u RT t a nde ne E
oo e t m sic ea et sb t t c sJi e c a t l M e1 9 9n4 . S a
wd
y
a
m o ar t
x t e n e
w t eh
t s h ae ca
s wo tt dai 5 0e - 6f 0t m d rhr
i aoi E dn l idi
eu u s
e
T
S
,r
en n f ige ol do fi f & st oi A ns t a Gnl l E a a n AvmCt
Ea e ,, AI Pr
h ,Lyo
e
v

3
i

t r
A , Pu
aZ Np a
Wn ,mJ t n 0 Doa o a , ad 1z C
sc n
i h, t dn
rc i e
i Ae sn
n t g
t
i y n
s M
t i in cM i h o i Ng cr ea
bnT s r,o V
ea is x nark W
gaa i, si n E ,i
atm d, h e
nx
a ai m c i
nn e
e o
pn
f
s
o E
w n o vne e g r oas r a i l mo npe t l iee d arns, s u l,f f ihdi c si e tn e t l
oet
t
ie it h
t
li
as
oaciwt r m
ibii t p w
e
lvt afl ser nlh
p opi i e n
xte m t
a i ce a t
a nt d ep
gpe r
o s
. tn o s g tw
o ud ndn c eh n ot ot owsoe
r s th r t k b tp iia e r l t s ti lc uTi s il r a rh w
ln rey
i g qa s
l e
ee o
d ck
t rni
et f a l d
tuho a s c c k ese
ue at
m
wml hr e p efe d lia t f s rer
tu ghoo l
t m eh
e u
tm
l op
wo i r o t v r voo
O
i rn fi
nt
i nd ce w
i e er e
a s nrt
rf e be q t hu io r eo i m
e e n
o ge
i
n
l
i
p
t
t
a
a
t
m
G
s
k
w
t
o
i
t
e
R
r
c
n
i
u
g
m
E
i
e
a
b
b
q
t
s
u
r
i
a
s
t
n
i
n
a
g
,
e
d
t
s
- h
t
e
u
d
y
fn
s
oa o w m t s pr h
dlh r if e a oce
te oao ny d unai p e rdr ns s/ t
eoa tg t m
r e h hm
e ei an
t
yv g
a
o sr u
b t gm - i st
t f - eac da
i h t no w s oh l d t p oeoika r t ykl i uc e i sep a t l de .
P

x
s

ond

r
e
do

p ie p A
ovt
t si t t
wwee
b h
etcoe
f e r epo e a r
r eae c k t
ehr e
w
s
ts
t
r ao
wup fa d i l di
c o etbs fo t cnho s ao noht t tt c r aou h hegt pr pr
nei eeh t c i w ds hbe p
oa
u e t s s td ef e c Auhss d o s ol s ee
( u act r ot ae gyl n h et nnbs gi i
o . it t e k
sei s
es c
f
ma a
w
t
e t mrt h ib a hee nl o ei sp osen er coa h t l it opo n o af mril t ) t wweo he a
sp a rs
o
a
s
t l e e s s ap n a ro tc t t i c hi pu a n t . d
y

g
n n
rtdp
p
xt
o
re x u
ec a
r
-

cae
rp
te
sns
e
e

t oi T
at i ee
s s wanss c i i o c t nsai t sd h ui se encse t n eosmv d f g c
ea
o
r
l
a
l
o
en t
t fj g e h xo b ia e m ur Viu so n l ra eu senes t s e wo ce ar e i et n t g hg r ra
oe .
u
po Aa
w
nf t
d o a p o a r otl w
mi ca i ep.tl wa n tsr t i s
re w k s
i o se
l
ee
(sc
ldsf
nfra rtpnay o io wos m p iun t omi s o r c ta h t s ucf
dn eo yo s d) n
i
e
e2 r hi 0 x- w
e m
o c ei t r n r urw l ti wt eo ea th ee
sx it r r ei
ehn e oa wcag g c t t a hh i
s
s
e vx
ai T m
f o ie t n xh oe p t e ar h.r e tot wi hca y i i epth aa ne sr t o rts t e2 nr kt0 d -h l me
ie yne
.
fx
e- a rs oc t r l s i e h s l x
e
m
.

o
E px
a w xm w i en h e re e s0 ow
s
g os o e4 am n iyr t t w
i s eo ovne t r a u n e o i tn e
oe 6 m n j t w
i
o bt nf o r
utHuh o h t iast e e trlh e t . 4of s0 e -heeo m f i ak ne hs u i t n ae
s
l
n
ytr 6l 0e -het m at f i m Tnie i ls u xw
or ot oa eps r es ipt s pt cr. a as h e
e af y n
t qo u
m u
n u
o
sn i
wf
to e
oir f a d nd io co ft n f iuo hee o rptt r ena ic n
cm
t ec I sia
hd n df ge i
t
ioa o
nc
yu
l s
so
t
u w no oto ha ebinh s p avseb a e e et e etti d f ae r onohe w ncrpn s nee au td n e
ss
e
s s rs o
e eunp
t stot
fr To i el to iht f e cd i i f eorhfts d f iae e i np m
rr e ec doni n
nt t cs gd
sh e
a
v
,
e
bnr r
e ei d e ee v l nt s ( eea n w& PiFh wn el i orpo e o I wsrw
hh el o l
rrseot
t n ssis , g
m
r
eu e l
s ytil
moh bi - is l i egtc e dcshogs hl errs o i i hared b i p sl noend d ug
r d h ee ot t a n a n se
l dr , y
e
f
f
e
c
t
.
c0

ug

no e
g o
om
u c
a

&
p r o E0 s s(i ot s6 om ar4 i v nvF u e s0 t ot e n sw
o su) u p. ei rt i s ss cr tE ehs ue a a d d e yc
a o g so o n st eh
i p t x kt w
piaho e a f l l ccabe
dsali i o rss ooi i a m
l st u f ynu ur p che
,l ht

t:u

gg

s p
a
l

a r
a

w hsc

T o w se
p a
w
t a
n
s

pW c af i a m
x hl

oa e

lt

a to

pe

t rfn r a

oooe

hb

er

r l .

h rdx y i ta a
oe do t n g w t tu s i rt
aha t t
be eS hmeo ye u vn e t oe p ? u p
rx
i a e e t
and h f ss ad r r s o a yor u a on
rom c es aw eu h
rann n
oy f o
os
ob pu r r ea res o e o pnyv e pl
o eed e r
un
xv r t
pa

4
Present Your Ideas in a Soecified Format (20 minutes-.
In a successful relationship,
Write a unified paragraph

both parties must be willing to compromise.

in which you perform the following tasks:

-- Explain what you think the statement quoted above means.

-- Explain your own position on the subject.
-- Give an example to support your position.
Exnlain the Meaninp of a Quotation

(20 minutes).

Talking is the worst possible way to get things done.

Briefly explain what you think the quotation above means and give an example to
illustrate your interpretation.
Revise a Comoosition (20 minutes). Examinees were given a poorly written passage of
about 350-400 words and asked to revise and edit the passage so that it corresponds to the
conventions of standard written English, treating it as a first draft of one of their own
compositions. Examinees were instructed not to remove any ideas or to add any, but rather to
concentrate on finding weak places that need changing. (This exercise was not suggested by
the GRE Writing Committee but rather was included at the request of the GRE Research
Committee.)
Other Exercises. Three other kinds of exercises were also administered.
However,
because the GRE Writing Committee subsequently decided that these exercises were less
promising than the others, they were dropped from further consideration, and therefore are not
included in the analysis. These exercises involved (1) discussing the relationships between two
statements or alternative views; (2) describing an object, event, process, etc., vividly and
precisely, and (3) reflecting on the examinees previously written essay about the meaning of a
quotation (the examinee was asked, for example, to anticipate a criticism of his or her essay and
to describe and refute it).
Ouestionnaire.
After testing, all study participants
Appendix A), providing, among other information:

completed a questionnaire

(See

1.

perceptions of their prior success with different kinds of writing and with various
writing tasks

2.

information about undergraduate grades (overall average, average in courses

requiring writing, and grade on the most recent assignment for which a written
product was required) and

3.

a self-evaluation of how the examinees writing skills compare with those of peers in
the same field of study.

S
o t

i pc
W
hx t m e
hc
ok ot er isRh afoe yC va e i fh l s t e ( n w
o
a s
u w b
s
s
ac t o w er i eec
de w
) x,s
ca i o e rsh n e b rdt t d t
r w icyrae i onw pieed d e sia
ai
ro
h o mo
it
of p
hnr
be ae t
r
eys aT dl
oeo h enr
.g e lex r
, ep v I s a s o
c
r
e oe e o t r nxs
hrh efeo
r e tw o e s r c o oa cl r m- he nt d oso
o-s r6c r
a l nf l g e1 i i , n
ef
se
d
i w
r s i nktt 6i i pn l g loa t= rs
a h ei
n nro s d ts etr e
i
e
v eq
i a u dt e n c et md e a u oss ft t ne e f hrt if oy el m
e i w -ef
Ao 0s r me 2 cn i o e n
c e l f re e r) ( xn e o oi t t,e a t t h nhso or uM
a Qe fac
c fe wu a i am
d s n
e s e ci ol oa
l t
t 3 y-l e spsa colade riaih ven l nfat ehstld , oe e rt sc u o l a
ro dos i s e c r r i m
f ic n a bt i j o n
uT e
h 0 t
2 e
e
l ( x
a Ca e
o Rw
m cr s m coa eo oac sp rsr r ve anoi d nf
g li s is n
ev
di
(t
o c n i
e
e c r molf t 6 fp hi i i e org = thgr ol
y( t evo pm a lev j i rlo m
re
v. i) t cd ae vfc lf l n oe a s r e d i r t
y

d c
r e

r
r

ls
s

a e
t s
n s ed
d
ea sn y s s dd s
g =s
h oso n g e
dt n
r ni e n d b
sh
gis =e
t e
i
on e
s

e
t iS cn to sre t i h ns g e rtew gr nsr i ia t a t m
piu s np e gl ra pee st d n i
e
n
ts ce a v
hr Sn
p aei
o w
l
g dl c
en
l
ni yi t a gk e oh w
n te rh sei .fat i t im n p gt l
e
c
s
u
bT m
re ei h w
qtl .fu eaebra so at ws e r r ts i at opctm1 e( p efn le efom
pe0 w a A
enr g r e ) s t .
l
m e
r ki x e eo n s a t s o pa c yam bps
,er l r p e sl ,er u s l a p ve da be ra
c pi e osns
eopdd t
owr u
i T
l
o h lp we a f g a e r s
s m e uh d oe qzrs oi a r l um
rp a pai wf lcena eh g ps i ness g d h t s vl
yle e
y
o
g d
o tr
i e ct r
sb ap o
oe oht ro m m
t t t hh aoo peb u r p sgsl u a p i r t r n a oe g
m
a r
y e e N ga v
tns siet tn e ad coe ohli o f s il ead n m
pg af pi a sl h e s c e
f ss
o B cs e r ct
a o u h s e re
t nc e m
p n r gnei i ca so i tsn sauh e ehr f e wf rew
ot eso , ayhr m
w
l tpwnei l r c ewi i sntl kpt a xe o e n s
t
a ns s.es ex ecco woedr nrov hit a otn eerg m
som pn ou l e v Te d
h
o es
s fec co
r pf i rn
go a
i
i y
d
, t e t n io pl hpa - i bp cfa es gi r -d yesc
fi v eae x o aa- m
rm
d r ph ol y- ee i m
e
l md
e
d i
r
f oo ca um s nc e d do s
c fsm os
ir oi pme a i s H
prhla ea rr t r so a . a ww p n b a eop l
s
t ev r
sae
dm
r iv e epe h m. rif e t ddts o t eg x u rt d e y hi , v
vae n r ao oi t e sru da hr fm
p e p el e s

e
h e

ha
fs
. n
s w
u
i e
yi ep
l
u
c r e
pt o
e
a
e

o
g
i
e
shb n o a i
e s t t a it yn
. d e r

r
c
d
e
s
r

C e g o ran w n r iels i at sd m
wil nps r e gl c oe ooas6q r -c ehp nu doc s i ocena ta
lnt l e
,t i o
w
o
to w e rhwc i
t e d e h r o h ii
te rt s u r h o nf
a e t c i r u a l r vi t e n y a e esd d e
t e s o k t Er u r d es ry e . w
a
da e yt r e s ye t novs eha h em
a i ahpt e leh e ts l sf ba
ud
f i a tee mto etda oeng r at c c e ei g
o rm
ra n dr eR fd e e w
drr m
a ud pe e r r ts m
j a a
oi l
fa geb
e ud h fs en i ht
lr su
w
f t ce oos b o ngw
a nr k n a c a a i rk b
fs g ld
ore
a
nc t
p i d a h cpn ee e A
rh oa r sc e a sf ssne hu
lie tt t gt , hwpa l ae (y p n e mn r d s a t i
rtr l o cf r o e afs ee s e w, nb l
up
B o e ts si r dch a eo s rebt rse
s gs yes r
a , d ai e
rw . ab pre sd e rwyded r os e b esyf s
la eypt u a in ss uon d su an dr ob
ters
seen
Pr
ee wl

l
r

s
a

c
e

ro
n

t
s

sl ee e
tm
w ar s adac o ots tcesr
eh6 ne os d- sp e mou n i f ne t saoa snc aorn l yut
rs d s s e
ar rs it te iedR an e g w
a e d em e lr n s t p a o sc t s l or uo oh m
g e ue e a fr rs d
aa . m
g d
mi
w e m
hr m
e sco m e bi t wov r e bai ear t
t t epfry n ae v p as om
r li
e, e ti f y n c
r
eR e aw d d e r t sei
t ovsr
h t fae t e qa o u lhc o a t r l we iutr he t fi y t em
aei
n
ge
t t ra t cxn iif d o hno s hoi
iet t r o gd b a
p i f hi m t t e r t o ame i r b r tg d ls sef e,a t n oe h d
ne
e a h T) s w
hei w e rwe
rn p i n t
et r
so rh (
v aee o t
i nwmc r e d s a e e
gsl
e d fi es n t obe c l
oa e e
t v
u a
r n
e t
s

s of Wri$&_&j&
As stated above, the general strategy was to relate performance on the prospective
writing tasks to several different indicators, each of which might reflect a slightly different aspect
of writing skills. Three broad categories were differentiated, as follows:
1.

self-evaluations,

that is, student perceptions

2.

self-reports of writing-related

3.

readers assessments of samples of writing submitted by students

achievements

of their own writing skills

and accomplishments

Information in the fiist category came from subjects responses to several items in a
questionnaire that was administered to all participating students. The questionnaire included an
adaptation of an inventory developed by Ward and Carlson (1984) who asked secondary
students how successful they had been with various processes of writing and with sundry kinds of
writing assignments. Ward and Carlson found that such reports correlated in the .3Os with a
number of other measures, including the College Boards Test of Standard Written English, a
measure of the knowledge of conventions of standard written English (.39).
Information in the second category was derived from responses to a 12-item inventory of
writing accomplishments.
This inventory was an abbreviated version of a longer, more
comprehensive measure based on the work of Baird (1979) and extended by Lawrence Stricker.
The particular adaptation used here involved only writing-related accomplishments, and
respondents were not asked to provide documentation of their achievements, as was required by
Baird.
The second category also included information about students grades -- both in courses
that involved writing and on the most recent assignment for which a report or similar exercise
was required. Reports of overall undergraduate grade averages were also collected.
The third category included evaluations of writing samples (personal statements and
course-related writing assignments) that, as mentioned above, were supplied by students and
evaluated by trained readers for this study. Professor-assigned grades, as reported by students,
were also examined as a basis for evaluation. A summary description of indicators within each
category is given in Table 1.
Results
The Sample
Table 2 compares selected characteristics of the study sample with those of all GRE
General Test takers (1991-92), the population of interest. Generally, the study sample appears
to be reasonably representative of the GRE test-taking population. Differences are apparent
with regard to the slightly higher proportion of women and arts/humanities majors in the study
sample relative to the GRE population, and the somewhat higher average GRE verbal and
analytical scores of the study sample.

Table 1
Summary Description

of Independent

Indicators of Writing Skills

Descrintion

Categoty
Self-evaluation
Comparison

fPercentions1

with peers

. . . . . . . . . . . . . . . . . . . . . . . .

How do you think your writing compares with that of other students in your field?
(Responses on a S-point scale from well below average to well above average.)

Success with the processes of writing . . . . . . . . . . . . . .

Reported success in college courses with four different writing activities -- thinking about
assignments, organizing, revising, and developing an effective writing style. (Responses
on a S-point scale ranging from not at all successfuP to extremely successful.)

Success with various kinds of writing . . . . . . . . . . . . . .

Reported success in college courses with seven different kinds of writing -- personal,
creative, persuasive, analytical/critical, descriptive, examinations, and applied.
(Responses were on a 5-point scale ranging from not at all successful to extremely
successful.)

Evaluation of responses to study exercises . . . . . . . . . .

How would you rate the writing you did for the tasks you were given [for this research
study]? (Responses were on a 6-point scale that corresponded to the holistic scale used
to score essays.)

Self-renorts

of Achievements/Accomnlishments

Grade average in courses involving writing

Grade on most rmnt
Documentable

writing assignment

accomplishments

..........

in writing

Readers Evaluations of Student Writ&

.........

.........

Grade average in courses that required at least some writing.

Grade on the most recent assignment for which a report, essay, etc., was required.
The number of writing-related accomplishments reported. Responses were to an
inventory asking about 12 different accomplishments, such as writing (and publishing)
articles, technical manuals, speeches.

Samnles

Writing sample evaluated according to

(a) grade assigned by professor
(b) holistic rating provided by specially trained
readers

Study participants were asked to provide a typical example of writing they had done for
a course. The request was for a paper of 10 or fewer pages drawn from the students
recent writing. Acceptable examples were essays, term papers, book reviews, lab reports,
proposals, or case studies. (Writing samples were scored by readers using a 6-point
holistic score scale.)

Personal statements

A personal statement, statement of goals, etc., that the student wrote as part of an
application to graduate school. (Personal statements were scored by essay readers using
a 6-point holistic score scale.)

. . . . . . . . . . . . . . . . . . .. . . . . . . .

Table 2
Background Characteristics of Study Sample vs.
GRE Test-Taking PopuIation

Background

Characteristic

General Test
GRE verbaI mean (SD)
GRE quantitative mean (SD)
GRE anaIyticaI mean (SD)

514 (120)
548 (128)
563 (123)

486 (118)
555 (139)
537 (129)

Femaie (%)

65

54

Ethnicitv %
American Indian
Asian American
Black
Hispanic*
White
Other

1
7
11
4
75
2

1
3
7
4
83
2

89
41

N.A.
41

7
7
23
25
7
21
11

4
9
16
22
14
27
9

12
29
34
18
7
cl

12
26
37
17
7
cl

English Best Language (%)

Degree Objective (% Doctorate)
Undergraduate Major Field (%)
Business
Education
Arts/Humanities
So&I Sciences
Engineering
Naturai Sciences
Other
Underrzraduate Grade Averape (%)
A
AB
BC
C- or lower
m.

Study Sample
(N = 470)

1991-92 GRE Test Taking Population

(N > 277,000)

GRE popuiation data are based on unpublished statistics compiIed from GRE General
Test examinees responses to the background information questionnaire that
examinees complete when registering for the test. Responses for the study sample are
based on Ns of 394 to 470.

*Includes Mexican American and other Hispanic.

the study sample.

There were no Puerto Rican examinees in

Table 2
Background Characteristics of Study Sample vs.
GRE Test-Taking PopuIation

Background

Characteristic

General Test
GRE verbaI mean (SD)
GRE quantitative mean (SD)
GRE anaIyticaI mean (SD)

514 (120)
548 (128)
563 (123)

486 (118)
555 (139)
537 (129)

Femaie (%)

65

54

Ethnicitv %
American Indian
Asian American
Black
Hispanic*
White
Other

1
7
11
4
75
2

1
3
7
4
83
2

89
41

N.A.
41

7
7
23
25
7
21
11

4
9
16
22
14
27
9

12
29
34
18
7
cl

12
26
37
17
7
cl

English Best Language (%)

Degree Objective (% Doctorate)
Undergraduate Major Field (%)
Business
Education
Arts/Humanities
So&I Sciences
Engineering
Naturai Sciences
Other
Underrzraduate Grade Averape (%)
A
AB
BC
C- or lower
m.

Study Sample
(N = 470)

1991-92 GRE Test Taking Population

(N > 277,000)

GRE popuiation data are based on unpublished statistics compiIed from GRE General
Test examinees responses to the background information questionnaire that
examinees complete when registering for the test. Responses for the study sample are
based on Ns of 394 to 470.

*Includes Mexican American and other Hispanic.

the study sample.

There were no Puerto Rican examinees in

Descriotive Statistics for the GRE Writinp Exercises

The four Issue topics were comparable in terms of difficulty, with mean scores (sum of
two readers) ranging from 7.7 to 7.9 (SDS from 2.0 to 2.2). As noted earlier, the majority of
participants wrote two essays -- on either topics A and B or topics C and D. The correlation
between fiist and second essays was S5. (The reader is reminded here that, by design, for each
writer one topic was disclosed before the test and the other was not introduced until the test
administration.
Also, for each examinee one essay was written in 40 minutes and the other in
60 minutes. In all likelihood, these differences resulted in a lower correlation between essays
than if both had been administered under exactly the same conditions.) The interreader
correlations ranged from .77 to .80 for the four essay topics.
The means (sum of two readers) and standard deviations for the shorter exercises were
as follows. For Meaning of a Quotation the mean was 4.3 (SD= 1.5) on a 1 to 3 scale (sum of
two readers); for Specified Format, 8.0 (SD = 2.5) on a 1 to 6 scale (sum of two readers); for
Revise a Composition, 3.2 (SD= 1.1) on a 1 to 6 scale (only one reader was used for this
exercise). The interreader correlations were .83, .94, and .83, respectively for these three
exercises. (The estimate for the third exercise is based on a double reading of only a sample of
responses.) These shorter measures related differentially to essay scores (r = .32, -.02, and .64,
respectively), and none related very strongly to the others.
Descrintive Statistics for Writinp Indicators
Comoarison with Peers. Generally, study participants held themselves in high regard with
respect to their writing skills. A majority reported themselves to be above average -- either
somewhat above (48%) or well above (20%) -- when compared with peers in their major field of
study. Another 25% regarded themselves as being about average, but only 6% felt they were
either somewhat or well below average. This same evaluation was reported again about four
months later by about one quarter (N= 111) of the initial respondents. The correlation between
early and later ratings was .70, suggesting moderately high reliability.
Evaluation of Performance on Writinp Exercises. Study participants exhibited somewhat
greater variation when asked to estimate how well they had performed on the writing exercises
used in the study. Students rated their performances as follows:
Thorough and insightful (6)
Well developed (5)
Clearly competent (4)
Limited (3)
Weak or seriously deficient (2 or 1)
These frequencies

appear to approximate

4%
27%
41%
23%
6%

a relatively normal distribution.

Success with Processes and Rinds of Writing. Table 3 shows the specific activities and
kinds of writing about which participants were asked. As can be seen, respondents generally
reported that they had experienced considerable success in writing. For each writing process
and type of writing, a majority of subjects indicated that they had been either quite successful
or extremely success ful. For each activity and kind of writing (with the exception of creative

10
writing), fewer than 10% of the participants reported that they had been not at all or not
very successful. The same questions were asked about four months later of a subsample
(N= 111) of participants. The correlations between initial and follow-up responses for two scales
-- processes and kinds of writing -- were .73 and .70, respectively, again suggesting a
reasonably high level of reliability. (Internal consistency estimates based on coefficient alpha
were .66 and .75.)
Grades and Accomolishments. Participants generally reported very high grades, both in
courses that required writing (68% reported either an A or an A- average) and on their most
recent assignment that required writing (80% reported either an A or an A- grade). Thus, there
was relatively little variation among students on these particular indicators.
The degree to which students reported various writing-related achievements since high
school is shown in Table 4. As expected, most of the 12 accomplishments listed on the survey
were relatively rare. Only 5% of the respondents said that they had achieved 6 or more, and
33% reported none of the accomplishments. The mean number of accomplishments was 1.7
(SD = 1.9).
Writing Samoles and Personal Statements. More than 90% of those taking part in the
study brought at least one writing sample with them to the testing center. Various features of
the samples (when and where they were written, the amount of time spent writing them, and the
grades that were awarded) are shown in Table 5. A slight majority of study subjects (54%)
reported that the sample had been written more than a month prior to the testing date, but
within the previous year. Most samples were written totally outside class. Some samples (20%)
were produced in relatively short order (two hours or less), and others (34%) required
significantly more time (ten or more hours). Despite our instructions for subjects to provide a
tvpical example of their writing, most samples had received very favorable evaluations by
professors, according to participants reports. A majority (62%) reportedly received grades of A
or A-, and significant numbers (29%) were not graded.
The essay readers trained for this study awarded, on average, relatively high scores to
evaluate the papers that students brought with them (M=4.7, SD = 1.1 on the l-to-6 holistic scale
that was used). A random sample of 22 papers was scored independently by two readers,
yielding an interreader correlation of .68. This modest level of agreement suggests that, given
the relatively minimal level of training provided for evaluating these samples, readers may have
based their judgments on somewhat differing points of view. Given the highly diverse nature of
the samples, it is perhaps surprising that a single, generic scoring rubric resulted in agreement
that reached even this level.
Fewer than half of the study subjects provided personal statements, possibly because
students regarded these documents as being of a more confidential nature than course-related
samples. It seems more likely, however, that many students had not, at the time of testing,
needed to write such statements. Of those providing personal statements, a significant
proportion (37%) said that they had received at least some help in drafting their statements,
and an even larger percentage (61%) acknowledged having had help in editing and revising
them. Like the writing samples, personal statements were also judged, on average, to be of
relatively high quality on the 6-point scale that was used (M = 4.6, SD = 0.9). Double scoring of a
random sample of 29 statements yielded a reasonably high interreader correlation of .78.

11
Table 3
Reported Success in College with Various
Writing Activities and Kinds of Writing
Percentage

Activity or Rind of Writing

Mean

Reporting

SD

Somewhat
Successful
(or less)

Quite
Successful

Extremely
Successful

Activitv fProcess\
Thinking about an assignment (e.g.,
developing ideas, gathering information)

4.2

0.7

15

SO

35

Organizing (e.g., making outlines,

deciding on the order to present ideas) . .

3.9

0.8

28

46

26

Revising (e.g., improving sentence

phrasing, rearranging ideas, correcting
grammar and punctuation)
. . . . . . . . .. .

3.8

0.9

33

44

23

Developing an effective writing style (e.g.,

expressing ideas clearly and in interesting
ways)............................

3.9

0.9

33

43

24

4.0

0.9

25

40

34

Creative writing (e.g., a poem or short

story)............................

3.6

1.1

47

30

23

Persuasion (e.g., arguing a position or

writing a letter to the editor) . . . . . . . . . .

3.8

0.8

38

42

20

Analysis/criticism (e.g., reviewing a book,

movie, article, theory, work of art) . . . . .

4.0

0.8

27

43

30

Description/explanation
(e.g., describing
an experiment on how something works) .

3.9

0.8

30

45

25

Examination writing (e.g., long essay

answers) . . . . . . . . . . . . . . . . . . . . . . . . .

3.8

0.9

34

44

22

Applied writing (e.g., preparing an

application or resume)
.. . . . . . . . . . . . .

3.9

0.8

27

48

25

Rind of Writing
Personal writing (e.g., describing
experiences or feelings as in a journal)

&&.

&ale

. .

N= 466-469. For kinds of writing, results are based on only those who reported doing
each kind of writing. Examinees who reported they had not done a particular kind of
writing (N = 4 to 69 for the various activities/kinds) are not included in the calculations.
is 1 = not at all successful to 5 = extremely successful.

12
Table 4
Writing-Related

Accomplishments

Reported by Study Sample

Percentage Reporting
Engagement

Activity

28

1.

Wrote a feature article, column, or editorial that was published

2.

Wrote technical manuals or other instructional material, for pay, for

a company or other organization ............................

23

Wrote advertising or public relations material, for pay, for a

.............................
company or other organization

22

4.

Wrote poetry, fiction, or essays that were published

21

5.

Wrote a letter to the editor that was published

6.

Was on the editorial staff of a publication or a radio or television

station ................................................

15

Authored or coauthored a paper that was presented

meeting ...............................................

13

3.

7.
g.
9.

Authored
journal..

....

.............

................

20

at a scientific

or coauthored an article that was published in a scientific

..............................................

Wrote a speech for someone else that was given at a large public
gathering (i.e., over 100 people) .............................

11
10

10. Invited to participate in a writers workshop sponsored by a

statewide, regional, or national organization
...................

11. Wrote a play that was publicly performed or a screenplay for a film
...................................
thatwaspubliclyshown

12. Wrote the script for a dramatic or comedy show for radio or
television that was publicly broadcast .........................

m.

Percentages

are based on 142 respondents,

13
Table 5
Characteristics

of Writing Samples

Characteristic
When written

Where written

Time spent writing sample

Grade on sample

Category

Percentage

Within the month prior to testing

17

One month to one year before testing

54

More than a year prior to testing

29

Totally or mostly b class

Mostly outside class

13

Totally outside class

g3

lko

20

hours or less

Three to nine hours

46

Ten or more hours

34

Ungraded

29

B or less

10

A-

15

47

14

Table 6 displays the correlations among the various manifestations of writing skill
collected for this study. As is apparent, the relationships among the indicators are clearly less
than perfect, probably because these indicators either tap distinct aspects of a more general
construct of writing proficiency or, if measuring a single aspect, reflect it with less than perfect
reliability. Assuming the former, the best markers of a general construct appear to be
comparisons with peers, grade on the most recent writing assignment, and self-reports of success
with the processes of writing and with different kinds of writing. The median correlations of
these indicators with all of the others in the set were .41 to .42. The most distinct indicators
appear to be professor-assigned writing sample grades and quality of personal statements
(median correlations with other indicators of .06 and .09, respectively.) Writing-related
accomplishments were relatively unrelated to the other indicators also, exhibiting a median
correlation of .13 with other indicators. The degree to which these weak relationships with
other indicators reveal a specific contribution of these indicators to measuring writing
proficiency (vs. simply a lack of reliability) is unclear.
Relationshios

of GRE Writing Exercises to Writing Indicators

Individual Indicators. Table 7 displays the correlations of each indicator of writing skill
with each of the writing tasks investigated in the study. For the expository essays, two sets of
correlations are reported -- one based on performance on both essays written by participants
and the other based on the median correlation over first and second essays.
Correlations with essays were modest (mainly in the .3Os) for each of the nontest writing
indicators. The indicator having the highest correlation with essays was comparison with
peers; the indicators exhibiting the lowest correlations were writing samples and personal
statements. Correlations of indicators with two other exercises (Meaning of a Quotation and
Specified Format) were, with few exceptions, negligible. Two anomalies were significant
(p < .Ol) correlations with reader-scored writing samples, and one significant (p c .05) correlation
between Meaning of a Quotation and the personal statement. Performance on the Revise a
Composition exercise bore modest relationships with the various indicators.
In light of the extremely heterogeneous nature of the writing samples that were
submitted, it seemed prudent to disaggregate them into more homogeneous, and therefore more
comparable, subsets. Iwo classifications were used to define subsets: the amount of time
students had spent writing the papers, and students undergraduate major fields. The rationale
for the first was that, compared with documents prepared over a long interval, samples written
in a relatively short time period might relate more highly to timed essays written in a testing
situation. Analyses did not, however, confirm this expectation: essay scores had the weakest
relationships with samples reportedly prepared in an intermediate amount of time (3 to 9
hours). The correlations of essay scores with writing samples, by amount of time and type of
evaluation were as follows:

Table 6
Correlations
Indicator

SD

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8a)

1.

Comparison with peers

. . . . . . ,. . . .

465

3.8

0.8

2.

Success with processes . . . . . . . . . . . .

466

4.0

0.6

.55

3.

Success with kinds of writing . . . . . . .

468

3.9

0.6

53

.69

4.

Self-evaluation of study responses

. . .

464

4.0

1.0

.45

.43

.46

5.

Grade average in writing courses . . . .

463

5.9

1.0

SO

-49

.45

.29

6.

Grade on most recent writing

assignment . . . . . . . . . . . . . . . . . . . . .

451

6.3

0.9

.41

.42

.42

.26

54

7.

Accomplishments

142

1.7

1.9

.25

.ll

.24

.20

.16

.09

. .

297

6.5

0.9

.24

.32

.26

.13

.34

.51

-.05

8b. Writing sample (reader graded) . . . . .

363

4.7

1.1

.06

.04

.08

-.04

.lO

*.03

.13

-.04

9.

155

4.6

0.9

.20

.09

.09

.21

.13

-.04

-.08

-.Ol

in writing . . . . . . . .

8a. Writing sample (professor graded)

w.

among Several Indicators of Writing Proficiency

Personal statement

. . . . . . . .. . . . . .

(8b)

.24

Scales for variables were as follows:

Variable
Variable
Variable
Variables

: 1 = well

1
2
3
4, 8b, 9

Variables 5, 6, 8a
Variable 7

:
:

below average to 5 = well above average.

Average of four activities on a scale of 1= not at all successful to 5 = extremely successful.
Average of seven kinds of writing on a scale of 1 =not at all successful to 5=extremely successful
1 =seriously deficient in writing shills to 6=welLdeveloped response demonstrating mastery of the elements of
effective writing.
l=D or less to 7=A.
Sum of yes responses to 12 different writing accomplishments.

Correlations are based on samples of from N=48 (personal statements with accomplishments), N= 83 (writing sample -- professor
graded with personal statements), N= 103 (writing sample -- reader graded with personal statements), ... to N=466 (for several pairs
of variables).
For Ns of 48, 83, and 103, correlations of .29, .22, and .19 are significant at the .05 level, and correlations
significant at the .Ol level, two-tailed test.

of .37, .28, and .25 are

e l b aT

C
efvi t aonrehAo
f srowtacidnI o
f oi r W

i
I

t
d

.
.

2
N

S
(

= 3

3
N

S
(

= 3

4
N

S
(

= 3

swdun i k f c o w i c
1
1
10
0

r
1

fy d uote s
r
1
10
0

. e d5a r G
a
N ( = 2

n
1

iv w
1
19

. e d6a r G n o tmn e c e r
N ( = 2
1
1
.

7
N

A
(

19

c
c
on mi w p
= ,1 2 , 3 3 ) 2 2 3 4

W e l pma s rrosseforp( i
( = ,1 1 , 7 6 7 1

. b 8
N

W e l pma s r
( i
( = 2, 6 , 8 5 8 1

9
N

P
(

s
= ,1 )1853 , 83

60

4 9 1h . 6
,
,

60

rs

Er

er
n

cg
g

e s pi nv eR
i ae
o C
t
r
m

n c
a

4.

s 4 1/ .
)

s .

4 2 e2.

8s

s3h 1i 2 . 6 n 8 0/ .g
0,
,
,
)

8 3 3.

n .

a4 2 s3.

4e m

1 3 .

e 3 p9 0 . 9
8,
,
,

ov 5 0/ .
)

1g r i
8,

3 2 e s.n
,
,

8eg1 .
n 9 1/ . g
)

2 1

5 2 3.

/ g*

3 0 3.

5 d0/ . .

).

3 2 1. 0

4 1 1.

1n 2 1.

t 5

t3 i4 s 0 . 4 n

,
na.

gd1
,

4
aa n .
1
t

5
3 . / 53 .

- e8

dd g1 9 2ee. 8
,
)

dr9 4/ .

7
e 1 7 3 .m
,

l /l e.

3 1 .

7 1 .

ml t s r irFe bm
eu n
n e rhi t r pe d nr uy a s s e os e a hi t
c
if e oh t i
e hd nTo c .e s s e hinnat i d em e f o c t
tw
ts r i odf nd naio c e s si y a sr sre

c r o Fs i fN, o0 3 f d1 n a
i3
n f o g. d. n 5a 1i e . r 4a 2s

e. t n

ir
re i
3 6

e
i

6 1/ .

1
1

3 7t i0 . 8
is .
t
4
1
3,
,
,
im

r
th

oe
S a
m t amur oF o
Q oI c

s3h e5 1 . 2
9,
,
,

r o i 3 . /a5u3t.
68
6
3

a
0

0,

.s

&e

gr t
t

Pt w

e ti
s. t
71
7
5

gt

NA I DEM
.

a )

oe t
s.c
60
6
5

wtnemnogi ssa r
0
1

. a 8
N

e .

ec r
0

79

ix
i

xW
M
pa Q

t.
1

r Er
Pr t

o
s cy a s s E

ri c
10
0

i
0

r
s

swr e e p
1
19

wu

c
i

(n

.nos i1r apmoC

N ( = 2
.

c 0 n f 0 o g. o0d. n la il 0e . r 3a , r 2s
t e 6ha lt O
5, I e. vedelI i a t -ow,t
,. t se t

4 2 .

w
en h mt u s f od w
o it
w t
r
h

, t e h6a 5t 0r0, I e. ve l d n ,a

,ce

e i

st
e

ol

17
Hours

Readers Score

Two or fewer
3 to 9
10 or more

Professors Grade

.29*
.13
.36**

.22
.12
.2g*

*p < .05, two-tailed, **p < .Ol, two-tailed

Relationships also varied somewhat by major field of study. The correlations, by major
field of study, between essay scores and writing sample scores (for essay readers and professors)
were as follows:
Major Field
Arts/humanities
Social sciences
Natural sciences/engineering
Business/education/other

Readers Score
.45**
.43**
.06
.09

Professors Grade
-.06
.25
.55**
.13

(Sample sixes were insufficient for computing comparable correlations

the shorter writing exercises.)

by major field for

Comoosite Indices. Several indices of writing proficiency were constituted on the basis of
logically related subsets of indicators, as classified earlier in Table 1. Each individual indicator
was first z-scaled to a mean of 0, standard deviation of 1. The resulting z scores were then
added to form several composite indices, which were differentially related. Index 3, based on
evaluations of writing samples, exhibited negligible correlations with the other indices (z= -.07 to
.13). Index 1, a composite of self-estimates, correlated moderately with each of the three selfreport indices (I= 55 to 64). (The three self-report indices were highly related to one another,
as they shared common components.)
The correlations of the alternative writing exercises with each of these indices are given
in Table 8. This table shows moderate correlations of essay scores with the various indices -correlations that are somewhat higher than those based on individual indicators. Though
available for a relatively small subsample of study participants, an index based on self-reported
grades and writing accomplishments bore the strongest relationship to essay-based scores.
Again, the essay task seems to have the most potential, and the Revise a Composition exercise
the next most promise.
Relationshins

of GRE General Test Scores to Writing Indicators and Essavs

Table 9 features the correlations of each nontest writing indicator with scores from each
portion of the GRE General Test. Overall, GRE verbal scores bear a stronger relationship to
these indicators than do either quantitative or analytical scores. Comparing these findings with
those in Table 7, it appears that essay scores are, on average, slightly more highly related to the
various indicators of writing than are any of the scores from the current GRE General Test.

18
Table 8
Correlations of Writing Exercises
with Ahernative Indices of Writing Proficiency
Writing Exercise

Index (Ns)
1. SeIf estimates
Processes, kinds of writing, and
comparison with peers
(N = 299, 106, 115, 169)
2.

b. Grades in writing courses, most recent

writing grade, accomplishments in
writing
(N = 136, 30, 23, 31)

3.

m.

Specified
Format

Revise a
Composition

Essay

WQI

WI

W)

.44/.39

.22*

.lS

.41***

.42/.37

.17

.19*

.31 ***

.59/.53

.19

.ll

.40

.36/.3 1

.14

.13

.39***

.3 11.27

.22

.41*

.26

Self reuorts

a. Grades in writing courses, most recent

writing grade
(N = 290, 104, 111, 163)

C.

Meaning of
a Quotation

Grades in writing courses, most recent

writing grade, professor grade on writing
sample
(N = 184, 71, 75, 114)
Readers evaluations
Reader score on writing sample and on
personal statements
(N = 71, 20, 24, 40)

AII correlations involving the essay exercise are significant at either the .Ol or the .OOl level. The
significance of other correlations is indicated as fohows.
*pc.o5, **p<.o1, ***p<.oo1.

19
Table 9
Correlations

of Writing Indicators with GRE General Test Scores

GRE Scores

Indicator
1. Comparison

with peers

Quantitative

Analvtical

.43

.04

.20

2.

Success with processes

.22

-.06

.lO

3.

Success with kinds of writing

.26

-.08

.14

4.

Self-evaluation

of study responses

.24

.Ol

.16

5.

Grade average in writing courses

.36

.06

.28

6.

Grade on most recent writing assignment

.30

-.oo

.23

7.

Accomplishments

.17

-.26

-.03

8a. Writing sample (professor graded)

.18

-.03

.05

8b. Writing sample (reader graded)

.16

.lO

.18

9.

.23

.24

.25

.24

.Ol

.17

Personal statement

MEDIAN
m.

Verbal

in writing

Ns=449 to 466 for indicators (1) - (6).

Ns= 142, 295, 361, and 155 for indicators (7) - (9), respectively.

20
Students scores on the essays written for this study also correlated with GRE scores.
The correlations between a single essay (median over both essays written) and GRE verbal,
quantitative, and analytical scores were 53, -.Ol, and .30, respectively. The corresponding
correlations for the Revise a Composition exercise were .63, .09, and .49. (The Meaning of a
Quotation exercise was much less strongly related to GRE scores, I= .29, .14, and .19, and the
Specified Format exercise was entirely unrelated to GRE scores.) There is, therefore (as
expected), some overlap with scores from the current GRE General Test for both the essay and
the revision exercises. An issue here, then, is the extent to which these exercises tap more
general (verbal) reasoning skills in addition to writing proficiency. These figures are not so high,
however, as to preclude a unique contribution by a writing measure. By comparison, GRE
Subject Test scores, which have been shown to contribute uniquely to the prediction of academic
performance when used in combination with GRE General Test scores, correlate at least as
highly with the General Test (Briel, ONeiU, & Scheuneman, 1993). For example, in 1990-91 the
correlations for the Subject Test (Physics) that was least highly related to the General Test were
.18, 59, and .36 with GRE verbal, GRE quantitative, and GRE analytical scores, respectively.
Comparable correlations were .79, .70, and .69 for the Subject Test in Sociology, the test most
strongly related to General Test scores. More typical of correlations were those reported for
the Economics Subject Test (52, .66, and 56).
Discussion
Several writing tasks were administered to volunteers recruited from the GRE test-taking
population. Volunteers represented the population quite well in terms of ability level and
demographic characteristics.
Study participants reported a variety of other information about
their writing skills and experiences with writing, and most submitted a course-related sample of
writing and/or a personal statement written for application to graduate school.
Maior Findinps
Among the major findings were the following:
l

The various nontest indicators of writing skills were weakly to moderately

interrelated, suggesting either that they embody different aspects of writing
proficiency or that they are only moderately reliable. Very probably, each indicator
also reflected somewhat different extraneous influences.
The writing exercises that were investigated related differentially to the various
nontest indicators. Each of two shorter exercises requiring examinees to provide
their reflections on a quotation and to write in a specified format exhibited only very
weak relations with the indicators. A third 20-minute exercise, involving the revision
of a poorly written composition, performed significantly better (and nearly as well as
the longer essay). The expository essay task, however, appeared to be the most
promising of the lot, generally, with respect to its correlations with other
manifestations of subjects writing skills. Still, however, correlations with studentsupplied writing samples were weak. This latter finding corresponds to results of the
National Assessment of Educational Progress comparisons of classroom writing (of
fourth- and eighth-grade students) with writing performed under strictly timed
conditions (Gentile, Martin-Rehrmann,
& Kennedy, 1995)

21
.

Although the quality of writers essays was not especially strongly related to any of
the nontest indicators, it did relate modestly to most of them. Moreover, essay-based
scores were somewhat more highly related to several composite indices of writing
proficiency based on sets of logically related individual indicators than they were to
individual indicators.
Performance on expository essays was, on balance, related somewhat more strongly
to alternative indicators of writing than were GRE General Test scores, suggesting
that a writing test comprised of essay topics would contribute to the current battery
of GRE. measures.
An exercise involving the revision of a poorly written essay might also be worthy of
further exploration. It is noteworthy that this 20-minute exercise, scored in our study
by only a single reader, bore significant correlations with several indicators of writing
skill. Had this exercise been given equal time with the 40- and 60-minute essays, it
might have fared even better. Furthermore, performance might have been more
impressive still if, instead of using holistic scoring methods designed primarily for
expository essays, we had employed the more detailed (but time-consuming)
procedures developed by Godshalk, Swineford, and Coffman (1966) and studied by
Breland, Banner, and Kaplan (1993).

Studv Limitations
The limitations of the study apply principally to the nature of the nontest writing criteria
that were used to gauge the meaning of the alternative writing exercises. First, no claim is made
that, as a collection, the assortment of evidence assembled here constitutes an especially
compelling theoretical framework that adequately spans the construct of writing proficiency.
Each of the indicators has its own weaknesses, some of which are shared with others; however,
each contributes, we believe, in at least some small way to the appraisal of writing skills.
Comparisons with peers and other self-assessments, for example, appear to have
significant validity (as gauged by correlations with other objective indicators of ability) in a
number of different contexts (Mabe & West, 1982), particularly in academic settings (e.g., Baird,
1976), and most especially in self-assessments of foreign language skills (Blanche & Merino,
1989). As Upshur (1975) noted with regard to self-assessments of language skills, learners often
have access to the complete spectrum of their successes and failures, whereas other indicators
may represent a much smaller segment of experiences. Self-evaluations suffer, however, from
their somewhat dubious objectivity: although individuals may sometimes be the best judges of
their own capabilities, they are undoubtedly in many circumstances also the least able. Evidence
that most people regard themselves as above average is not confined to Garrison Keillors
fabled Lake Wobegon: there was ample proof of this phenomenon in our data. Moreover, selfassessments may have differing bases from person to person, and they may be influenced more
heavily by some characteristics (e.g., those that are more readily observed) or events (e.g., the
more vivid or unusual ones) than by others. In addition, personal traits such as modesty, selfconfidence, and defensiveness may color the appraisal and reporting of ones own competencies.
Self-reports involving more factual matters may sometimes be suspect also.
Contamination may result from forgetting (or selective remembering), responding according to

22
socially accepted norms, especially for some kinds of information, or deliberate falsification.
Nonetheless, such reports have proved to be of sufficient accuracy for a wide variety of
purposes. This applies in particular to reports of grades and other academic accomplishments
(Baird, 1976; Freeberg, 1988; Freeberg, Rock, and Pollack, 1989; Laing, Sawyer, and Noble,
1987; Sawyer, Laing, and Houston, 1988).
The writing samples that we collected have their own characteristic strengths and
limitations as validation criteria, both in terms of the samples themselves and the manner in
which they were evaluated. A concern with the course-related samples, for instance, is their lack
of standardization: the samples we studied were, obviously, written for a variety of different
purposes and audiences. A similar concern applies, to a somewhat lesser degree, to personal
statements. A potentially greater problem with personal statements, however, at least as
samples of writing, is the uncertainty that often accompanies their authorship. A majority of the
participants in our study acknowledged that they had obtained help at least in editing and
revising their statements, and many said they had received assistance with their initial drafts as
well.
Writing samples based on professors assignments, on the other hand, although
potentially the most compelling source of validity evidence, suffered still other shortcomings. As
mentioned earlier, although study participants were asked to submit tvoical samples of their
writing, they seemed to furnish the & examples (as suggested by the high grades that most
samples had received). As expected, the samples formed a heterogeneous collection, ranging
from an engineers memorandum on results of a study of centrifugal pumps to a paper on
applying neo-Aristotelian criticism to the beverage container recycling effort in California to a
personal account of the life and times of an exotic dancer. To the best of our knowledge, this
effort represents the first time that holistic scoring has been applied to such a variety of
disparate papers from various academic fields of study. Not surprisingly, then, some readers did
not feel fully confident, at least initially, in judging all of these pieces. Moreover, the holistic
scoring rubric used in the study did not fit all samples equally well. Stretching an all-purpose
rubric to encompass such a wide variety of tasks surely reduced the reliability of evaluations.
We were able to gather some rudimentary information on the nature of the writing
samples -- for example, when they were written and how much time was involved in writing
them. In retrospect, however, other, uncollected information might have been even more useful
in classifying the wide variety of writing samples submitted, and in identifying specific
circumstances in which the relationship between the judged quality of test essays and that of
writing samples would have been stronger. Information about the objectives of the assignment,
the directions that were given, the specific requirements (e.g., page length), and whether the
assignment was chosen by the student or specified by the teacher might have been constructive.
Some quite elaborate schemes have been devised for classifying writing assignments (e.g.,
Purves, Soter, Takala, & Vahapassi, 1984) and the application of these might have proved
informative. Breaking out the study sample by undergraduate field did suggest that these
relationships were greater in some fields (arts/humanities
and the social sciences) than in
others.
In addition, students reported the grades that their professors had assigned to the
samples. However, the shortcomings of academic grades -- for example, that they reflect
differential standards both among courses and among professors -- are well known. Further, the

23
grades assigned to the samples provided by study participants were undoubtedly based on
characteristics other than (or in addition to) the quality of students writing, and they were quite
restricted in their range, thus precluding very high correlations with any other variables.

Despite the myriad qualifications detailed above, the study does, we believe, constitute a
reasonable beginning to accumulating the kind of evidence required to establish the validity of a
GRE writing test for facilitating the selection of prospective graduate students. Indeed, even
with the limitations of the criterion indicators, two of the exercises investigated here had quite
respectable relations with composite indices of writing shill. The study results also provide clues
as to alternative methods of investigation and additional kinds of evidence that might be
pursued in any future efforts.
One kind of evidence that should be gathered eventually, we suggest, is information of a
more prospective nature. All of the information gathered here was either concurrent with or
retrospective to the writing that subjects completed for this study. Confirmation needs to be
provided in terms of the relationship of GRE writing exercises to students success in meeting
the writing demands of graduate study. In addition, a more focused sampling of schools and
departments within schools, instead of the broad-based representative sampling that was elected
for this study, would help in collecting more homogeneous samples of course-related writing as
validation criteria.

2
E

.
u

s
1
A nn
e o oa i tcr
ol ee et inso edit eh vtrr x e ,ae R
th rs c te i a Cs r se s , ae o x w e
rm c
i
y , e r r se n bd o ae oe e r n a
nt l rds
he yi r lr eour rc eoe u s sl s d fias t ea nm bs h l a el i
o
s w
e
cr
c
ra w o o
eiv rt e r t ho xht
eh f r h t r e ea o l e t sbn r u eho lacd ea r i ea vt a e d d ti
t rh c n r T
i or r e
l c o p eme ffa a sCh x o t
R e
t s w
hr a et n eaw w r.
aIt i e9 cd
4
h r
ef t r i i n ao t a f e e c r o e hm swp tat ro ien iw eoe srt B
xot- nehnw
ha mr ur hac et pi Ss s r e se ,
d
ma r o o i f e
l t he e ss o ft rt r ei w d rm ea o ater o rut . s e) fa 7t e 9m . hd ( o
aFi w u h sr t t
h
e
r
dl
a e dr b a eb t e s afemn
ip e e l io oo enyw e en rh s d t t e t i f cahm , a aot ovC e t w
Rr
i
rt
n w
h yo t i
ua
b la vm d.
eo
O
s
l
t
.
te n
t
c
ae
t cs n o r n

a hA
inf gt
i ra n o t fie cee on rta mhe ap l sia tm
tr a i r so e o e nn xe e w
g r ch c o i m
ts
lf c, g y n a io soa s s
e
r
o
eo
rnw n ao
l icv teda erytehC i ,a l ehhb wl or et
e oi n dt d i - c O
a as 3t oclOr o-s fei r0u i g.soo thrhi . d rft atol n v y mi

ep s u , t
as i .
r t

2
R

f r
. . lB o
La L (e if
U r
ae c nn a i r m
t n aE
x
E
p

o.

.lB
E

os 1srd
B

, it p e 9 n s
o
a

e
g pkr
r

e La L
v ( ei
D fr n 1 d
o a,f i
9
o d no
Ra
dG l m (a i nsB os R i oi o N
et na8 asp cP
r o ur

7 rt
d

oN e Y 6

s
uC e

d ow

7c eau ct mcav oue m

d9 nap frlt ig ese hd ) m oe n t n s
1di drN n .E et.c c e i - Jvt T r o e S
n
,1: e
R

h c n aa l B
cP .& M
i B. (
e S l ef e, l g1f a- ura pogs f ns ae 9 sl ms i m oes n 8tI k n
ir
f 9h o
s r e s h c a e et
e g a a ru g n an L
L d
3 3 e
1 9 3
a 4 _ 0
r.

se o

n a

.r B.
B
B

n a
t

H t M (n
R o
o

e T m d e1 fa i r hs urs o e9swe s ea s c e mA kem

8tre g en i I t I rl o 3i C e(l
N
e Ra8 n p E or 3 R
oo i N d -rt8 T aet.N
6c nonY a S3i, r C m
toe n . aE r x- ow
E k
a
r
d
.

ee .r B
tH M. B
u M .W . &
a .Ko
, R. M m
n( a
I ,f o n1 p on a c
e9 l v
s
n e so n oc m
o s eee r ot
vi
t
f w
a os e srot ( ea us r d tsa ti rmPG e
n Rh t t
l x a
n
oE
i
t B a
cP
ur oi dN n.E ec c e i a JvtT r o e Sn
.:r e
d s

n a l e .r B
H M C
. R a J.
R
m
o . J . M n, p
n.
o I ki & t r & ae oNc nnY a i r C m
t e n aE
x ow
E

n a
o

l e .r B
i
st
a

Me M
.,. , &
. os R
Bl

D
, ,.oA ( r
o
a I

H M . & Gs
t J. ( n a eA cf m, to 1sc me yos rpd i ea d an9irs i i ns r s
scf a Ju k d o i eE
ol . M
8I 2 1 .u- 9 11 1
r
6

a
t

c A ,
r

afone 7n
n ,

, m .l eI Bi n e JN ur.B Oe i h K eA.c & Sl . . , ,.

J D (
, (E
Gd
t s1 s Rm
. e
e
a
d
n
s h i ci ncr t a e e ro smeoa e rt r era tu adaf etna ti r oG n x. o d Rh
eE
. mn a or g i o r t p a
cP
ur
i dN n .E ec c e i Jvt T r o e S
n
,: e
s

h c r
. e c

h g

i
E

L S . &. lH
S
(

. F

r 9ea
r e
)
,s

ce i
t

9 ri
g

oc o w d t

a
e) T9
r ce

rn .n, Eor e im M
o tK
t &
i.n G i
.rD ( c
sT , e oa1 fd e w t a oau s9d aur r c q d c 8 e s s
R
G l (a . B
n) R o 7R o1 i - N
e 5t a8 ap
rcP oo ur d irdN n .Et. ec c e i Jvt T r o e S
n
,: e

. F e.
v N i E (r t
e g e I I o C (
B
i
m
a
x
E

k 1 wr
d e

E a
o
t

. .C
P L ( o
T fa: v1s ot s ih o le w i s bh9 aspc e rm
f aA rereE sn8 R
eeo t r G vi ( 4i r B e t
a e s e R
R l
N
ea 8R
n p o E oo i R 2 Ntr 8 Ta t. cP -oru Si4dN n E c. 1 e - Jt T o
i v r e S

v: )t
:3 I

:)

c 9
r

o w) ,
a i
5n
, :1 e

f 9 ut

l
s

i eI e p.L nW
co RG s 8 .sira r nE g e .0 n,
g
Ra . J ( yo
T fe , d a1 s w oh nm
,c y. o . 9 m
tC
nEo . e o p i wp d r ( i g 3. mr usHoa bin nl 1l Nr ilE . cs
bd i -J a t l) e e e :t, 5
f oA i
1oemr t cr & s s 9eeh i de
s t 8eba
v u
u P
8ee
Ii hs
d ) rat
R o
N
e Ra8 p E r 8 R
oo N d -r8 T et.N
5c onY a S8, r C toe n . E r - ow k
B
o
a
r
d
.

N,. Ek rRc a D
l oAI e. so& Piev cs vi J.l ,(kta fnepA , i, 1o r t bcr
s s 9 he
de
t e8
w e g eI l ePl o CI h(
s
aB
R sot
N
e R
ea8 p i E r 9 R
oo N d -ro8 T
t. N
8 o Yn S9,
ae c nn a i r C m
t n aE
x oE
BI
o
a I
r
d e .
g

s is
:1 I

vr
oe

u 9
. n r
e

a,G e n . t C
i l Ae ,M .
,
l n o
Nfi
t t h1 r n9wA os9e

gi

.
J: &mK e on n eoJd H
y ,r( . s . 1W se
9aih i ln9tn tc 5od
2orm
. a nt iE W:or t iC
a iPptD
U
aen .D
c gS u . d o E s
f

k l fan h s ed oeGn,
m
i . Fw I e ,S . . r
F u& C o fs f m W
a enaE , ( h. e. 1T
m9
6
n h c ro a e s:h epiRa ( r g kto n o M
. a r. ow n) N e o 6 i N
Ym
C a
oE xn t r l a En c le

ti t e n t l r a H, l
i
l et
a o rc
ro(
e

. R i T & .W
ihug r uad ncd ue a t vse

H
t.

o ,
l c

L. e ,J S . r a aR s & -N
we i i Jy u( t
l a
o
o c
f
i : e AC h I
T
A m e r i cC a n

t -a
sM f P A
e I
l . s io s y l ah n a
c
n
o

ai

o . , Mi
te

e
a
,
sn

r ,

ei

eu

n
a

9s, r m 9eo
,8
5

. o-D E w
& .F n ee oM
i r Er (wu . p s .rs l v Ero,
( G BR ER
9 3 - 2e 6 A ) .

e l eof a ocs
p

S .

a eiaH (e

i t .r. 1M
s mn ea l9 ub oe rc 9p

r s u 3 ,J.

da) oo E
1. eu f

i d A tr s p i

pne

.
.t o e

s ft

us n o i dAt m
a ce ure id cE a nn
8
.

o
t

aa

&

.d

s m
i

d . c n,
coa
u
: Ise eoAC w
8ph I a Tr

n -

me

2r M

c u

2
,
w
7p

Ie o

t iU ea v icpuJt (c el sij .b1aO h fv .9 eumo 7 o r ro5 o , r rp)f e sa .hl s i at E nl S c O L

I L P n .
o a .eB tS 4
Cg i D e nT
p
d n rn
. 7ogp( 9 onE
P1a
l-odi l 7 s shn 6a tk 9. s n e1y ) am s , Wa: t
sm oW

d c)s

hg ) s
8e i
g

y v
5

a A ( l e. 1Ta
9o, a dr 8w
4a , o)r
t
Re e s he ae i r ct , hh Tsnei al cm
z htoin nE g f f

-,aR L. w Jf Hy o ua s et lM
o n( , r .i 1Ae, c c u9 r oa ncs&v8 f i 8
g
o c
rf
s t ua d e( ,n A
t sR Ced syTe a Rr c h t eS
A m e r i cC a n oT
l eP r o lgs r a me t .
gi
en

ti

a fu

e
8e

e f 0s d t ) sw h . ari o
2
9
6
fs fs , s eis
o

ee ivl i t nci le j .b gO
t
n
az 1 l , 6
a.
S
en

.w

.ib
e o

.&
I . Wl I
,S G ( . e . 1V a l i9 do- is t y8 f a 2
o a) t b idf l A
. i rt ney : a
J v oo As u f nP r D n a l 2 a8 0,i - 2l 9 e6 .
d

rRe R ( i. o . 1F l
f9ail n v o7cylri v ge 6rtd,a nd euso) a t e t.rp
R e s e a Jr oc uh r nu a 1l , , 2
5
1
3

s,

o4efe ar h ) aib vt . c tit

ooyi E u fns
r gP

i ,P s . uA
s , C a Sr. . p ,Av a T . hoe aSa &s Vk t,
o g ar p s o l i mf c oys
c s s
a
8
.
53
1
m R

W r i t i1 n1 g , - , 9

oe. 1A
i nec cvr u9 r oai bc,s v8 gt f c 7 d al n),
s t u d e( n A
t s RCe s Te a Rr c h
oT
l eP r o lgs r a me t.
gi

t. t Mi Da n &l .dC r eail c kL lm

e (er , a . s1r V
Mc
u
d i E
n
2 2

i
a

,P

l 5o w
J xc t

o w
6
f a)r
e B
g

p8 r o b 0l W
e m h: )m a .t e o s
i
e 4 2 8 d1 ,- 2 9 1 .

A gi n i ns s e s t s a g n oi s ws er si ts A
i n g .

u lE s(
a . 1M e l 9
um e eH r
yj n bi g hu
osr c
l ete
o et p h s

em

s
5

W enWo i( h.r e .t 1Ti r c 9

As a
nP
e o
lM

o)

hr
o-

a
E

1l

t , W a . W Cr & . C aa r l sS ooBn (, e . e . 1A o rs 9 or o 8a of 4 fi h E )n ipn l iP. s l h : I (e C o I l l e g e

2 o
n - B .o R 4:o iS N T8kt e. E oa Rr w
N pn R
8 eo i N
4
oYmC a1
roE x6n t rl a )En c tle .
e
g
B
o
a
r
d
.

2
e

W
f i gr g iGen Ps w,( .d . m
u
t
s1
,
& &
S Fa
n J o s s ery - B a s s .

7
.

9. E . g & st& gpt h9u mme uao s l en

a
n

od 3
c

f
i

AppencJix A

A-l

GRE W

Y D MJ

LR T

QI

UA

IS

About the exercises you were given to write on today

1. Did you have enough time for: (Circle one number for each.)
Yes, more
than enough

Yes, just
enough

No, not
quite enough

No, not
nearly enough

I did not get

an exercise
of this length

A. The 20-minute exercises

B. The 40-minute essay . . .

C. The 60-minute essay . . .

2. Gverall, how would you rate the writing you did for the tasks you were given?
(1 = lowest to 6 = highest)
Seriously deficient

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

Weak . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Limited

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...3

Clearly competent

. . . . . . . . . . . . . . . . .. . . . . .. . . .

Well developed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Thorough and insightful

. . . . . . .. . . . . . . . . . . . . . . .

preparing for
b this test

3. How did you prepare for this test? (Circle one number for each statement.)
No

Yes

A. I spent time thinking generally about the topics that were sent ..............

B. I thought about specific points or concrete examples to cover for the topics ....

C. I brainstormed

1
1

G. I asked someone to read and comment on my essays .....................

H. I read the information

1
1
1

about ideas ........................................

D. I did some reading about the topics that were sent .......................
E. I wrote an outline for one or both topics

..............................

F. I drafted a sample essay for one or both topics

on how the essays are scored

I.

I read the exemplary essays you sent

J.

Other (please describe below)

Other:

.........................

....................

.................................

......................................

2
2

A
2-

ph

4 Wm

di

you

ow.

( r

W a

het

mi

oT
w
o

t f
F

n r

ar o

one

et

.
.
.

7 W

dir . you

.
.

.
.

, 8g Ra
e

ar

lt

.t

t. aa .

. ss . I

.s

you r so i

o.u

o ut
o.u

0 (

.
.

p tS Q

s.

s.

es

2
t

8 l i uo

.d

3 .

.d

i I.

tw
p

e?ti r

e. l

y4.

e.

nn

id

.
.

.
.

ti

.
.

.
.

.
.

sn

o ut

.o

r
r

..

..

. .-

. . 6.

. A

u0

u o. 0 ot u0

2 h

. .

4.

i
.
.

C .

. . c ..

3.

. -

2.

r .

1.

r 0. .

. .a

..

an
e

s . . 5 .s .

. .e

D o.. l

..

. 4

W
sa . not

g.

..

rhe i tpyou c p r sl fu w
oh e m
si a h
t ofr i e
i ur
n ca
mg l
None

little

A moderate
Y
amount

. . . . . . . . . .

i . . . .t . .

i 1

I did not provide a personal statement

g 2

div o ryou get ie dn

ce
b

B E

. . 7

A. Drafting

t .

1y

s. i

g nhet t p.o n i s
?ti d (
one i Cn

t
.l 3

s uo

n .

ss s .

i .

adi

d o e ?ti e

B . .

s s
a

p eog

sI

.A
.
-

t .

a g

t tf

t .2o h .

h M

.1
b

c u t. l .

3b .t 9 ho
A.

.
.

1b t .2 ho

n r.

1h

.l

ts

an

o o

uf

A .

as

c uo

m ro. t

ot.

lo

o
T

an

wl rn you

.n

. a rs .

. . . . . . . . . M
. . . .i . c . . .

e . uh

gao

ic un r t

. l

pbii e

you

t t .

i. or o t

e c m6 m
A uow
ohb i

n.

m rtt
etc,)

( at ih

ogao w .

not . b

i oc

s ur a

t?ti i

I d.i

T.

ra

i
gao o

te

w ot

o.

ha M

i w

t t

5 W
sa

hetb

3e

A substantial
amount

4
i 4

..........................0

nad

I
.

A-3

About your writing

9. Generally, how would you describe how quickly you are usually able to write reports, etc.?
Compared with most other students, I probably write:
1
Much more slowly ...............................................
2
Somewhat more slowly ...........................................
3
.................................................
Aboutthesame
.4
Somewhat more quickly ...........................................
5
Much more quickly ...............................................

10.Please think about your past experiences with tests that have required you to write within a limited
amount of time. How often have you felt pressured or frustrated because of difficulty finishing?
.
Almost always..
................................................
2
Usually .......................................................
3
Sometimes .....................................................
.
Seldom .........................................................
Hardly ever .....................................................

11. How do you think your writing compares with that of other students in your major field?

Wellabove average ...........................................

Somewhat above average .......................................
About average ...............................................
Somewhat below average ..........................................
Well below average ...............................................

4
3
2
1

iences with the process of writing

12. For each of the activities listed, please indicate how successful you have been in your college courses.
(Circle one number for each.)
5 = extremely successful
4 = quite successful

3 = somewhat successful
2 = not very successful
Ektremely
Successful

A. Thinking about an assignment (e.g.,

developing ideas, gathering
information)
. . . . . . . . . . . . . . . . . . .
B. Organizing (e.g., making outlines,
deciding on the order to present
ideas) . . . . . . . . . . . . . . . . . . . . . . . .
C. Revising (e.g., improving sentence

phrasing, rearranging ideas,

correcting grammar and punctuation

D. Developing an effective writing style

(e.g., expressing ideas clearly and in
interesting ways) . . . . . . . . . . . . . . . .

Quite
SUCC~fUl

1 = not at all successful

Somewhat
Successful

Not wxy
successful

Not at all
successful

A-4

About uxnpleting variow writing akgnmtmts

13. For each of the kinds of writing listed, please indicate how successful you have been in your college
courses. (Circle one number for each.)
Extremely

Quite

successful

Somewhat
succe.5sful

Not very
SUC~fUl

Not at all

I did not
do this

A. Personal writing (e.g.,

describing experiences or
feelings as in a journal) . .
B. Creative writing (e.g., a
poem or short story) . . . .
C. Persuasion (e.g., arguing
a position or writing a
letter to the editor) . . . . .
D. Analysis/criticism (e.g.,
revie&ng a book, movie,
article, theory, work of
art) . . . . . . . . . . . . . . . . .

E. Description/explanation
(e.g., describing an
experiment or how
something works) . . . . . .

F. Examination writing (e.g.,

long essay answers) . . . . .

G. Applied writing (e.g.,

preparing an application
or resume) . . . . . . . . . . .

About your grades

14. Approximately,

what grades or grade averages did you get (A) overall in college, (B) in courses that
required at least some writing, and (C) on the most recent assignment for which you had to write a report,
essay, etc. (If your college does not use letter grades, please mark the grade that is the closest equivalent.)
(Circle one number for each.)
09
w
MoiZcent
Courses
writing
Overall
requiring
assignment
writing
average
Grade
. .
A...........
A- . . . . . . . . . . . .
. .
B
. .
B-*::::::::::
. .
C
. .
C-Y:::::::::
D or less . . . . . . .

7
6
5
4
3
2
1

7
6
5
4
3
2
1

7
6
5
4
3
2
1

A-5
15. May we use, as examples in our research reports, samples of your writing?
identified by name. (Circle one number for each.)
--

You will not9 of course, be

No

Yes

1
1
1

2
2
2

A. Personal statement . . . . . . . . . . . . . . . . . .
B. Writing sample you brought with you . _ . .
C. Essays you wrote today . . . . . . . . . . . . . _ .
Your signature
Social Security Number
(for payment purposes)

---

Comments

--

----

If you have any comments about this study or about the writing exercises, please make them here.

Thanks for your help.

5 4 0 2 0 - 1 4 3 0 .2 Y