Computers & Education 119 (2018) 14–30

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

A mixed-methods study to identify effective practices in the

T
teaching of writing in a digital learning environment in low income
schools
Rebecca Jesson∗, Stuart McNaughton, Naomi Rosedale, Tong Zhu, Victoria Cockle
Woolf Fisher Research Centre, Faculty of Education and Social Work, The University of Auckland, Private Bag 92019, Auckland 1142, New Zealand

AR TI CLE I NF O AB S T R A CT

Keywords: This paper reports on the teaching practices identified as effective for students' writing progress
Digital pedagogy in a digital learning environment. The study is situated within a design-based research part-
1 to 1 environments nership between researchers and a group of urban schools serving culturally diverse students
Writing from low income communities who have implemented a digital pedagogy innovation which
Literacy
includes student device ownership, wireless access and a shared pedagogical approach. The re-
Implementation
search design logic was to select demonstrably effective teachers as ‘case studies’ in order to
understand what effective teachers in the innovation did that promoted greater progress in
writing. Qualitative analyses of selected teachers' class sites and students' individual blogs
identified features of teaching practice hypothesised to promote student development in writing.
To strengthen our understandings, teachers were interviewed to check the comprehensiveness
and validity of our interpretation. Classroom observations from these case study teachers were
compared with observations from a larger group of teachers to investigate whether identified
practices were differentially employed by these effective teachers. Finally, the effects on student
writing achievement of the relative presence of these practices in all observed classes were
predicted using a hierarchical linear model. Our findings indicate effects of using digital tools in
ways that promote complex compositional tasks, discussion and critical thinking. The study adds
to a growing number of studies that investigate the nature of effective pedagogy within a digital
environment. It contributes to the identification of promising practices for the design of more
effective instruction in writing within classes that have ubiquitous digital access.

1. Introduction

1.1. Review of related literature

In considering how learning to write might occur in digital classes we draw on a theoretical frame in which development is
conceived as a co-construction between learners and more expert others, through which learners come to know the goals, the actions
and the conditions relevant to recurrent activities (Gee, 2001) within the ongoing practices that occur within and across settings and
groups (Rogoff, 2003). In learning to write, such development entails increasing expertise in achieving one's communicative goals
(Kress, 1993). These activities are dependent on resources and mediated by tools in those settings. Within these relatively stable

∗
Corresponding author.
E-mail addresses: r.jesson@auckland.ac.nz (R. Jesson), s.mcnaughton@auckland.ac.nz (S. McNaughton), n.rosedale@auckland.ac.nz (N. Rosedale),
t.zhu@auckland.ac.nz (T. Zhu), v.cockle@auckland.ac.nz (V. Cockle).

https://doi.org/10.1016/j.compedu.2017.12.005
Received 27 April 2017; Received in revised form 10 December 2017; Accepted 12 December 2017
Available online 15 December 2017
0360-1315/ © 2017 Elsevier Ltd. All rights reserved.
R. Jesson et al. Computers & Education 119 (2018) 14–30

patterns, learners are conceived as active. Individually and collaboratively, learners are engaged in constructing new ideas and ways
of participating in activities by negotiating, internalising or transforming the meanings available to them, thereby becoming in-
creasingly agentive in their expertise (Göncü & Gauvain, 2012).
Given this conception of development, learners develop writing expertise by coming to know the goals and actions recurring in
the activity of writing, through interaction with others using tools and resources within that environment. Using this frame provides a
means to interpret the interventions that have commonly been identified as effective in supporting students to develop expertise in
writing, through meta-analysis (e.g., Graham & Perin, 2007; Graham, McKeown, Kiuhara, & Harris, 2012) and through narrative
review (Myhill, Fisher, Jones, Lines, & Hicks, 2008). These classroom practices can be characterised as developing expertise through
students' strategy use, for example in Self-Regulated Strategy Development (Graham & Harris, 2005) or their knowledge about
writing, for example through teaching of text structure (Rose, 2009). Increased expertise is supported through tools (e.g. word
processing) (Graham & Perin, 2007; Morphy & Graham, 2012) and discussion about writing (Myhill et al., 2008). Tutorial properties
of these approaches can be described as a combination of explicit instruction in aspects of writing, and in-task support while students
write (Graham & Harris, 2016). The tools and resources that teachers and students draw on include models, rubrics, indicators,
mnemonics and frames to support these activities (Myhill et al., 2008; Graham et al., 2012).
Whereas studies of effective practices focus on the planned teaching activity to support development, studies of effective teachers
foreground the role of the teacher to create an effective learning environment and design appropriately challenging, purposeful and extended
writing tasks (Gadd & Parr, 2017). Within this frame, researchers have identified a number of ways that teaching practices are effectively
employed. Langer (2001), for example, in a study of highly effective schools, identified that the teachers in those sites consistently provided a
supportive environment, which she described as characterized by balance, variety, and authenticity. Interaction patterns in these teachers'
classes were designed to extend children's thinking often in collaboration with peers, and included thinking about the nature of learning
through metacognitive conversations and connections. Similarly, Parr and Limbrick's (2010) study of six effective teachers of writing con-
cluded that a feature of the supportive environment provided by the teachers stemmed from clarity about the aims, specificity of the goals and
coherence between these goals and the activities in which children were engaged (Parr & Limbrick, 2010). Theoretically, these features are
supportive through tutorial properties which enhance the ability of learners to understand the goals and actions required to succeed in the
writing endeavor, and the provision of support and tools to achieve those goals.
Increasingly, digital environments are becoming part of the educational tools available to teachers and writers. Word processing
has been found to be a supportive tool for composition length compared with traditional writing modes (Bangert-Drowns, 1993) and
supportive of students having difficulty learning to write (Morphy & Graham, 2012). In digital environments, where all students have
access to digital devices, they also have access to the internet and the vast array of digital resources. Moreover, in such environments,
teachers can curate resources and support via class sites. Zheng, Warschauer, Lin and Chang's meta-analysis (2016) suggests an
average effect size of Cohen's d = 0.2 (e.g., Durlak, 2009) for writing achievement in 1-1 learning environments.
Arguably, in a digital environment, the changes in outcome are dependent upon changes in pedagogy to support writing (Zheng,
Warschauer, Lin, & Chang, 2016). It is argued that digital learning environments offer potential increases in autonomy, collaboration,
personalisation and creativity (McLoughlin & Lee, 2008). Studies identify increased writing quantity (Grimes & Warschauer, 2008)
and increased engagement in writing processes (Lowther, Inan, Ross, & Strahl, 2012; Yang & Wu, 2012). In terms of tutorial
properties, studies of digital learning environments identify potential for an increase in learning-focused interactions and higher level
thinking skills (Grimes & Warschauer, 2008; Lin & Dwyer, 2006; Somekh et al., 2007; Yang & Wu, 2012). In their survey of writing
teachers, Purcell, Buchanen, and Friedrich (2013) reported that teachers see benefits for connecting with an audience, collaboration
and creativity. These descriptions provide initial evidence that, within the digital environment, there is opportunity for the nature of
school writing activities to change given teachers' beliefs and the nature of their practices. There is also evidence that the nature of
the support to achieve writing expertise can be different, through increased in-task support from peers and teachers, (and arguably
the self) given the digital opportunity for ongoing iterations, and the opportunity to access resources and tools that support that
process (Sylvester & Greenidge. 2009). However, all these studies agree that the positive potential for writing can only be realised
through pedagogy that supports these enhancements, while avoiding associated pitfalls.
Theoretically then, more effective teaching will result from the extent to which the digital learning environment is employed in
ways that promote greater expertise in writing. Empirically, there is some evidence to suggest that in digital environments a number
of practices known to be effective in writing instruction can be enhanced through digital pedagogy. Studies identify increased
engagement in writing, either through more writing or through more activities that support writing development, such as revising or
evaluating one's writing or analysing other people's writing (Zheng et al., 2016). Similarly, studies document the possibility for shifts
in the nature of the writing that children undertake (Grimes & Warschauer, 2008), offering the balance, variety and authenticity
similar to those documented within the task environments provided by effective teachers (Langer, 2001). Moreover, changes are
documented for enhanced interaction between children and the groups they belong to, for example through greater opportunity for
discussion (Maninger & Holden, 2009) and responsive teaching (Russell, Bebell, & Higgins, 2004). Finally, more effective instruction
might arise from the greater opportunity for in-task support; from the teacher (Corn, 2009), through support from peers (McLoughlin
& Lee, 2008), or from the tools which students can draw on within writing activities.

1.2. Background to this project

New Zealand has a history of innovative and effective literacy programmes emerging from local innovations in which teachers,
either in professional groups or in association with researchers, design and implement new teaching approaches to become more
effective (McNaughton, 2011). The present study is situated within an improvement initiative in a self-formed group (cluster) of

15
R. Jesson et al. Computers & Education 119 (2018) 14–30

urban schools serving low income communities, with high proportions of indigenous Māori or Pacific Nations descent. The pro-
gramme is a digital learning initiative developed by the schools supported by a charitable trust. This initiative has formed a part-
nership with researchers to develop a design-based research collaboration. The overall shared objectives of the collaboration are to
increase student achievement so as to achieve greater equity, and in so doing identify what parts of the programme can and should
drive further improvement.
Within the innovative nature of the schooling initiative, new understandings about effective pedagogy in that context are needed,
so that novel approaches are not just innovative, but also educationally sound. To support that aim, in this article we use an analysis
of the pedagogy of six of the most effective teachers of writing within the innovation to describe the features of current practices that
are hypothesised to contribute to outcomes. We then test these hypotheses by checking whether the distribution of practices looks
different in case study teachers' classes than in a wider sample of teachers, and by testing whether use of these practices can be
associated with accelerated progress for students across one year. In doing so we seek to contribute to the body of work focussed on
how technology might contribute to educational processes and outcomes (Bebell & O'Dwyer, 2010).
Given the evidence that pedagogy is a key to improved instruction in digital environments, the current study seeks to use the
likely variability in teaching approaches within a digital learning innovation in order to understand how the approaches might best
be used to support progress in writing. The innovation has involved a research partnership using design-based research in iterative
cycles to design, test and redesign practices to enhance instruction. Our intention was to understand the effectiveness where it
occurred, so that observed effectiveness within the constraints of the particular context might drive further innovation. The reviewed
research and theory suggest specific sites within classes that offer explanatory power, namely: the nature and quantity of students'
engagement in writing, the effectiveness of tutorial properties, the in-task support through interaction and the use of tools and
resources. This explanatory power, we argue, will support further redesign and scaling within the immediate context, while si-
multaneously advancing theoretical understanding of the pedagogy that underlies successful digital innovations.

2. Methods

The study sits within a design-based research collaboration between researchers and a group of collaborating schools. The
partnership is intended to contribute research strength to an existing innovation by adding the needed explication, description and
redesign for innovative solutions in local sites (Anderson & Shattuck, 2012). The study has the simultaneous goals of advancing
theoretical understanding and intervening in responsive ways to meet needs within a specific context (McKenney & Reeves, 2013).
Rather than assess the overall effectiveness or impact of digital teaching as compared with previous or non-digital approaches, in this
particular study our aim was to identify the most effective practices from within the context, thereby supporting the schools' ability to
work toward increased use of such practices. In this regard, the approach was to develop capacity in the system for continuing to
change in ways that can be considered an improvement (Bryk, 2015; Fishman, Penuel, Allen, Cheng, & Sabelli, 2013). Our analytic
approach used the variability in achieving increased progress rates in student writing outcomes as a basis for checking the features of
more effective pedagogy. In this study, we use the class (and its teacher) as the unit of analysis, in an effort to explain the variability
in outcomes across classes.
The study employed a sequential mixed-methods design with four phases (Leech & Onwuegbuzie, 2009). We used the variability
in the schools' student achievement data across classes as a basis for selecting case study teacher participants. We then investigated
patterns of instruction which might explain that variability, using qualitative case study research methods, thus describing the
features of pedagogical innovation in those effective teachers' classes. Using the hypotheses generated by the case study analyses we
compared the relative frequency of practices within classroom observations of case study teachers with those of a wider sample of
teachers. Finally, we checked whether the features we identified could be associated with corresponding gains in student achieve-
ment in all classes observed across that whole year.
Evidence consisted of two primary sources, and three complementary sources. The two primary measures were repeated measures
of student learning and achievement, and observations of classroom instruction (of both ‘case-study’ teachers and a group of vo-
lunteer ‘comparison’ teachers). A further three measures were collected and analysed to complement and nuance our understandings
of the ‘case-study’ teachers' practices. These were: analysis of case study teachers' planning and task setting (delivered via class
Google sites); case-study teachers' students' individual blog posts; and post analysis debriefs with the case study teachers. Table 1
outlines the sequential phases, the measures collected, and their contribution to the study:

2.1. Context and participants

The programme occurs as an educational innovation in a group of schools in a predominantly low income suburb of Auckland,
New Zealand. The schools employ a digital learning environment, including 1:1 digital devices for all students from Years 4–13. The
teachers have adopted a shared pedagogy, in which the learners engage in composing multimodal digital creations which explain
their learning, and then post these on their individual blogs. Together the schools were seeking to raise achievement for their students
generally, and in particular in the underperforming area of writing. The cluster initially comprised six primary schools and one
secondary school, and case studies were selected from the primary schools, as in this context progress in writing could be more
defensibly attributed to the pedagogy of a particular teacher. Each of the schools is identified as being in the lowest band on
socioeconomic indicators and with high numbers of Māori (indigenous) and Pacific nations students. The majority of students in the
cluster were Pacific nations (64%), followed by Māori (26%), students of ‘Other’ ethnicities (7%), and New Zealand European (3%).
There were equal proportions of male and female students. Five of the six case study teachers were female.

16
 R. Jesson et al.

Table 1
Phases of research development.

Phase Project Year Purpose Evidence Analysis approach

1) Selection of case study Year 1 To identify effective teachers: students on average made high gains Student achievement data (e-asTTle) Average (difference from norm) achievement levels and
participants and achieved close to national averages by year-end in the previous from beginning and end of year. gains.
year (n = 6 teachers)
2) Case studies Year 2 To generate hypotheses about how effective teachers were using the Teachers' planning sites, students' Thematic analysis across the data sources.
digital learning environment to achieve gains for students blogs, teacher interviews.

17
3) Classroom observation
comparisons
4) Association between observations
and student achievement
Year 2 To test whether the case study teachers (n = 6) looked different Contrastive analyses of initial Comparison of relative frequencies, and tests for
from a wider sample of teachers (n = 16) using the hypotheses classroom observations. independence of distributions using Chi-Squares and
Fisher's Exact Tests.
Year 2 To test whether frequency of observed practices, hypothesised to Complete classroom observation Hierarchical linear model testing whether identified
support learning, can be associated with gains in student data (beginning and end of year) practices predict student achievement gain.
achievement (n = 606) in the total sample of teachers (n = 25) over Student achievement data
a year. (beginning and end of year)
Computers & Education 119 (2018) 14–30
R. Jesson et al. Computers & Education 119 (2018) 14–30

2.1.1. Achievement measures

To select our case study teachers, we drew on the achievement results on standardised tests writing from Year 4 (9 year olds) to
Year 8 (13 year olds) from the previous year. Student achievement data were collected from the schools, using the e-asTTle as-
sessment tool (http://e-asttle.tki.org.nz/) which is an online standardised assessment tool commonly used by schools in New Zealand.
The tool allows teachers to develop a customised, but standardised, writing test from a bank of prompts aligned to the curriculum
achievement outcomes for the New Zealand Curriculum (Ministry of Education, 2007). Tests are scored by teachers, for each of the
rubric elements and then entered into the tool as a ‘rubric score’ of 1–6 or 1–7 depending on the writing element. Based on the
teacher-assigned rubric score, elements are transformed to a scale score using Item Response Theory. Results are presented relative to
New Zealand Curriculum outcomes and a norming sample of performance for students in Years 1–10. Using the tool, the schools in
the participating cluster agreed on a common writing purpose with parallel tests for the beginning and end of each year.
The overall writing score is a composite measure of ideas, text structure and language, organisation, vocabulary, sentence
structure, punctuation and spelling. Moderation of writing scores resulted in high levels of agreement between teachers on the scores
for samples (< 90% scoring similarity in line with the test manual). Once collected, the raw data (students' scale scores) were then
transformed into a relative measure of achievement by subtracting each score from the normalised mean score for the students' year
level. This provided a norm-referenced, relative score.

2.1.2. Teachers
Twenty-five teachers in total were participants for this study: six were participants as ‘case study’ teachers; nineteen more vo-
lunteered to be observed teachers (sixteen in Term 1 and nineteen in Term 4). The teachers selected as instrumental case studies were
drawn from all those teachers with digital classes for whom repeated measures of student achievement data had been collected. Their
selection drew on two criteria. The first was a progress criterion; those in whose classes the average gain score across one year was
greater than average normative progress for their year level across the country. The second criterion was achievement level: those
classes whose average end of year scores were either above or not significantly different from that of the national norming sample.
Case study teachers were thus selected from those classes who had both comparatively high average levels (not different from the
national norming sample) and high average gains across one year (at or above national norms) in the previous year. Our achievement
analysis identified seven teachers as possible case study participants. Fig. 1 shows the seven possible case study participants'
classrooms' beginning of year and end of year achievement results in Year 1 of the project. In the figure, the circular dots represent the
average (mean) score of the classes, as different from the national sample for their respective year levels. The vertical lines represent
the range of achievement scores (as different from the national sample) achieved by the class as a whole as a 95% confidence interval.
Note that the line connecting the average scores are purely illustrative to show average gain over the year. While seven teachers were
selected as potential case study teachers, only six were able to be recruited as case study teachers for this project. These six case study
teachers agreed to in-depth qualitative class site and blog analysis as well as classroom observations in Year 2, which were used to
nuance our understandings about effective teaching and learning in a digital learning environment.
The number of teachers observed in each school depended on the size of the school, and therefore reflects the relative influence of that
school on overall cluster achievement. In small schools, only one teacher from each of the levels (4–6 and 7–8) was selected. In the largest
school, three teachers from each level (n = 6) were selected. The use of volunteers, while satisfying the voluntary nature of research
participation, also meant that observed teachers were likely to feel competent implementing the cluster-wide digital pedagogy, and so the
observations could likely capture features of pedagogy untroubled by the ‘teething’ issues of moving to 1-1 digital use.

Fig. 1. High level and gain classes for possible case study selection.

18
R. Jesson et al. Computers & Education 119 (2018) 14–30

2.2. Data sources

2.2.1. Case studies

The case study investigation focussed on the teaching in May and June (Term 2) of Year 2 of the project. Three strands of
qualitative analysis were conducted: a) analysis of case-study teachers' students' blog sites; b) case-study teachers' planning sites; and
c) case-study teacher interviews. Case-study teachers' blog sites were analysed by gathering (copying and pasting to a spreadsheet) all
evidence of the tasks that students were given, the texts that supported those tasks, and any resources or tools accompanying tasks.
The teaching was then tracked through to the teachers' students' blogposts. Blog sampling followed a purposeful approach, totalling
the blogging frequencies of all students per teacher, and selecting the two top bloggers for both months (n = 4 per teacher) for
analysis. We selected highly engaged bloggers on the rationale that these would offer greatest exemplification of the range of assigned
tasks. Each blogpost and associated digital creation was included in the corpus for thematic analysis.

2.2.2. Teaching observations

Observations of the volunteer (non-case study) teachers and the case study teachers were undertaken using an observation tool
developed by the research team to capture descriptions of the teaching across the schools (see Appendix One). The main purpose of
the observations within the wider collaboration was formative: to give teachers a picture of the overall nature of teaching that the
student body experienced, for review and redesign purposes. As such, descriptive analyses of the initial set of aggregated observations
were shared with the schools in Term 1 and formed the basis for ongoing professional learning conversations in schools. However, for
the purposes of this study, once case studies had been conducted, initial observations were reanalysed, for comparisons of activities
between the case study teachers' classes and the non-case study teachers' classes. Observations were repeated in the final term (Term
4) in the same year. This second round of observations were also used formatively with schools as with the initial observations. They
also served for use in a secondary analysis, to determine associations between teaching practices across a full year and student gains
in achievement in the Hierarchical Linear Model (HLM).
Within the observation tool, teaching was conceived to include both direct interaction with students and the provision of tasks
and resources using the class site (see Appendix 1). The observations were based on samples of 3 min intervals, in which the observer
alternated between observing the teacher (typically interacting with a group of students, but at times the whole class), and the
independent learning activities being engaged in by the group of students working away from the teacher (if applicable). During the
3 min intervals in which the teacher was observed, a judgement was made about the main teaching activity (which included: question
and answer sessions, lecturing, conferencing, extended discussion, or behaviour management). Observed instances of feedback were
noted. The teaching foci within each interval were also coded and this included any teaching about: item knowledge (e.g., naming
parts of speech); strategies (e.g., discussing ways to evaluate one's success); practicing taught skills (e.g., finding and underlining
topic sentences); activating prior knowledge (e.g., making connections to previous lessons or out of school knowledge); and critical
thinking/critical literacy (e.g., identifying flaws in argument logic/identifying positioning of groups by a text). During the 3 min
intervals observing students' independent activity, the observer recorded the types of texts, sites and the nature of the activities with
which individual or groups of students were engaged. The observer roved around the whole class, noting all activities observed,
thereby capturing the range of independent activities. Opportunities for student decision making were noted, as were patterns of
collaboration, and whether independent activity was managed digitally or by the teacher orally. Over the course of a lesson, six
intervals of teacher observations were collected, and the same number intervals observing the students working independently of the
teacher. This produced a total of 96 observation intervals of the non-case study teachers (n = 16) and 36 intervals for the case study
teachers (n = 6) at the initial timepoint in Term 1. From these records, descriptive quantitative analyses could show patterns of
pedagogy and task provision, as well as the range of tools drawn upon by students. Moderation of the observations was conducted by
the research team, by independently observing a lesson with each of the other observers, using a common timer. Six observations at
each timepoint therefore had two observers independently completing the observation form. These independent observations pro-
duced an agreement level of < 80% across the four observers. For each independent observation, identified disagreements were
argued over, with an endpoint agreement reached, so as to increase reliability over time.

2.3. Analysis

2.3.1. Case study teacher artifacts

For each of the teachers participating as case studies, evidence of their planning and task assignment was collected from their
class site, and compiled into a spreadsheet. We theorised that the nature of activities and tools assigned to students, and the supports
that they receive, would impact on their learning. Thus, for each case-study teacher, we collected evidence about the:

• nature of the tasks, tools, and supports provided;

• espoused learning intentions;
• descriptions of the desired products and genre;
• written modes and other multi-modal forms;
• metacognitive prompts, intertextual links, and levels of task/text complexity;
• process and crafting instructions;
• opportunities for open-ended learning, collaboration, and student decision making and agency.
19
R. Jesson et al. Computers & Education 119 (2018) 14–30

Within each category, the tasks assigned were linked through to the published creations within students' blogs. Within each of
these categories, teachers' and students' artefacts were analysed using an inductive approach, through open, then axial coding
(Strauss & Corbin, 1990) seeking to understand similarities and differences between the teachers, and using the literature on effective
teaching of writing as a guide to understanding how the digital pedagogy might enhance that effective teaching. The initial analysis
frame consisted of nine overarching analysis codes emerging across the categories (variety of forms, deliberate instruction, skills
focus, amplifying children's resources, lowering transaction costs, building language, student agency, collaboration, complexity).
However, within each of these codes, axial coding identified a number of recurrent themes. These themes emerged as our initial
hypotheses (see Results section). As a validity check on our interpretation, an interview was conducted with each case study teacher,
wherein we presented our themes, and each commented on whether they agreed that the themes represented a pedagogical focus,
and how they considered that focus in their teaching.

2.3.2. Comparative analysis of teaching observations

For each of the categories collected using the observation framework (i.e., teacher activity, feedback, teaching foci; student
activities, texts, sites, and collaborations), descriptive analysis considered the prevalence of each categorical variable as a percentage
of intervals containing those elements. In Phase 3, the frequencies of activities were compared between the sample identified as
effective teachers (n = 6) and the observation only teachers (n = 16) using descriptive and inferential statistical analyses. Chi square
analyses were used to identify the degree to which the differences in distribution were likely to represent a true difference between
the two samples.

2.3.3. Multi-level modelling

Due to the multilevel structure of the data, i.e., students nested within teachers and teachers nested within schools, multilevel
mixed models were employed to investigate how key practices of teaching writing impacted on students' achievement. Models were
estimated using R (Version 3.2.2, R Core Team, 2015) with the lmer() function from the lme4 package (Bates, Maechler, Bolker, &
Walker, 2015).
The model we used to understand relationships between identified observed aspects of writing instruction and student e-asTTle
writing achievement was built in an iterative fashion. We first fitted an unconditional intercepts model to the normed e-asTTle
writing gains to understand how the variance was distributed over the different levels (i.e., schools, teachers, students). We then built
a baseline model to control for variables that potentially moderated the relationship between aspects of writing tuition and student
writing achievements (gender, ethnicity, initial achievement). This model was then tested against the inclusion of classroom ob-
servation variables, which were entered into the model as numbers of intervals teachers or students were observed to be engaged in
the different activities.

2.3.3.1. Unconditional model. We added one random effect each for schools, teachers and students to the regression model. The
biggest variance component accounted for by the unconditional model was for students (n = 606; σ ̂ˆ2 = 7368; 79% of model
variance), followed by teachers (n = 25; σ ̂ˆ2 = 1925; 21% of model variance), and schools (n = 8; σ ̂2 = 0; 0% of model variance).
This indicates that the between-school group variation is not statistically different from zero. As a result, this component was
restricted to be zero in all subsequent analysis.

2.3.3.2. Baseline model. Descriptive analysis showed that there were important student-level variables that need to be controlled for
before assessing the effect of teaching practices, in particular students' gender, ethnicity, year level and initial (beginning of year)
achievement level. The baseline model served as the starting point against which the contribution of key aspects of teaching writing
were tested.
To assess the impact of observed key practices of writing tuition on students' e-asTTle writing outcomes, the variables in question
were added sequentially to the baseline regression model. Each model was tested against the baseline model using ANOVA to
determine which model was the best fit for the data. The best model fit for the data was the baseline model. While the inclusion of
teaching variables was not more parsimonious, because of the intention to understand the contribution of teaching, the full model
was developed and presented including some non-significant variables, acknowledging the attendant decrease in AIC/BIC through
the addition of further parameters, but intended to increase our understanding of the relative prediction of teaching practices for
student progress (see Table 2).

Table 2
Model fitting with additional explanatory variables.

AIC BIC

Model 1 7018.0 7057.7

(student level variables)
Full model 7025.7 7100.7
(student level and teaching practices)

20
R. Jesson et al. Computers & Education 119 (2018) 14–30

3. Results

3.1. Case studies

The case study analyses were used to generate initial hypotheses about why these teachers' classes might be able to higher rates of
progress than was average in the context. These qualitative analyses of sites and blogposts suggested a number of shared features,
which seemed likely to promote progress.
In all classes, students posted digital artefact to their blogs as a forum for publishing. A large volume of writing was a noticeable
feature of student blogs in all the case study teachers' classes. However, there was considerable variability between students in how
often they posted on their blogs. In the highest average posting class, for example, the maximum number of posts in one month was
35, however the lowest posting student only posted four times. The teacher of this class suggested that students were engaged in more
writing across all learning areas, and that this increased volume of writing was a contributing factor in her students' accelerated gains.
Self-initiated blog-posting was an additional feature of some students' blogs. These students included posts about issues important to
them beyond the school environment, such as an uncle's birthday or their family's holiday. Such recreational writing, according to
their teacher, was a result of the digital learning environment, and was previously unseen by the teacher. Case study teachers were
unanimous that blogging was also intended to provide a window for parents into students' lives at school, but teachers differed as to
the extent to which they thought this audience connection motivated students to post on their blogs. Some suggested that the more
public nature of blog publishing dis-incentivised lower achieving students; in contrast others described how a digital environment
motivated their students to write, unencumbered by poor handwriting, spelling and grammar constraints.
While blogging was a feature of the shared pedagogy across the initiative, case-studies also indicated that increased learning
might be the result of undertaking and completing complex compositional tasks. Case study teachers' sites revealed the provision of
multiple texts and creation activities requiring synthesis, comparison, or application of information from these. Students' blogs
provided evidence that they authored both traditional writing products, and digital learning objects (DLOs), which were multi-modal
compositions with communicative purposes similar to those typically assessed through writing (e.g., an animation explaining the
workings of the inner-ear). Blogs indicated that students had engaged in multi-modal authorship employing combinations of video,
presentation, animation and audio (e.g., flip-o-rama animation; science documentaries; recorded book reviews; vocabulary pod-
casting) To support these composition activities, teachers' sites provided students with links to resources and online reference
documents. These multiple sources potentially provided students with repeated exposure to language, content and text conventions,
which was seen when students appropriated vocabulary in their compositions (for example, a report about the sinking of the Titanic,
which included the nautical term ‘listing’).
Another feature of all the case studies was the promotion of ‘time on task’. In all the classes, teachers used a class Google-site for
planning, resource dissemination and lesson management. Teachers also reported that the design of the site allowed students to
efficiently access support, such as templates, models, resource links and ‘rewindability’ (the ability to access previous work and
resources) and without having to wait for teachers or other students. All the case study teachers made writing frames available for
download and students' blogs showed that they used these open-ended templates for their own purposes. Most teachers also employed
online writing rubrics, engaging students in personal appraisal of their own writing according to a criteria list.
In case study classes, teachers used their class sites to provide access to written or visual support. Case study teachers' interviews
indicated that they believed there was potential to be increasingly responsive to individual student needs as a result of knowing more
about where children were at. Students' blogs, functioning as online learning portfolios, provided opportunity to view individual
writing development over time, identifying patterns of weakness and aspects of improvement, as well apparent attitudes to writing
and the propensity to post. Teachers reported an enhanced ability to provide individual or group differentiation through the ease with
which they could, in one teacher's words, ‘tweak’ their class site to provide access to differentiated resources, and activities.
A further way that teachers felt they could be more responsive was through the use of feedback. Teachers reported that the digital
environment (e.g., using Google docs) meant that they could read and respond to students' writing in real time as students were
engaging in that writing. They also noted a shift in the form of their feedback, from simple praise to ‘commenting’ on a doc, which
students responded to and then marked as ‘resolved’ or erased. Teachers also reported that they often will display a student's writing
and their feedback immediately on the large screen, so that other students might also learn from another's writing as a model, or
benefit from that same piece of advice. Teachers tended not to use blog commenting as an opportunity for formative feedback; rather
these comments were typically descriptive and celebratory, rather than generative.
When asked specifically about in-task support, case study teachers identified peers as the greatest source of support, and the use of
digital means to develop what they called a “community of writers”. The use of blogging, which served as publishing and a digital
archive of writing was felt to facilitate ‘transparency’ amongst students, through sharing, having their writing used as models and in
contributing to the improvement of what one teacher described as a “critiquing writing community”.
The synthesis of these analyses generated six hypotheses about the nature of digital activities and practices that impact student
achievement in writing, and were therefore predicted to be observed more often in case-study teachers' classrooms. These hypotheses
were:
Hypothesis 1. Increased time spent writing, thinking about writing, or learning to support writing, is associated with increases in
achievement in writing. This hypothesis arose from the combination of the volume of student writing expected by the teachers in the
case study classes, and also their reported efficacy of the workflow, which might result in both increased time for students to engage
in learning, and also increased time for teachers to provide instruction or support.

21
R. Jesson et al. Computers & Education 119 (2018) 14–30

Hypothesis 2. Increased achievement in writing is associated with practice in crafting compositionally complex texts, through
mixing, recrafting, and synthesising across multiple texts. Students are expected to navigate texts provided in a range of modes, and
complete a range of open-ended activities.
Hypothesis 3. A well-managed digital learning environment will show increases in student time spent ‘on task’, as the class plan and
associated resources are all easily accessible by the students via the class site, and students will be able to work at their own pace.
Hypothesis 4. Similar to Hypothesis 3, another affordance of the class site is decreased teacher time spent on instruction and lesson
set-up, and increased teacher responsiveness due to greater awareness of student's strengths and weaknesses in writing, allowing for
greater individual or group differentiation.
Hypothesis 5. In well-managed digital environments, teachers will be able to be more responsive to student's writing via the use of
timely, specific feedback, which is generative in nature.
Hypothesis 6. In well-managed digital environments, students will have increased opportunities to collaborate with their peers,
either online through collaborative work on a shared document, or face-to-face.

3.2. Classroom observations

Hypothesis 1. Increases in writing achievement are associated with increases in time spent writing, or completing activities that
support writing.
We hypothesised that there would be differences in the amount of time spent writing, thinking about writing, and/or learning to
support writing in case-study teachers' classrooms. However, chi square analysis revealed no differences in proportions of time spent
engaged in extended writing between the two groups X2(1, N = 264) = .01, p = .94.
Hypothesis 2. Increases in writing achievement are associated with increased time engaged in complex text composition using a
variety of modes, and open-ended practice activities.
The comparison of the observations of case-study and non-case study teachers' classrooms indicated that, in case study classrooms,
students were more frequently engaged in more ‘open-ended’ compositional activities compared with students in non-case-study
classes (see Fig. 2). Such activities included: creating Digital Learning Objects (DLOs), using open-ended writing templates, or other
extended writing activities. In contrast, students in non-case study classes were more frequently engaged in activities classes as
‘narrow’, including use of constrained practice worksheets, navigating or organising, or playing a skills practice game. The difference
between the two distributions was significant (X2(1, N = 264) = 12.41, p < .001). In case study teachers' classes, there was some
evidence of increased connections across contexts using the blogs. Both written and multi-modal authoring were apparent in the case
study classes and there was evidence of links to popular culture and non-traditional genres in these authorship opportunities.
Hypothesis 3. Increases in writing achievement are associated with more time spent ‘on task’, as opposed to ‘off-task’ due to
behaviour or resources management.

Fig. 2. Comparison of open-ended and closed activities for students in case-study and non case-study classes.

22
R. Jesson et al. Computers & Education 119 (2018) 14–30

Fig. 3. Percentage of intervals in which given independent activities were observed.

While ‘time on task’ was not a variable easily captured by the observation tool, some support for the hypothesis of promoted time
on task was provided by the differential amounts of time students spent working on different activities. For example, students in non-
case study teachers' classes spent 30% of the observed intervals finding and organising resources, and navigating across different
sites, compared with only 14% of observed intervals in the non-case study classes (see Fig. 3). However, there were no statistically
significant differences in proportions of activities between the two groups (X2(1, N = 264) = 0.94, p = .33).
Hypothesis 4. Increased writing achievement is associated with increased individual or group-level differentiation.
We were not able to assess this hypothesis through the classroom observations, as evidence of differentiation was apparent from
the teachers' class sites, and learning differentiation is not a feature of the classroom observation tool. However, we observed no
evidence of differences in proportions of time spent by teachers using differential teaching methods between the two groups (X2(6,
N = 264) = 3.41, p = .76). Case study teachers spent on average as much time giving instructions as non-case study teachers. They
also spent similar amounts of time, asking questions, and modelling. While case-study teachers spent slightly more time roving to
support students who were working on tasks and in conferences and discussion and less time managing behaviour, differences
between the two distributions were not significant (see Fig. 4).
In terms of the tutorial properties, differences in teaching were more apparent in the focus of the teaching interactions (e.g., item or
strategy teaching, practice opportunities, activating prior knowledge (APK) and criticality). As illustrated in Fig. 5, when case study teachers
were working with children, they were less often observed focusing on supporting students to practice taught skills (most often seen where
teachers worked through set activities with students). Instead they were more often observed engaging students in conversations requiring
critical thinking about logic, credibility or positioning in texts (X2(4, N = 264) = 12.75, p = .01).
In addition, classroom observations revealed different patterns of independent student activity (see Fig. 6) X2(7, N = 264) = 31.44,
p = .001. In case study classes, students' independent activity revealed differential use of apps and sites to support such creation activity
(creation tools), including video, photograph, animation, design and mind mapping tools. While blogging was a feature of the case study
classes, the patterns of independent activity showed that students in non-case study classes spent significantly more time blogging, while
students in case-study teachers' classrooms spent comparatively more time composing and creating away from their blogs, and comparatively
little class time either posting or reading blogposts. Another notable difference in the use of tools was the absence of learning games in case
study teachers' classes with far more time allocated to student creation1 (see Fig. 6).
Hypothesis 5. Increases in writing achievement are associated with appropriate, timely feedback, which is generative (also known as
feed-forward).
We observed slightly smaller proportions of time spent in purely evaluative feedback (e.g., “well done”) in case study teachers'
classes, possibly aligning with the finding that the focus was less often on practice of taught skills (see Fig. 7). However, we observed
equivalent levels of descriptive (describing the qualities of the writing), and slightly less generative feedback (often called ‘feed-
forward’ by teachers). However, contrary to our hypothesis, the differences between the distributions were non-significant (X2(2,

1
Games were seen in one case study teacher's class as the topic for written ‘game reviews’, rather than as practice activities for skills.

23
R. Jesson et al. Computers & Education 119 (2018) 14–30

Fig. 4. The percentage of intervals wherein the named teaching method was employed for most of the interval.

N = 121) = 1.82, p = .40.) It may be that feedback occurred which was not captured in observations, either due to timing (because
teachers reviewed student work out of school hours), or as teachers suggested, because students erased it, having resolved the
comments.
Hypothesis 6. Increased achievement in writing is associated with increased opportunities to collaborate with peers in a “community
of writers”.
We observed opportunities for collaboration in the case study teachers' classes. The creation of DLOs, the use of animation, video
and voice-over meant that group/partnership work was common. Co-authorship of writing was also common, with students amal-
gamating independently written text sections, or writing simultaneously using Google docs (computer mediated collaboration),
editing and co-authoring a primary text on one device (face to face collaboration), or deconstruction/construction of sample texts as a
class. However, as indicated in Fig. 8, traditional forms of collaboration dominated, with students talking to one another (face to face)

Fig. 5. Percentage of all intervals in which each teaching focus was observed.
24
R. Jesson et al. Computers & Education 119 (2018) 14–30

Fig. 6. Percentage of all intervals in which each student tools were observed.

about their work (single author). Differences between the distributions of collaborative types as shown in Fig. 8 were not significant
(X2(3, N = 73) = 3.71, p = .29). Notably, almost 50% of the observation intervals in case study teachers classes included some form
of multiple authorship, this occurred in non-case study teachers classes in 28% of observed intervals.

3.3. Hierarchical linear modelling

The results of the Hierarchical Linear modelling are present in Table 3. The model suggests a number of effects that are features of
the students themselves. Across our whole sample, students in the total sample of observed teachers' classes (case study and non-case
study) gained approximately 10 fewer e-asTTle points as they increased in year level: younger classes made more gains than older
ones. Boys made 15.74 e-asTTle points less gain on average than girls. The model also suggests that in all our participating classes,
lower initial achievement was associated with higher gains, with each increase of a single point in beginning of year score being

Fig. 7. Percentage of all intervals in which each feedback type was observed.
25
R. Jesson et al. Computers & Education 119 (2018) 14–30

Fig. 8. Percentage of all intervals where any collaboration types were observed.

associated with a small decrease (0.33) in gain over the year, indicating that across the classes, teachers were, on average, able to
make slightly better gains with lower achieving students.
In terms of the practices that were hypothesised to make a difference using the case study teachers' practices, being in a class
where students engaged in more time writing was estimated to have a large effect. When student level variables are controlled for, an
increase in one interval where extended writing was observed was associated with an increase of 51 e-asTTle writing points (relative
to norm) over the year. This is equivalent to approximately a year's gain in addition to norm (see https://e-asttle.tki.org.nz/Teacher-
resources for a detailed breakdown of the norms). The model also suggests an extremely large effect of being in a class which had
more intervals in which the teaching focus was critical thinking or critical literacy. Numbers of intervals where instruction for critical
thinking was observed were low, and the effect is not statistically significant (p = .12), however the estimated size of the effect where
it was observed (74 e-asTTle points in addition to norm) is worth noting.
The model also indicated an unexpected finding, not featured in our initial hypotheses. It suggests that increased frequency of
intervals including some form of student decision making (agency) was associated with decreases in writing gain. However, there is
no information here about the relative quality of choices. We have some evidence therefore to suggest that giving students choices is
not necessarily conducive to greater achievement in writing, and that being in a class where more student choice was observed was
associated with relative loss. Possibly, effects were not associated with choice per se, but over what choices students had, and what
they choose to do. In our data, being in a class that had greater numbers of intervals where students were given choices of some sort
was associated with significant losses of any other cumulative effects.

Table 3
Contribution of observed practices on gain in e-asTTle writing score across on year.

Estimate Std.Error t.value p.z

(Intercept: NZ Maori/Girl) 86.85 42.34 2.05 0.04

Year Level −10.68 4.83 −2.21 0.03
Gender: Male −15.74 6.38 −2.47 0.01
Ethnicity NZ European −23.84 20.29 −1.18 0.24
Ethnicity Others 17.49 12.72 1.37 0.17
Ethnicity Pasifika −1.42 7.45 −0.19 0.85
Difference Norm Beginning −0.33 0.02 −13.51 0.00

Digital Learning Object 2.12 35.49 0.06 0.95

Open Ended Template 40.20 30.61 1.31 0.19
Extended Reading Multiple 20.39 41.93 −0.49 0.63
Extended Writing 51.06 26.04 1.96 0.05
Conferencing/Extended discussion 23.82 52.07 0.46 0.65
Strategy 12.22 49.92 −0.24 0.81
Critical 74.00 47.78 1.55 0.12
Agency −76.09 31.76 −2.40 0.02

26
R. Jesson et al. Computers & Education 119 (2018) 14–30

In summary, qualitative analyses arising from analysis of case studies provided us with six hypotheses of effective practices in digital classes.
To check whether these practices could be discerned in the classroom observations of case study teachers compared with non-case study
teachers, frequencies of observed practices at the beginning of the year were compared. Some evidence in support of hypothesis one was
found: in case study teachers' classes, more intervals were noted where students were engaged in creating DLOs, working with templates or
engaged in writing. There were fewer intervals of time where students were engaged in constrained worksheet tasks or practice using games.
The observations also provided some evidence that students in case-study teachers' classrooms spent increased amounts of time engaged in
more open-ended tasks, as compared with the students in non-case study teachers' classes. Classroom observations also gave some support for
hypothesis three, with relatively fewer intervals of time in which students were observed to spend time navigating and organising their
documents. However, evidence against the hypothesis was also found, with case study teachers using similar teaching activities as non-case
study teachers (e.g., Q & A; lecture/model), and similar kinds of feedback. We found some evidence in favour of the fourth hypothesis
through the types of teaching foci that were observed: case study teachers spent greater proportions of their time working with students
around critical thinking or critical literacy, and comparatively less time supervising practice. Some additional evidence was found in the
number of intervals in which students used creation tools rather than learning games. There was no evidence of a difference in feedback
provided to the students between the groups (hypothesis five). Evidence in favour of hypothesis six was found in the number of intervals in
which students were jointly authoring texts.
In the final phase of the study, hierarchical analyses were used to estimate the effects of the prevalence of the identified practices
across both observations in the year on students' achievement gains that year. Support for two of the hypotheses was found. A
significant effect was found for classes where students were more often engaged in extended writing, and an effect approaching
significance was found for classes where students were more frequently engaged in critical thinking discussions with their teacher. A
previously unexamined negative effect was found for the presence of decision making, although the data do not tell anything about
the nature of the student choices that made up the observed decision.

4. Discussion

Our findings are consistent to some degree with other studies that have investigated the effective teaching practices in writing (Graham &
Harris, 2016), effective writing interventions (Graham et al., 2012), and effective teachers of writing (Gadd & Parr, 2017; Langer, 2001; Parr
& Limbrick, 2010). Like those studies, our findings suggest the importance of engaging in writing and in engaging in critical discussion. Unlike
these studies however, our findings suggest shifts in the nature of what is composed, the authorship of the writing, the use of time, and related
to that, the use of the digital tools for independent activity. While the findings may resonate with research about writing, differences also
align closely to the more open ended dialogic and multi-modal themes within digital research.

4.1. The nature of the writing that is done

Our study provides some evidence in support of the claims that the nature of writing can change in digital learning environments. Calls for
a more expansive definition of writing align with a view of a writer as designer, by combining images, graphics and sound with text
(Merchant, 2007; Sheridan & Rowsell, 2010). Whereas all classes used blogs for publishing, in case study teachers' classes, the writing lessons
themselves allowed for greater engagement in open ended activities, including both traditional writing (word processing) and multi-modal
composition. Like Gadd and Parr (2017) we found that effective teachers of writing provided students with open and complex tasks; and like
Warschauer (2008) we saw the opportunity for a broadening of the conception of the nature of these compositional tasks in digital en-
vironments, with the provision of these broader tasks by teachers who still get increased gains in standardised tests. Our data also suggested
that student decision making might be associated with decreased progress. The meta-analyses of the effects of increased agency on
achievement only find effects where classroom tasks and games provide complex authentic tasks (Karich, Burns, & Maki, 2014). This
expanded conception of ‘writing’ explored by researchers interested in digital literacies (Sheridan & Rowsell, 2010) coincided in our study
with an increase in the open-endedness, and therefore potentially the challenge, of compositional opportunities. However, while the op-
portunity for this broadening was arguably available to all teachers, there was differential uptake, with many teachers engaging students in
constrained or practice-based activities when working independently of the teacher.
A more prosaic, but arguably important, finding is that effective teaching enabled students to engage in more writing, evidenced
by greater writing volume and more time spent in activities likely to support writing, including the wider multimodal composition.
This aligns with research that suggests more time spent writing increases writing expertise (Graham et al., 2012); with research in
digital classes suggesting that students with computer access write more (Grimes & Warschauer, 2008), and with studies which
suggest that much of the contribution of computers in classes is composition and editing of texts (Zheng et al., 2016). However, the
inclusion of more creative approaches to composition, including DLO, alongside extended writing and open ended templates builds a
picture of increased expectation for communicating with an audience as the primary purpose for that composition, rather than the
practising of skills through worksheets and games.

4.2. The tutorial properties

While there are claims for the potential of digital technologies to offer opportunities for feedback and personalised instruction
(McLoughlin & Lee, 2008), our findings suggest that the effective teachers were distinguishable in that they capitalised on the digital
environment by engaging students in discussions to build critical thinking and critical literacy. Given the similar prevalence of
discussion activities across teachers, the differences between case study teachers and others were noted in the focus (qualities) of the

27
R. Jesson et al. Computers & Education 119 (2018) 14–30

discussion, in that critical thinking was the purpose for discussion. This focus on criticality raises the academic expectation in the
classes for exploring quality argumentation, reasoning in texts, but also aligns with arguments for the increasing necessity of young
people's critical awareness about power and positionality of texts (Merchant, 2007).

4.3. The support for writing

Like studies of effective teachers and effective teaching practices, our findings build a picture of highly supportive environments
for students while they write. Using the class site, this included curated provision of tools, resources, and tasks. Arguably, a shift in
the focus of teaching interactions, from the explicit teaching of strategies and practising of skills towards more critical thinking, could
have been enabled by a digital teaching environment where tools, rubrics and frames offer the potential for the sorts of direct
instructional support traditionally supplied face to face. This is a question of careful tool use: moving the more scaffolded instruc-
tional support to the class site so that face to face interaction can focus on extended critical engagement. This finding highlights the
increased power of teachers to effect learning for those they are working with, and to concurrently design highly effective learning
opportunities through task design and provision of supports on their class websites independent use when students are not with the
teacher. Similarly, in-task support through collaboration was high. While support of students helping one another was ubiquitous,
case study teachers more often allowed or encouraged multi-authorship, involving both face to face conversation and computer
mediated co-composition, arguably a more authentic form of composition in modern digital environments (McGrath, 2011).

4.4. The tools and resources

In case study teachers' classes, students were, as a group, using a wider range of tools and apps for creation. Although there is not
any in depth analysis of how well students used these sites, or what they used them for, the classroom observational data suggest a
picture of wide and flexible creation and compositional activities. Notably, in case study classes students were less likely to be playing
games, including online learning games, arguably narrower in focus, and more controlled or constrained. Possibly, the case study
teachers can be characterised as allowing greater creative use precisely because they were competent teachers, confident in their
expertise, and having high expectations of students' abilities to use the tools in open ended ways. This might contrast with approaches
that focus on keeping students busily engaged in more constrained practice. Further research is required to ascertain whether the
purpose of the tool is regarded differently by students and teachers in higher gain classes.

4.5. Limitations

Our study was based in an innovation designed to address underachievement and promote school change. Thus, there may be
limited generalisability to higher socio-economic or higher performing contexts. Moreover, the context of New Zealand teaching is
one in where teachers have considerable autonomy over what and how to teach, given a wide curriculum and a conception of
teaching as inquiry (Ministry of Education, 2007). While this leads to considerable freedom to innovate, it potentially leads to
variability in teaching. The study is therefore likely to be most applicable to other contexts where teachers have equivalent levels of
autonomy, and are able to adjust their practice, based on descriptions of highly effective teaching.

4.6. Future directions

The present study sought to identify how the digital learning environment might be used to support increased achievement in
writing. To do so, we investigated the practices of highly effective teachers of writing in a context where each student had digital
access. Findings suggest the power of teachers to provide supportive environments that allow a range of compositional activities,
including traditional writing, that foster increased amounts of time spent writing and to engage students in critical thinking.
The prevalence of multi-modal creation and multiple-authored composition in the pedagogy of effective teachers of writing
creates somewhat of a new question about why these sorts of composition are employed by effective teachers, but did not in
themselves predict achievement on standardized tests. While it makes intuitive sense that compositional skill generally might be
fostered through collaboration and might also transfer across modes, this is an area requiring further investigation. In our data, the
creation of DLO did not in itself significantly predict higher gains in writing, however, like discussion (and possibly student decision-
making), it may be that the underlying qualities of the activity are more educationally powerful than the observable form. It seems
likely from our study that the teaching of writing is a multifaceted art, which includes providing and supporting multiple and
extended opportunities to compose and to talk critically about the content of that composition with one's collaborators and teachers.

Acknowledgements

The Education Trust of the participating schools and the University of Auckland provided funding for the research reported here.
We are indebted to the students, families, teachers and principals and their Boards of Trustees for enabling this research to take place.

Appendix 1. Classroom observation tool

28
R. Jesson et al. Computers & Education 119 (2018) 14–30

29
R. Jesson et al. Computers & Education 119 (2018) 14–30

References

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. http://dx.doi.org/10.
3102/0013189X11428813.
Bangert-Drowns, R. (1993). The word processor as an instructional tool: A meta-analysis of word processing in writing instruction. Review of Educational Research,
63(1), 69–93.
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. http://dx.doi.org/
10.18637/jss.v067.i01.
Bebell, D., & O'Dwyer, L. (2010). Educational outcomes and research from 1:1 computing settings. The Journal of Technology, Learning, and Assessment, 9(1), Retrieved
01/02/2017 from https://files.eric.ed.gov/fulltext/EJ873675.pdf.
Bryk, A. S. (2015). 2014 AERA distinguished lecture. Educational Researcher, 44(9), 467–477. http://dx.doi.org/10.3102/0013189X15621543.
Corn, J. (2009). Evaluation report on the progress of the North Carolina 1:1 learning technology initiative (year 2) (NC state board of education report)Raleigh, NC: Friday
Institute for Educational Innovation, North Carolina State University.
Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34(9), 917–928.
Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. O. R. A. (2013). Design-based implementation research: An emerging model for transforming the
relationship of research and practice. National Society for the Study of Education, 112(2), 136–156.
Gadd, M., & Parr, J. M. (2017). Practices of effective writing teachers. Reading and Writing, 30(7), 1551–1574. http://dx.doi.org/10.1007/s11145-017-9737-1.
Gee, J. P. (2001). Reading as situated language: A sociocognitive perspective. Journal of Adolescent & Adult Literacy, 44(8), 714–725.
Göncü, A., & Gauvain, M. (2012). Sociocultural approaches to educational psychology: Theory, research, and application. Apa Educational Psychology Handbook, 1,
125–154.
Graham, S., & Harris, K. (2005). Writing better: Effective strategies for teaching students with learning difficulties. Baltimore: Brookes Publishing Company PO Box 10624,
MD 21285.
Graham, S., & Harris, K. (2016). A path to better writing. The Reading Teacher, 69(4), 359–365. http://dx.doi.org/10.1002/trtr.1432.
Graham, S., McKeown, D., Kiuhara, S., & Harris, K. R. (2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational
Psychology, 104(4), 879–896. http://dx.doi.org/10.1037/a0029185.
Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools – a report to Carnegie Corporation of New
YorkWashington, DC: Alliance for Excellent Education.
Grimes, D., & Warschauer, M. (2008). Learning with laptops: A multi-method case study. Journal of Educational Computing Research, 38(3), 305–332. http://dx.doi.org/
10.2190/EC.38.3.d.
Karich, A. C., Burns, M. K., & Maki, K. E. (2014). Updated meta analysis of learner control within educational technology. Review of Educational Research, 84(3),
392–410.
Kress, G. (1993). Genre as social process. In B. Cope, & M. Kalantzis (Eds.). The powers of literacy: A genre approach to teaching writing (pp. 22–37). London: Falmer Press.
Langer, J. A. (2001). Beating the odds: Teaching middle and high school students to read and write well. American Educational Research Journal, 38(4), 837–880.
Leech, N., & Onwuegbuzie, A. (2009). A typology of mixed methods research designs. Quality and Quantity : International Journal of Methodology, 43(2), 265–275.
http://dx.doi.org/10.1007/s11135-007-9105-3.
Lin, H., & Dwyer, F. (2006). The fingertip effects of computer-based assessment in education. TechTrends, 50(6), 27–31. http://dx.doi.org/10.1007/s11528-006-
7615-9.
Lowther, D. L., Inan, F. A., Ross, S. M., & Strahl, J. D. (2012). Do one-to-one initiatives bridge the way to 21st century knowledge and skills? Journal of Educational
Computing Research, 50(6), 1–30.
Maninger, R. M., & Holden, M. E. (2009). Put the textbooks away: Preparation and support for a middle school one-to-one laptop initiative. American Secondary
Education, 38(1), 5–33.
McGrath, L. (Ed.). (2011). Collaborative approaches to the digitalLogan, UT: Computers and Composition Digital Press/Utah State University Press. Retrieved April 6,
2017 from http://ccdigitalpress.org/ebooks-and-projects/cad.
McKenney, S., & Reeves, T. C. (2013). Conducting educational design research. Routledge.
McLoughlin, C., & Lee, M. J. W. (2008). The three P's of pedagogy for the networked society: Personalization, participation, and productivity. International Journal of
Teaching and Learning in Higher Education, 20(1), 10–27.
McNaughton, S. (2011). Designing better schools for culturally and linguistically diverse children: A science of performance model for research. London: Routledge.
Merchant, G. (2007). Writing the future in the digital age. Literacy, 41(3), 118–128. http://dx.doi.org/10.1111/j.1467-9345.2007.00469.x.
Ministry of Education (New Zealand) (2007). The New Zealand curriculum. Wellington, N.Z: Learning Media.
Morphy, P., & Graham, S. (2012). Word processing programs and weaker writers/readers: A meta-analysis of research findings. Reading and Writing, 25(3), 641–678.
http://dx.doi.org/10.1007/s11145-010-9292-5.
Myhill, D., Fisher, R., Jones, S., Lines, H., & Hicks, A. (2008). Effective ways of teaching complex expression in writingDepartment for Children, Schools and Families
Research Report DCSF-RR032. University of Exeter.
Parr, J. M., & Limbrick, L. (2010). Contextualising practice: Hallmarks of effective teachers of writing. Teaching and Teacher Education, 26(3), 583.
Purcell, K., Buchanen, J., & Friedrich, L. (2013). the impact of digital tools on student writing and how writing is taught in schools. PEW Research Centre. Retrieved on 30
November 2017 from http://www.pewinternet.org/2013/07/16/the-impact-of-digital-tools-on-student-writing-and-how-writing-is-taught-in-schools/.
Rogoff, B. (2003). The cultural nature of human development. Oxford: Oxford University Press.
Rose, D. (2009). Writing as Linguistic Mastery: The development of genre-based literacy pedagogy. In R. Beard, D. Myhill, J. Riley, & N. Nystrand (Eds.). The SAGE
handbook of writing development (pp. 151–166). London: Sage Publications Ltd.
Russell, M., Bebell, D., & Higgins, J. (2004). Laptop learning: A comparison of teaching and learning in upper elementary classrooms equipped with shared carts of
laptops and permanent 1: 1 laptops. Journal of Educational Computing Research, 30(4), 313–330. http://dx.doi.org/10.2190/6E7K-F57M-6UY6-QAJJ.
Sheridan, M. P., & Rowsell, J. (2010). Design literacies: Learning and innovation in the digital age. Routledge.
Somekh, B., Underwood, J., Convery, A., Dillon, G., Jarvis, J., Lewin, C., et al. (2007). Evaluation of the ICT Test Bed project final reportBecta. Manchester Metropolitan
University: Centre for ICT, Pedagogy and Learning Education and Social Research Institute.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques (3rd ed.). Newbury Park, CA: Sage Publications.
Sylvester, R., & Greenidge, W. L. (2009). Digital storytelling: Extending the potential for struggling writers. The Reading Teacher, 63(4), 284–295.
Warschauer, M. (2008). Laptops and literacy: A multi-site case study. Pedagogies: An International Journal, 3(1), 52–67.
Yang, Y.-T. C., & Wu, W.-C. I. (2012). Digital storytelling for enhancing student academic achievement, critical thinking, and learning motivation: A year-long
experimental study. Computers & Education, 59(2), 339–352.
Zheng, B., Warschauer, M., Lin, C., & Chang, C. (2016). Learning in one-to- one laptop environments. Review of Educational Research, 86(4), 1052–1084. http://dx.doi.
org/10.3102/0034654316628645.

30