Summary of Mathematics inside the black box by Jeremy Hodgen and Dylan Wiliam

Mathematics inside the black box

Assessment for learning is any assessment for which the first priority in its design and
practice is to serve the purpose of promoting pupils learning (not for the purpose of
accountability, ranking or certifying competence)
An assessment activity can help learning if it provides information to be used as feedback
by teachers and pupils is assessing themselves and each other to modify the teaching
and learning activities in which they are engaged.
Assessment becomes formative when the evidence is actually used to adapt the
teaching work to meet learning needs. The purpose of formative assessment is to help
teachers to filter the rich data that arise in class discussion and activity so that
professional judgements can be made about the next steps in learning. It includes
feedback, peer and self-assessment.
Mathematics is a connected body of knowledge. To be successful pupils need to build up
a relational understanding of how ideas interrelate (Skemp, 1976).
Yet pupils at all levels of achievement have difficulty transferring and connecting the
mathematics of the classroom into other contexts. Many view mathematics simply as a
puzzling set of random procedures useful only in mathematics classrooms and
examinations.
Three types of feedback are essential to formative assessment: pupil to teacher, teacher
to pupil and between pupils.
Five principles of learning:
1 = Start from where the learner is. Pupils have to reconstruct their ideas as to merely add
an overlay of new ideas to existing knowledge tends to lead to an understanding of
mathematics as disconnected and inconsistent.
2 = Pupils must be active in the process learning has to be done by them, it cannot be
done for them. Teachers must encourage and listen carefully to a range of responses,
taking them all seriously whether they are right or wrong, to the point or not. They must
help pupils to talk through inconsistencies and respond to challenges. In such
discussions, teachers can fashion their interventions to meet the learning needs that have
been made evident.
3 = Pupils need to talk about their ideas. When pupils are talking about mathematical
ideas, whether in a whole-class dialogue or in peer groups, they are using and
constructing the language of mathematics. Talking the talk is an important part of
learning.
4 = Pupils must understand the learning intention. This is vital in order to learn and
requires understanding of what would count as good quality work (success criteria). Pupils
must also have an idea of where they stand in relation to the target/success criteria. Only
with both these pieces of knowledge, can children achieve the power to oversee and steer
their own learning in the right direction, so that they can take responsibility for it. Simply
providing lists of criteria of what makes for a good piece of mathematics is rarely sufficient
to help pupils progress. Pupils need to engage in mathematical argument and reasoning
in order that they and their peers can learn the ways in which the quality of mathematical
Bexley Primary Mathematics Team

Summary of Mathematics inside the black box by Jeremy Hodgen and Dylan Wiliam

work is judged. Peer and self-assessment are essential to this process as they promote
both active involvement and practice in making judgements about the quality of work
their own and of fellow pupils.
5 = Feedback should tell pupils how to improve. When feedback focuses on the student
as a good or bad achiever, emphasising overall judgement by marks or grades, it focuses
on the self (ego-involvement). Research has shown that this actually lowered performance
(Kluger and DeNisi, 1996), therefore performance would have been higher had no
feedback been given! Such feedback discourages the low attainers and makes the high
attainers avoid tasks of they cant see their way to success, for failure would be seen as
bad news about themselves rather than as an opportunity to learn.
When feedback is focused not on the person but on the strengths and weaknesses of the
piece of work (task-involvement) and what needs to be done to improve, performance is
enhanced especially if feedback includes ideas about HOW to improve. Such feedback
encourages all pupils, whatever their past achievements, that they can do better by trying
and that they can learn from mistakes and failures (Dweck, 1999).
These principles make substantial demands on teachers subject knowledge, not
only to make sense of what pupils say but also to be able to determine what would
be the most appropriate next steps for the pupil. This is not abstract knowledge
gained from e.g. a degree but rather a profound understanding of fundamental
mathematics (Ma, 1999).
Classroom dialogue
Through talking, exploring and unpacking mathematics, pupils can begin to see for
themselves what they know and how well they know it. By listening and interacting,
teachers can provide feedback on ways to improve their learning.
Implementing such an approach is a complex activity that involves the following aspects:
1 = Challenging activities that promote thinking and discussion
- the obvious answer is not always correct (encourages pupils to defend their
ideas)
Which of these statements is true?
A = 0.33 is bigger than 1/3
B = 0.33 is smaller than 1/3
C = 0.33 is equal to 1/3
D = You need more information to be sure
All of these answers are to some degree correct and justifiable. The problem
provides an opportunity to differentiate between different levels of understanding of place
value and equivalence.
Challenges must be directed at a wide range of ability and achievement levels
providing an opportunity for pupils to learn from each other.
Challenges must be set so that increasing ability and achievement does not
necessarily increase a pupils likelihood of getting a correct answer (must be accessible
to all and have a range of answers)
- using what we know about pupils mathematical understanding (asking a
question can focus pupils attention on what they need to do to improve their
understanding e.g. what is confusing about the problem?)

Bexley Primary Mathematics Team

Summary of Mathematics inside the black box by Jeremy Hodgen and Dylan Wiliam

The mathematics curriculum is content-heavy. There is a lot to learn and limited time
in which to learn it. As a result, even successful pupils can have difficulties with relatively
simple ideas in new or unusual contexts.
- problems with more (or less) than one correct answer (pupils generally expect
mathematical problems to have one and only one correct answer but there may be more
than one or no answer!)
In what ways could this sequence be continued: 1, 2, 4....... (e.g. 7, 11... or 8, 16...)
Draw a triangle with sides 4cm, 6cm and 11cm
- generating mathematical structure (Identifying similarities and differences can
enable pupils to begin to generate mathematical structures for themselves e.g. giving
children different sums on cards and asking them to sort them into groups why have
they chosen them to be similar/different?)
- closed questions can sometimes be valuable
- generating different solutions
Often we give students the message that school mathematics is about getting
answers to problems, whereas our actual aim is to enable them to learn mathematics.
Asking pupils to generate different ways of solving a problem is one way of focusing their
attention on the process of mathematics. Knowing one solution can help pupils generate
and understand another, and this can enable them to understand the connections
between different mathematical areas.
- mistakes are often better for learning then correct answers
Children can be given a set of calculations (with answers) and asked to identify
which are correct or incorrect. In addition to helping some children identify mistakes they
make, it focuses the children on the process rather than the outcome. Mistakes must be
valued in the classroom.
- using textbooks
Pupils could be asked to identify four questions from a textbook, two which they
consider easy and two difficult. They could construct model answers to the easy
questions individually and with a partner for the hard questions. Discussion and
questioning must go hand in hand with this and opportunities for pupils to learn from each
other promoted.
- using summative tests formatively
Analyse gaps, give the children the mark scheme and ask them to construct model
full mark answers, pupils identify easy and hard questions and work together to solve,
work in pairs to complete the test, in pairs make a harder test and mark scheme........
- good problems are not universal
Not all activities will work with all children at all times. They are dependent on the
existing knowledge of the children and must engage them.
Bexley Primary Mathematics Team

Summary of Mathematics inside the black box by Jeremy Hodgen and Dylan Wiliam

- generating challenging activities

The task of generating appropriate and challenging activities can only be done by
the teachers themselves. It is helped by opportunities to work with other teachers.
When planning a challenging activity think: In what ways does the activity promote
mathematical learning and talk? What opportunities are there for the teacher and pupils to
gain insights into the pupils learning? How does the activity enable the teacher and the
pupils to understand what the pupils need to do next?
2 = Encouraging pupil talk through questioning and listening
In mathematics classrooms, teachers tend to work too hard while the pupils are not
working hard enough, resulting in the old joke that schools are places where children go
to watch teachers work!
Where pupils are actively involved in discussion, not only do they learn more but also
their general ability actually increases (Mercer et al, 2004).
This is only possible if classroom discussion develops beyond a series of rapid-fire,
closed questions which often only include a few pupils and allow little time for reflection,
towards an atmosphere where the activities are so structured that they offer real
opportunities for thinking.
By listening more to pupils, teachers learn more about what pupils know and how well
they know it.
More pupils have more opportunity to express their ideas through longer contributions.
Pupils have more opportunities to learn from their peers.
By being listened to, pupils realise the teacher is actually interested in what they say and
are therefore encouraged to say more.
Talking less gives the teacher more time to think about the interventions they make.
Responsive questioning (responding in the moment to pupils ideas) is more
complex than simply generating questions. Teachers need to anticipate how pupils may
respond and generate appropriated interventions and questions.
Evaluative listening = Teachers listen for the correct answer and when pupils give
partially correct answers they respond with Almost or Nearly. This encourages the belief
that teachers are more interested in getting pupils to give the correct answer, rather than
finding out what they think about a particular idea (Davis 1997).
Interpretive listening = Teachers listen to what pupils say in order to work out why
the pupils respond in that way. They are interested in what and how they think more than
the correct answer.
3 = Strategies to support all learners to engage in discussion
The wait time in many mathematics classrooms is very low, less than one second.
Increasing wait time to around three seconds can have very dramatic effects on the
involvement of pupils in classroom discussion (Askew and William 1995). This is only
effective for higher order questions (not simple number facts for example). Pupils need
structure as well as time.
4 = Peer discussion between pupils
This is essential in the formative classroom as it helps all children to engage
discussion leading to better understanding of problem and clarity of ideas. Flipping a coin
or jigsaw idea to provide feedback will ensure all children are ready to contribute.
Bexley Primary Mathematics Team

Summary of Mathematics inside the black box by Jeremy Hodgen and Dylan Wiliam

5 = Rich and open whole-class discussions

Teachers need to value all mathematical contributions mistakes and partially correct
answers included and encourage pupils to identify/challenge any ideas they disagree
with or do not understand as pupils can be reluctant to give answers that they think may
be incorrect. Children must be challenged about correct as well as incorrect answers and
teachers must practise ways of avoiding letting pupils know whether an answer is correct
through body language, facial expression or type of comment. Teachers can make
deliberate mistakes for children to pick up on this encourages the attitude that we can all
learn from mistakes and the mathematical content is the important part not is the person
saying them is good at maths.
Feedback and marking
- marks dont give learners the advice on how the work or the understanding can be
improved.
- marks emphasise competition not personal improvement.
- marks demotivate low attainers and provide no challenge to high attainers.
- marks and comments are a waste of time as children only focus on the mark (Butler
1988).
- Useful comments every two or three weeks are more useful than a mark on every piece
of work.
- Time needs to be allocated for children to read, respond to and act upon feedback.
Class time gives pupils time to discuss with peers and ask the teacher or peers for
clarification.
Self and peer assessment
- It is only the learner who can do the learning.
- This requires pupils to have a clear picture of the targets in the learning journey ahead of
them and a means of moving forward to close the learning gap.
- Peer assessment helps pupils develop self-assessment skills.
- Pupils need training in these techniques.
- The key to success in both peer and self assessment is talking about mathematics.
- Use of red, yellow and green stacking cups all through the lesson not just at the end
(takes time to embed and gain confidence of children but very successful if handled well).
Final thoughts!
- Do slowly.
- Focus on one or two aspects at a time.
- Support of peers (meetings of range of peers and individual support of one close peer).

Bexley Primary Mathematics Team