Impact case study (REF3b)
Institution: Newcastle University
Unit of Assessment: UoA26
Title of case study: SETT: Self Evaluation of Teacher Talk
Steve Walsh, steve.walsh@ncl.ac.uk
1. Summary of the impact (indicative maximum 100 words)
SETT (Self Evaluation of Teacher Talk) is a reflective practice tool and a set of procedures which
enable teachers to analyse their use of language and interaction while teaching, with a view to
improving communication in the classroom, resulting in more dialogic, engaged class communities.
Its main impact is in the training and professional development of language teachers globally; it has
also been widely used in doctoral and MA research internationally.
The initial research has resulted in the publication of four research monographs (2006, 2011, 2013,
2017), more than 100 articles and has attracted research funding in the region of £1.4m.
2. Underpinning research (indicative maximum 500 words)
The research which underpins this impact case study dates back to 2006 with the publication of the
research monograph Investigating Classroom Discourse (Walsh, 2006, see below). The SETT
framework is a reflective practice tool designed to help teachers understand and improve their
communication with students. It comprises four classroom micro-contexts (called modes) and
fourteen interactional features (called interactures). Classroom discourse is presented as a series of
complex and inter-related micro-contexts or modes, where meanings are co-constructed by
teachers and learners and where learning occurs through the ensuing talk of teachers and learners
(Walsh 2013). The key to developing good practice is for teachers to acquire detailed profiles of the
interactions which take place in their classes as a means of understanding how learning
opportunities are created and how ‘space for learning’ (Walsh and Li, 2011) can be developed to
maximise learning opportunity. A key feature of SETT is its aim to create the kind of dialogic,
engaged learning environments which are being advocated in mainstream education in the UK and
beyond (see, for example, Mercer, 2009; Alexander, 2008).
The original research took place in a UK HEI and has since been extended to a range of contexts
around the world, where replication studies have been conducted. These include initial teacher
education programmes (PGCE) for English and Drama teachers; INSET courses for experienced
teachers; studies evaluating the value of classroom observation in the Middle East (Howard 2010);
on CELTA and DELTA programmes around the world (Certificate/Diploma in English Language
Teaching to Adults – offered by Cambridge Examinations); a primary science classroom; various
secondary EFL contexts; a number of university classroom contexts; in INSET programmes in Japan;
and in Irish medium secondary classrooms. In short, the framework has been used extensively to
promote awareness and understanding of the role of interaction in class-based learning and to help
teachers improve their practices.
The research to date has led to the construct of ‘Classroom Interactional Competence (CIC)’, which is
Page 1
Commented [AS1]: This section should focus on non-
academic impact – benefits, changes or effects that were
underpinned by research.
Commented [AS2]: Say something about the reach and
significance of the research impact – how many teachers were
trained, how many countries etc. Also say something about
the overall benefits – what has the teacher training led to?
Improved communications, more engaged classes?
How has it been used in doctoral and MA research? What is
the non-academic impact arising from this?
Commented [AS3]: This isn’t relevant here
Commented [AS4]: Outline who conducted the research,
include name, position and time at NCL.
Commented [AS5]: Please number the publications outlined
in section 3, and cross-reference to the text using those
numbers. See the top tips handout for guidance on the
suggested format.
Commented [AS6]: This section should solely focus on your
research, not the research in this area more widely. See the
top tips handout for guidance on what to include.
Commented [AS7]: Make it clear that it was Newcastle.
Commented [AS8]: Are you saying that these studies could
not have been conducted in the way they have if your
research had not happened?
Are you involved in any of these research projects as a
partner or are they independent from your research?
If yes to both of the above, then this paragraph should be in
the section 4 – since it’s not your research, but new activity
that is underpinned by your research – though to do this you
would also need to outline what the non-academic impact of
their studies has been (i.e. training, changes in teaching
practice, benefits to students etc) within the REF time period,
i.e. 1 August 2013 to 31 July 2020.
What evidence do you have, or could you get, that
corroborates the impact?
Impact case study (REF3b)
used to consider how teachers might develop a level of interactional competence which creates
opportunities for learning and which highlights the importance of CIC in all classrooms. CIC is
defined as ‘teachers’ and learners’ ability to use interaction as a tool for mediating and assisting
learning’ (Walsh, 2011, p158). CIC acknowledges the centrality of interaction to teaching and
learning and highlights the need for teachers to attain CIC in their own contexts. The research on
CIC, using SETT, has led to answers to questions such as:

 How are meanings co-constructed in the unfolding classroom interaction?

 What do participants do to ensure that understandings are reached?
 How do students and teachers deal with repair and breakdown?
 In what ways are participants able to create, maintain and sustain ‘space for learning’?
 More importantly, how does CIC influence learning?

The other major contribution of the SETT framework is as a reflective practice (RP) tool to help
teachers access and understand their teaching more fully. Recent research (see for example Mann Commented [AS9]: This is something that you should also
be presenting in section 4, where you have evidence of how
and Walsh 2017) highlights the ubiquity of RP while acknowledging that there is a need for tools the tool has led to better understandings by teachers and what
benefits or changes this has led to.
and procedures to facilitate professional development. The research on SETT and CIC goes some
way in addressing this shortcoming by providing practitioners with a tool to reflect with and a focus
to reflect on.

3. References to the research (indicative maximum of six references)

Publications Commented [AS10]: Please number the references so you
can cross-reference to the narrative in section 2.
Mann, S. and Walsh, S. (2017). Reflective practice in English Language Teaching: research-based You should also annotate each publication with a brief outline
principles and practices. London and New York: Routledge. of how you know it’s 2* or above, and whether they have been
or will be submitted to RAE/REF as an output.

Walsh, S. (2016). ‘The role of interaction in a blended learning context’. In McCarthy, M. (ed.) The
Cambridge Guide to Blended Learning for Language Teaching. Cambridge: CUP.

Walsh, S. (2013) Classroom Discourse and Teacher Development, Edinburgh: Edinburgh University
Press.

Walsh, S. and Li, L. (2013) ‘Conversations as space for learning’. International Journal of Applied
Linguistics, 13/1.

Walsh, S. (2011) Exploring Classroom Discourse: from practice to theory. London: Routledge.

Walsh, S. (2006) Investigating Classroom Discourse, London: Routledge.

Grants Commented [AS11]: Please provide in table format. See

the top tips handout for a suggested layout.
Walsh, S., ‘Enhancing interactional awareness to promote active learning’ (QUB TQEF (Teaching
Quality and Enhancement Fund), June 2004, £5,000.

Morison, S., Walsh, S. ‘Centre for excellence in inter-professional education (CEIPE)’. May 2006,
£1.25m. DEL (Department for Education and Learning, Northern Ireland) funded.

Walsh, S. and Winter, E., ‘Critical reflective practice in initial teacher education: a comparative

Page 2
Impact case study (REF3b)

study’. January 2005, £6,650 awarded by British Academy.

Walsh, S. ‘Higher Education small group talk’, October 2010, £3,750, Newcastle University.

Walsh, S. ‘Characterising interactional competence in higher education small group talk’, April 2013,
£17,500, British Council.

Walsh, S. ‘SETTVEO (Self Evaluation of Teacher Talk through Video Enhanced Observation)’. April
2016, £16,000. British Council ELTRA project.

4. Details of the impact (indicative maximum 750 words)
The SETT framework and accompanying reflective practice procedures have been used and are still
being used internationally in the preparation and professional development of language teachers.
The main impact is that SETT enables teachers to reflect on and modify their interactional practices
Commented [AS12]: This is very clearly presented, but
lacks evidence in the main body of text. Please number the
evidence in section 5 and cross-reference to the relevant text
here. Please also incorporate details from the evidence in the
narrative. I’ve highlighted key areas below which require
evidence.

so that their classes are more engaged, more interactive and provide learners with greater
opportunities for learning, or learning value. While teachers and teacher educators are the main
beneficiaries, the value to learners and learning cannot be underestimated. There is plenty of Commented [AS13]: And what evidence do you have to
back up these impact claims – on teachers’ understandings
evidence to demonstrate that learners learn better when they are challenged, engaged and involved and practices as well as the wider benefits to learners.
in a dialogic teaching methodology which values communication and interaction. SETT enhances
teachers’ understanding of their language use and interactive decision-making with the long-term
benefits this brings to learners.

SETT has been adopted in more than 40 countries worldwide and used in the initial and in-service
teacher education of more than 10,000 teachers. In the UK alone, it is employed extensively on Commented [AS14]: What’s your evidence?
internationally recognised and externally validated certified programmes such as CELTA and Trinity
– two of the main providers of English Language Teacher Education (ELTE) globally. In addition, Commented [AS15]: Evidence
SETT features strongly on the extensive range of MA degrees in TESOL (Teaching English to
Speakers of Other Languages). Typically, these programmes comprise very high numbers of
international students with the result that SETT is utilised more widely by students returning to
their home countries. In some countries, like, for example, Singapore, SETT has been adopted by the Commented [AS16]: Do you have proof of this?
Ministry of Education, while in others, such as Germany, Japan and Turkey, it is used on pre- and in- Commented [AS17]: In what capacity? And rolled out to
where/whom and how? Evidence?
service ELTE programmes.
Commented [AS18]: Please outline the wider benefits of
this. Evidence?
A number of PhD projects have been completed using SETT and the construct CIC (see above).
Commented [AS19]: We can’t include impact on
Notably, here at Newcastle, a corpus of 1m words was constructed and made available to teams of research/academia per se – are there wider benefits we can
researchers. The corpus, NUCASE (Newcastle University Corpus of Academic Spoken English) claim here?

comprises audio- and video-recordings from all three faculties. By interrogating the corpus,
researchers and practitioners alike can gain closer understandings of small group teaching in a
range of HE contexts. The NUCASE corpus has been employed in university staff development and
CPD sessions and has resulted in 7 PhDs being completed by members of the original research team. Commented [AS20]: Just in Newcastle or in other
universities?

The most recent impact from SETT has emerged from the creation of an App (SETTVEO), which is
currently being trialled and adapted in 4 countries (Spain, Turkey, Chile and Thailand). In each Commented [AS21]: When is it likely to be rolled out more
broadly in these countries? Before July 2020?
country, small groups of teachers use the App to record and tag their teaching and then reflect on
their interactions and classroom communication more generally. Combining SETT with the original
VEO (Video Enhanced Observation) app enables up-close and detailed understandings of CIC to be
gained and provides teachers with a profile of their teaching. As a reflective practice tool, the

Page 3
Impact case study (REF3b)
SETTVEO app is extremely powerful. In the final stage of this project, we will build an online
community of practice which will enable teachers to share both their teaching and reflections as a
means of identifying common issues relating to language teaching pedagogy, and, collectively
exploring solutions to those issues.

The following ‘snapshots’ of the impact of SETT are intended to provide evidence for the preceding
commentary. They are not intended to offer a comprehensive body of evidence, merely an indicative
summary taken from a range of contexts and using the voices of the users of SETT:

SETT in multicultural classrooms (France)

My data collection procedure involves a series of Action Point Cycles (please see attached) where
teacher herself chooses aspects of classroom discourse to focus on. As participants asked me for
some suggestions regarding action items I presented her with modified SETT which focuses on
various features of teacher talk (please see attached). So far we recorded and analysed one lesson as
a baseline and the main conclusion was that this session, which would be very typical for this class,
was very much teacher-led. As a result she would like to focus on introducing more balance into
classroom interaction through various forms of peer teaching. Commented [AS22]: Be clearer about what the impact is. Is
the introduction of more balance the change? Has there been
any follow up to see if this has led to wider benefits?
Pre-service teacher education in Germany
Dear Steve,
I hope this mail finds you well. You might remember that we talked about VEO and SETT during the
VEO conference in London this summer. Since my classes begin next week I wanted to ask you
whether you could send me the tag set of SETT so that I could use it with my students. That would
be great since we will talk about SETT as part of the course programme and I want students to apply
it to a given lesson. They would tag only in retrospect but this seems to be a perfect opportunity to
work with the SETT framework and practice classroom observation in close to real situations. Commented [AS23]: And how many students have been
taught with this method? Overall figures would help, as well as
follow up to explore what the wider benefits have been.
CIC in India
This study is an attempt to investigate the role of Dynamic Assessment (DA) in enhancing student-
teachers’ classroom interactional competence. Specifically, the study looks into the effectiveness of
DA in augmenting the conceptual knowledge and performance of second language teachers in the
domain of classroom interactional ability. Commented [AS24]: What’s the impact? If it’s being used in
other research projects, that’s good, but you still need to
outline the non-academic benefits of the research/research
process.

5. Sources to corroborate the impact (please list evidence you currently have and also the
evidence you plan to collect)Impact since 1 August 2013 (please provide brief details,
including dates):

Current: Commented [AS25]: Please number the sources of

evidence and cross-reference to the relevant text in section 4.
You also need to incorporate more of the detail of the
 The SETT framework is now being used on initial teacher education programmes like the evidence into the main narrative.

CELTA (Certificate in English Language Teaching to Adults), provided by large organisations In the final draft, it’s likely that only 10 pieces of evidence can
such as the British Council, globally (2013-present). Evidence: curriculum documents; be included.
academic and official reports; evaluation by stakeholders and end-users.

 The SETT framework is being trialled in Singapore by the Ministry of Education in the
training of Maths and Science teachers. (2014-present). Evidence: personal involvement as a
consultant; curriculum documents, evaluation reports.
Commented [AS26]: This isn’t independent corroboration –
but the others are helpful. In time, it may be useful to ask for a
testimonial that outlines the impact in full and draws on some
of the data from the other documents and report.

Page 4
Impact case study (REF3b)
 The SETT framework has been adopted by the Ministry of Education and HEIs in Turkey in Commented [AS27]: Could you get corroboration directly
the initial training of Turkish English language teachers at a number of universities. (2015 – from the Ministry?

present). Evidence: feedback from previous students who completed doctorates at

Newcastle, plus research papers and conference presentations documenting this innovation. Commented [AS28]: Since this is anecdotal, is there a
leading authority in Turkey who could attest to this more
broadly?
 The SETT framework is currently used to train EAL (English as an Additional Language)
teachers who teach refugees and immigrants in Ireland (2014-present). Evidence: feedback Commented [AS29]: How many teachers? And what are
from colleagues in Ireland, research publications and official documents. the wider benefits for the teachers and learners? Any
feedback which outlines this?
Commented [AS30]: What kind of documents?
Potential:
 Setting up a website where best practice in the use of SETT can be shared and disseminated,
and where it would be possible to establish an online community of practice. This would
require some funding (see below), but could easily build on the SETTVEO project which will
come to an end this summer (British Council funded).
 Collect testimonials and references from stakeholders, with brief supporting commentaries
to indicate how the tool has been adopted in a range of contexts, and what the benefits have
been to both practitioners and their students. Some of the networks from which such
evidence might be drawn are current and could easily be accessed.
 Collect evidence from research students who have used the SETT framework to show how it
has been used by (a) practitioners, (b) researchers. Evidence might include statements from
teachers, links to websites and abstracts from published research and from conference
presentations.
 Collect more detailed reflective commentaries from stakeholders in (for example) Europe,
Turkey, Thailand, China, USA, UK and Ireland where SETT is actively used. These
commentaries would provide a cross-section of the applicability of the tool in a range of
contexts and highlight its potential for adaptation.
 Offer a series of in-depth case studies of the SETTVEO app (see above) to demonstrate how
it has been adopted in four countries (Spain, Turkey, Thailand and Chile). Each case study
would highlight the benefits of the app for teacher development and reflective practice and
the importance of sharing practice through online communities of practice.

Please list any plans you have to further extend the significance and reach of this case study:
The plan now is to collect much more detail on the impact of SETT. As outlined above, this will entail
collecting evidence and assessing the actual benefits of the research in as wide a range of contexts as
possible. In order to strengthen this impact case study, the aim in the coming months would be to collect
much more information and evidence about the impact of SETT for both practitioner and research
communities.
With the SETTVEO research project due to end in May, there will be a period of dissemination and
publication to follow. The team will be attending major international conferences (both academic and
practitioner), producing a report for the British Council (circulated through their 80 offices globally), and
writing articles for peer-reviewed, international journals. In addition, there will be a teachers’ seminar at
the end of the project where participating teachers will offer comments and evaluations of SETTVEO.
For internal purposes please note any help you might need: Commented [AS31]: There is no more funding available
from Faculty in this academic year as it’s been fully allocated
Funding and staffing needed to: from previous calls.

I’d suggest speaking to Heather Smith (impact champion) to

(a) Develop and maintain a web-site. explore whether the School has any funds available.

Page 5
Impact case study (REF3b)
(b) Collect testimonials and evidence from stakeholders.
(c) Produce case studies which demonstrate how SETT has been used.
(d) Establish an online community of practice for both researchers and practitioners who
have used or are using SETT.

Page 6