Lecture 1

What is research
Research is the process of going up alleys to see if
they are blind
Marston Bates, American writer
Research is what I am doing when I don’t know what I
am doing
Von Braun, American engineer
If you steal from one author, it’s plagiarism; if you
steal from many, it’s research.
Wilson Mizner, American dramatist

Lecture 1 Compulsory Readings

Nunan. D. 1992. Research Methods in Language Learning.

Cambridge: CUP (P. 1 - 23)
Brown, J.D. & Rodgers, T.S., 2002, Doing Second Language
Research, Oxford: OUP (P 3-18)
Wisker, G. 2001, The Postgraduate Handbook, New York:
Palgrave (P. 113 - 126)
McDonough, J. & S. McDonough, 1997, Research Methods for
English Language Teachers, London: Arnold (P. 37 - 55)

What is research?
Research is a systematic process of inquiry
consisting of three elements or components:
A question, a problem or a hypothesis
Data
Analysis and interpretation of data
(Nunan 1992)

Research process
A process of formulating questions, problems, or
hypothesis, collecting data or evidence relevant to
these questions, problems or hypothesis and
analyzing or interpreting data (Nunan 1992).

The basic process of research

problem / experience / observation

hypothesis
investigation and experimentation to test the hypothesis
data gathering
data analysis and interpretation
confirmation or disapproval of the hypothesis
(Wisker 2001)

1.Truth and research

Two views:
External truths independent of the
researcher/observer

Truths are questionable, subjective

Two main approaches:

Deductivism and Inductivism

Deductivism
Begin with a hypothesis/theory
Search for evidence to
- support the hypothesis/theory
- refute the hypothesis/theory
Inductivism
Seek to derive general principles/theories/ ‘truths’ from an
investigation and documentation of single instances

Research traditions
Psychometric
Interaction analysis
Discourse analysis
Ethnography

Vanlier: two parameters

Interventionalist

Seclectivity

Brown: Primary vs secondary

Primary: basing on primary sources of information
Secondary: basing on the secondary sources of
information

Questions for Tutorial 1

What is research? What is a research carried out for? Find a research

report in an applied linguistics journal (such as TESOL Quarterly,
Language Learning) and point out the objectives and how these
objectives are achieved.

Present briefly some traditions in applied linguistics research with

illustrations from real research reports from applied linguistics
journals.

What are Deductivism and Inductivism in applied linguistics research?

Give some examples from real research studies in applied linguistics.

Explain briefly some of the following terms with illustrations from

applied linguistics research:
Deductivism Inductivism
Reliability Systematic
Validity Logical
Construct Tangible
Variable Replicable
Longitudinal Reductive
Qualitative Quantitative
Quantifiable Psychometric
Primary data Secondary data
Types of research papers in Applied Linguistics
Term paper
MA thesis
Prelim (qualifying paper)
Dissertation proposal
Grant proposal
Dissertation
Review of literature
Conference abstract
Conference paper
Squib
Book review
Journal article
Monograph, book
Textbooks
Qualitative research (Non-numeral research)
All knowledge is relative
Subjective element to knowledge
Procedures:
- observation
- interview
- field work
- open response questionnaires
 - transcripts
Quantitative research (Numeral research)
Obtrusive
Controlled
Objective
Generalisable
Outcome-oriented
Procedures:
- tests
- interview & self-reports
- closed response questionnaires
- numbers, figures

Reliability
The consistency of the results obtained from a piece
of research
Internal reliability: consistency of data collection,
analysis and interpretation
External reliability: similar results can be reproduced
by independent researcher
Validity
The extent to which piece of research actually
investigates what the researcher purports to
investigate
Internal validity: the interpretability of a research
External validity: the extent to which the results can
be generalised from samples to population
Construct
A psychological quality that cannot be observed but can only
be assumed to exit and can be explained through
observable behaviours (intelligence, proficiency,
motivation, aptitude etc.)

Construct validity: the construct under investigation must be

defined accessible to the outside observer, the construct
characteristics must be identifiable to reader
Variable
Something that may vary, differ over time

Some variables differ over time and among

individuals (language proficiency, motivation, self-
esteem etc.)
Variable vs construct
Variables represent the underlying constructs
Variable: what can be observed or quantified of the
human characteristics or abilities involved
Construct: the actual characteristic or ability that it
represents in human being
Systematic
Clear structure with definite procedural rules

Easy to read, interpret and evaluate

Logical
Step-by-step progression

Straightforward, logical pattern with underlying rule

and procedures
Tangible
Based on the collection and manipulation of data
from the real world

Data are well-defined quantity, rank or category

Replicable
The possibility of doing the research again under the
same conditions by an independent researcher

Replicating the research may yield similar results or

information to evaluate it
Reductive

The possibility of establishing patterns and

relationships among individual variables, facts, and
observable phenomena in a research

Questions for Tutorial 1

What is research? What is a research carried out for?

Find a research report in an applied linguistics
journal (such as TESOL Quarterly, Language
Learning) and point out the objectives and how
these objectives are achieved.
Present briefly different views of the nature of social
sciences and social reality. Give some examples
from real research studies in applied linguistics.

Present briefly a model of research process in applied

linguistics with illustrations from real research
reports from applied linguistics journals.

4. Explain briefly some of the following terms

with illustrations from applied linguistics research:
Deductivism Inductivism
Reliability Systematic
Validity Logical
Construct Tangible
Variable Replicable
Longitudinal Reductive
Qualitative Quantitative
Quantifiable Psychometric
Primary data Secondary
data
Social realty

Lecture 2 Compulsory Readings

Nunan. D. 1992. Research Methods in Language Learning.

Cambridge: CUP
(P. 24 - 50)
Selinger, H.W., and E. Shohamy. 1989. Second Language
Research Methods. Oxford: OUP (P. 135 - 152)
Brown, J.D. 1988. Understanding Research in Second
Language Learning : A Teacher’s Guide to Statistics and
Research Design. New York: CUP
Hatch, E. and A. Lazaraton, 1991. The Research Mannual:
Design and Statistics for Applied Linguistics. New York:
Newburry House
What is an experimental research?

Experimental research is carried out to explore the

strength of relationship between variables

2. Type of scale

Nominal scale
Ordinal scale
Interval scale
Ratio scale

Inferential statistics: The statistics which may be

inferred from sample to population
T-test: the test to compare two means (2 groups)
F-test: the test to compare more than two means
(more than 2 groups)
ANOVA: Analysis Of Variance, a procedure to
compare more than two means

Chi-square: calculate the different between the

expected frequencies and compare them with the
actual frequencies
Correlation: a statistical test to estimate the degree
(strength) of association between two variables
Control group: the group under normal situation
experimental group: the group under experimental
condition

True-experiment:
- Randomly assigning the subjects into control and
experimental groups
- Administering both pre- and post-treatment tests
(Highly controlled, highly selective)
Quasi-experiment:
- No random arrangement of subjects
Pre-experiment
- Only pre-treatment test
 The Logic of Statistical Inference
Mean X

The average of a set of scores

The central tendency of the scores

The Logic of Statistical Inference

Mode: The score that occurs most frequently in a set
of scores.
Median: the middle point in a distribution, the
midway between the two middle scores

The central tendency of the scores

Standard Deviation SD

Important measure of dispersion

Estimating:
1) deducting the mean from each individual score
2) Squaring the resulting figures to get rid of minus
signs
3) adding these together and dividing by the total
scores minus one ﾁ¨ Variance
4) Obtaining the square root of the figure ﾁ¨ SD

Standard Error SE
The standard deviation of sample means: placing a single
sample mean in relation to the population mean
Estimating: Dividing SD of a single sample by the square
root of the number of observations in the sample (N)

SE = SD : ﾁ ã N
 Questions for Tutorial 2
Question: Find a report of an experimental study in applied linguistics and give
your comments on the following checklist:
I. Introduction: Is the framework for the study clear?
A. Literature review: can you tell where the study fits in?
1. Is the background or rationale provided?
2. Is the relationship to previous research clear?
B. Statement of purpose: can you tell where the study is heading? Are any of the
following included?
1. Purpose
2. Research question
3. Research hypothesis
II. Method: Is the study replicable?
A. Subjects
1. Is the description of participants adequate?
2. Is the method of selection clear?
B. Materials
1. Is there a description of tests, questionnaires, rating scales, and so forth?
2. Do the variables represent reasonable operational definitions of the underlying
constructs or characteristics involved?
3. Is there a description of any equipment (when applicable)?
C. Procedures
1. Is there a description of the preparation of material, administration, scoring and so on?
2. Is there a description of the conditions during study?
D. Analyses
1. Is there a description of the arrangement and grouping of the data?
2. Are the statistical tests listed in order of use?
III. Results
A. Are all the statistical tests previously listed represented as results ?
B. Is there a prose explanation (optional)?
IV. Discussion/Conclusion
A. Is the original research question, or questions, answered ?
B. Is there an explanation of why the results were as they were?
1. If the conclusion is based on previous research, is it well supported and reasoned?
2. If the conclusion is speculative, is it qualified as such and well reasoned?
C. Are suggestions for further research provided?
V. References, Notes, or Footnotes
A. Are all the references cited in the text included ?
B. Are nay pertinent references missing?
VI. Appendixes
A. Are they necessary?
B. Are they complete?

Lecture 3
ETHNOGRAPHIC RESEARCH

Compulsory Readings

Nunan. D. 1992. Research Methods in Language Learning. Cambridge: CUP (P. 52 - 73)
Brown, J.D& Rodgers T.S. Doing Second Language Research. OUP. 2002 (P.53-112)
Coffey, A. & Atkinson, P. 1996, Making Sense of Qualitative Data, Complementary
Research Strategies, CA: SAGE Publications.
May, T. (ed) 2002, Qualitative Research in Action, London: Sage Publications.
McMillan, j. & S.Schumacher, Research in Education, A Conceptual Introduction,
Harper Collins College Publishers.
Wajnryb, R. 1992, Classroom Observation Tasks, Cambridge: CUP.
Newman, P. & Ratliff, M. 2001, Linguistics Fieldwork, Cambridge: CUP

What is ethnography?

Ethnography, which seeks to understand and represent the points of view of the members
of a particular culture, is a primary approach to data collection and analysis in
anthropology and sociology.

Ethnographic research
Ethnographic research is one form of qualitative research which concerns with studying
human behavior within the context in which that behavior would occur naturally and in
which the role of the researcher would not affect the normal behavior of the subjects.

Ethnographic research
Ethnography research requires:
- much training, skill and dedication
- a great store on the collection and interpretation of data
- question and hypothesis emerge during the course of investigation,
rather than beforehand

What is ethnography research

• Ethnographic research uses naturalistic observation to record systematically the

behaviour of the subject under study in its own settings. Its root is in ethnography:
anthropology and sociology and the focus is on cultural aspects of behaviour

Characteristics of ethnographic research

• Contextual: carried out in the context in which the subjects normally live and
work
• Unobtrusive: the researcher does not try to control or handle the phenomenon
• Longitudinal: the research takes a relative long time
• Collaborative: the research involves the interaction between participants

• Interpretative: interpretative analysis of data

• Organic: ethnographers may obtain data which do not support their original
questions or hypothesis but are suggestive of others.

Procedures for conducting ethnographic research:

• Define the phenomenon to be described.

• Use qualitative methods to gather data.
• Look for patterns in the data.
• Validate initial conclusions by returning to the data or collecting more data.
• Recycle through the process or the data.

Principles of ethnography
Two sets of hypotheses about human behaviours:

• naturalistic-ecological

• qualitative – phynomenological

Naturalistic-ecological

• Context has significant influence on behaviour

• Investigating natural contexts: field research, real life situation, real world

Qualitative-phenomenological

• Belief: there is no objective reality independent of subjective perceptions

• Human behaviour must be investigated with subjective perception and belief
systems of researcher
• No objective observer
• Cultural meanings are revealed by the behaviours of the subject under study

Principles of ethnography
• The use of participant and non-participant observation
• A focus on natural settings
• Use of subjective views and belief systems of the participant in the research
process
• Not to manipulate the study variables
• Holism and thick explanation

Low Inference Descriptors Vs High Inference Descriptors

• Low Inference Descriptors (LID): Behaviours easily observed and agreed

• High Inference Descriptors (HID):

Behaviours requires much inference

External reliability
• To what extent can the findings from a study carried out in a particular site be
generalized to other sites?
• Threat:
Based on detailed description of a particular context/ situation → difficult for outsiders
to conduct another research
To overcome this:
Explicit about 5 key aspects
• Status of researcher
• Choice of informants
• Social situation & conditions
• Constructs & premises
• Method of data collection & analysis
Questions to be asked:
• Is the status of the researcher made explicit?
• Does the researcher provide a detailed description of subjects?
• Does the research provide a detailed description of the context and conditions
under which the research was carried out?
• Are constructs and premises explicitly defined?
• Are data collection and analysis methods presented in detail?

Internal reliability
• Can an independent researcher come to the same conclusion as the original
investigators when he/ she analyzes the same primary data?
• Threat
Ethnographer rarely uses standardized instruments → difficult for independent
researcher to reanalyze data themselves

To overcome this:
• Use low inference descriptors
• Multiple researchers/ participant researchers
• Peer examination
• Use mechanically recorded data

Questions to be asked:
• Does the research use low inference descriptors?
• Does it employ more than one researcher/ collaborator?
• Does the researcher invite peer examination or cross-site corroboration?
• Are data mechanically recorded?

Internal validity

To what extent the research is measuring what it purports to measure?

To overcome this:
Employ data collection and analysis techniques:
+ ethnographer as a participant
+ informants interviewing
+ participant observation
+ ethnographic analysis

Questions to be asked:
• Is it likely that maturational changes occurring during the course of the research
will affect outcomes?
• Is there bias in the selection of informants?
• Is the growth or attrition of informants over time likely to affect outcomes?
• Have alternative explanations for the phenomena been rigorously examined and
excluded?

External validity

To what extent can research outcomes be extended to other groups?

To overcome this:

Describe phenomena explicitly so that they can be compared with other studies/ carry
out multiple-site investigation
Questions to be asked:
• Are some phenomena unique to a particular group or site and therefore non-
comparable?
• Are outcomes due in part to the presence of the research?
• Are cross-group comparisons invalidated by unique historical experiences of
particular groups?
• To what extent are abstract terms and constructs shared across different groups
and research sites?

Criteria for assessing ethnographic reports

• What are the goals of ethnography? What is the research problem?
• In what contexts was the research conducted?
• What is the group or case under study?
• What conceptual and theoretical frame works inform the study?
• What field techniques are used? For how much time? In what contexts? What was
the role of the ethnographer?

6. What analysis strategies were developed and used? What levels and types of context
attended to in interpretation?
7. What recurrent patterns are described?
8. What cultural interpretation is provided?
9. What is the contribution of the study to our knowledge of socio-cultural factors
involved in schooling in second language and culture?
10. What are the stated implications for teaching?

Questions for Tutorial 3

• What is ethnographic research? ( State the difference between an ethnographic
research and a psychometric research and give example from applied linguistic studies.

2. Find a report of an ethnographic research in applied linguistics and give your

comments on the following points:
- The research question
- The contexts the research was conducted
- What is group or case under study?
- What conceptual and theoretical frame works inform the study?
- What field techniques were used? For how much time? In what contexts? What
were the roles of the ethnographer?
• Find a report of a ethnographic research in applied linguistics and give your
comments on:
- What field techniques were used? For how much time? In what contexts? What were
the roles of the ethnographer?
- What analysis strategies were developed and used? what levels and types of
context were attended to in interpretation?
- What recurrent patterns are described?
- What cultural interpretation is provided?
- What are the stated implications for teaching?