What is Qualitative

Analysis for you?

 • Relating to how
Qualitative good something
is.

• A careful study
Analysis about
something.
 What is Qualitative Analysis?

Qualitative analysis is a
securities analysis that
uses subjective judgment
based on unquantifiable
information
 Purpose
Categories of Approaches
Techniques for Monitoring Student
Progress

Types of Qualitative Analysis

Feedback Methods
Referencing
 PURPOSE

NOT all topics in language nor

literature can be measured
statistically. Viewpoints, actions
and characteristics can’t
always be represented
numerically and so need a
qualitative approach.
 CATEGORIES OF
APPROACHES

1. Reflection
This approach is aimed at
gaining an insights into the
thinking processes
and opinions of the test
taker.
 2. Verbal reports or
verbal protocols are a
way of collecting
qualitative data. They
offer an insight into the
thought processes of
informants.
A number of variables
can be distinguished:
Talk/Think aloud
Concurrent
Retrospective
Mediated

Non-mediated
Some pointers when using verbal reports
in test analysis

Before:
➢ Consider language. Will the informant be able to
voice his/her thoughts in an L2?
➢ Is the concurrent verbal report more suitable
than the retrospective one or vice versa?
 During:
➢Tell the informants what a
verbal report is.
➢Give the informants the
opportunity to practice before
the real report starts.
➢Give feedback after the try
outs.
 After:

➢Try to process the data as

quickly as possible.
➢In any case, make notes
for future reference.
3. Diary Studies
informants keep a diary which
allows researchers to get an
insight into their thoughts.
Diaries are not often used in test
validation research, but they
have proven their worth in
research into learning
processes.
A number of
varieties:

Unstructured

Guided

Structured
 TECHNIQUES FOR
MONITORING STUDENT
PROGRESS
➢Make video and audio
recordings of a variety of
formal and informal oral
language experiences, and
then assess these according
to pre-determined criteria
which are based upon student
needs and curriculum
objectives.
➢Use checklists as
concise methods of
collecting information,
and rating scales of
rubrics to assess
student achievement.
➢Record anecdotal
comments to provide
useful data based
upon observation of
students’ oral
activities.
➢Interview students
to determine what
they believe they do
well or areas in
which they need to
improve.
 What is Self-Assessment?
Self-assessment promotes
students’ abilities to assume
more responsibility to identify
where they believe they have
been successful and where
they believe they require
assistance.
 What is Peer-Assessment?
Peer assessment allows students to
collaborate and learn from others.
Through discussions with peers,
high school students can verbalize
their concerns and ideas in a way
that helps them clarify their
thoughts and decide in which
direction to proceed.
 TYPES OF QUALITATIVE
ANALYSIS

1.Conversation Analysis (CA)

The assumption is that every
interaction is contextual and
has a stable and predictable
nature. It focuses on talk (
speaking notes)
 TYPES OF QUALITATIVE
ANALYSIS

1.Conversation Analysis (CA)

-From the transcripts, the

structure and turn taking will
be analyzed.
 TYPES OF QUALITATIVE
ANALYSIS

1.Conversation Analysis (CA)

-Pitfalls
* Data Quality
* Loss of Data
* Time
 TYPES OF QUALITATIVE
ANALYSIS
2. Discourse Analysis (DA)
DA is the analysis of “text
and talk as social
practices” and is mainly
concerned with power
relations, gender
inequalities.
 TYPES OF QUALITATIVE
ANALYSIS
2. Discourse Analysis (DA)
-the transcript is analyzed for adjacency pairs, turn
taking and repair.
-Special attention to;
*the effect of examiner behavior on test taker
performance
*The effect of test taker characteristics on
performance
*The effect of task type to performance
*Comparing test takers language ability outside of
test-to-test performance.
 TYPES OF QUALITATIVE
ANALYSIS

3. Text Language Analysis

An analysis of test input or test taker
responses for lexical richness,
rhetoric, genre, discourse markers,
grammatical complexity, etc.
-Pay attention to:
• time- test language analysis is time
consuming
• Data- processing of data
 TYPES OF
QUALITATIVE
ANALYSIS
4. Task Characteristics
This type of validation
research helps to examine
the test tasks and to
determine to which extent
they correspond to the test
goal.
 FEEDBACK METHODS

1. Questionnaires

2. Interviews
1. Questionnaires
-gather data such as
opinions and views that can
also be gathered through
interviews. The main
advantage of questionnaires
is the possibility to use a
very large informant
population.
There are roughly
two kinds of
questionnaires:

1. Closed

2. Open
Before administering
the actual
questionnaire,
it is useful to run it
through the
following process:
 Consider all possible issues that your
questionnaire should cover.

Write a draft.

Eliminate questions thematically to

spot overlaps.

Format the questionnaire and administer it

to a small group of target respondents for
feedback.

Rewrite the questionnaire.

Always avoid
the following:
➢Double-barreled questions.

➢ Unclear instructions

➢Questions that do not apply to the

respondent.

➢Questions that rely on memory.

➢Hypothetical questions and

biased options.
 2. Interviews
-they are flexible way of
gathering data. There are
various kinds of interview,
depending on the structure
and the number of
informants interviewed at
the same time.
 There are
various kinds of
interview:
Unstructured

Semi-structured
Structured

One on one

Group
 REFERENCING

Referencing is assigning students a

position in a rank order based on
their score on a tests.
1. Norm-referencing is
the placement of learners
in rank order, their
assessment and ranking in
relation to their peers.
 2. Criterion-referencing is a
reaction against norm-
referencing in which the
learner is assessed purely in
terms of his/her ability in the
subject, irrespective of the
ability of his/her peers.
 3. The mastery criterion-
referencing approach is one in
which a single minimum
competence standard or cut-off
point is set to divide learners
into masters and non-masters,
with no degrees of quality in
the achievement of the
objective being recognized.
 4. The continuum criterion-
referencing approach is an
approach in which an
individual ability is
referenced to a defined
continuum of all relevant
degrees of ability in the area
in question.
 Assignment:
Reflect on the statement and tell
if you agree or disagree. Then
explain why?
1. The instruments for peer and
self-assessment should be
collaboratively constructed by
teachers and students and that
teachers need to discuss the
learning objectives with
students.
Thank You