Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
EDUCATIONAL
RESEARCH
METHODOLOGY
Prof Dr John Arul Phillips
WELCOME TO HMEF5014
HMEF5014 Educational Research Methodology is one of the required courses
for the Master of Education (MEd) programme. The course assumes no previous
knowledge and experience in educational research but you are encouraged to
tap into your experiences as a teacher, instructor, lecturer or trainer and relate
them to the concepts and principles discussed. This is a three (3) credit course
conducted over a semester of 14 weeks.
COURSE SYNOPSIS
Topic 1 examines what is education, the scientific method, the criteria for good
research and the steps in the research process.
The first part of the Topic 2 defines theory and the role of theory in educational
research. The second part deals with the role of the literature review in research
and guidelines in writing a review.
Topic 4 presents different types of weak designs, true experiments and quasi-
experiments. Focus is on the reasons for using a particular design and the ethics
involved in experiments using human subjects.
Topic 5 examines the different types of surveys and the procedure in conducting
a survey. Sampling and generalisation of findings are discussed together with the
methods of data collection. The ethics of conducting surveys are also discussed.
To help you read and understand the individual topics, numerous realistic
examples are given to clarify definitions, concepts and theories. Diagrams and
text are combined in a visually appealing, easy-to-read module. The use of
illustrations, tables and charts reinforce important points and simplify the more
complex concepts.
INTRODUCTION
Lists the headings and subheadings of each topic to provide an overview of the
contents and the major concepts to be studied.
LEARNING OUTCOMES
This is a listing of what you should be able to do after successful
completion of a topic. In other words, whether you are be able to
explain, compare, evaluate, distinguish, list, describe, relate and so forth.
You should use these indicators to guide your study. When you have
finished a topic, you must go back and check whether you have achieved
the learning outcomes or be able to do what is required of you. If you
make a habit of doing this, you will improve your chances of
understanding the contents of the course.
SELF-CHECK
ACTIVITY
The main ideas of each topic are listed in brief sentences to provide a review of
the content. You should ensure that you understand every statement listed. If
you do not, go back to the topic and find out what you do not know.
Key Terms discussed in the topic are placed at end of each topic to make you
aware of the main ideas. If you are unable to explain these terms, you should go
back to the topic to clarify.
DISCUSSION QUESTIONS:
At the end of each topic a list of questions is presented that are best solved
through group interaction and discussion. You can answer the questions
yourself. But, you are encouraged to work with your course-mates and discuss
online and during the seminar sessions.
At the end of each topic a list of articles and books is provided that is directly
related to the contents of the topic. As far as possible the articles and books
suggested for further reading are accessible at OUMÊs Digital Library. In
addition, relevant internet resources are available to enhance your understanding
of selected curriculum concepts and principles as applied in real-world situations.
ASSESSMENT FORMAT
Please refer to myVLE.
Facilitator
Your facilitator will mark your assignments and provide assistance during the
course. Do not hesitate to discuss during the seminar sessions or online if:
Ć You do not understand any part of the course content or the assigned
readings;
Ć You have difficulty with the self-tests and activities; or
Ć You have a question or problem with the assignments.
(a) The most important step is to read the contents of this Course Guide
thoroughly.
(b) Organise a study schedule. Note the time you are expected to spend on
each topic the date for submission of assignments as well as seminar
and examination dates. These are stated in your Course Assessment
Guide. Put all this information in one place, such as your diary or a wall
calendar. Whatever method you choose to use, you should decide on
and jot down your own dates for working on each topic. You have some
flexibility as there are 10 topics spread over a period of 14 weeks.
(c) Once you have created your own study schedule, make every effort to
„stick to it‰. The main reason students are unable to cope is that they
get behind in their coursework.
3. When you have completed the topic, review the learning outcomes to
confirm that you have achieved them and are able to do what is required.
4. If you are confident, you can proceed to the next topic. Proceed topic by topic
through the course and try to pace your study so that you keep yourself on
schedule.
5. After completing all topics, review the course and prepare yourself for the
final examination. Check that you have achieved all topic learning outcomes
and the course objectives (listed in this Course Guide).
FINAL REMARKS
One again, welcome to the course. To maximise your gain from this course you
should try at all times to relate what you are studying with the real life. Look at
the environment in your institution and ask yourself whether they provide
opportunities for research. Most of the ideas, concepts and principles you learn
in this course have practical applications. It is important to realise that much of
what we do in education and training has to be based on sound theoretical
foundations. The contents of this course provide the principles for doing research
in education, whether it is in a school, college, university or training organisation.
We wish you success with the course and hope that you will find it interesting,
useful and relevant towards your development as a professional. We hope you
enjoy your experience with OUM and we would like to end with this saying,
„Education is a lifetime of learning, relearning and unlearning‰ by Alvin Toffler
(an adaptation).
INTRODUCTION
This guide explains the basis on which you will be assessed in this course during
the semester. It contains details of the facilitator-marked assignments, final
examination and participation required for the course.
One element in the assessment strategy of the course is that, all students should
have the same information as facilitators about the answers to be assessed.
Therefore, this guide also contains the marking criteria that facilitators will use in
assessing your work.
Please read through the whole guide at the beginning of the course.
ACADEMIC WRITING
(a) Plagiarism
(c) Referencing
All sources that you cite in your paper should be listed in the Reference
section at the end of your paper. HereÊs how you should do your Reference.
ASSESSMENT
Please refer to myVLE.
INTRODUCTION
„Research has shown that four out of five dentists interviewed used Plantoid
Herbal Toothpaste‰
„Years of research has shown that SlimTex Capsules will reduce your weight
in a matter of weeks‰
„Azlan is doing research browsing through consumer reports, catalogues,
brochures and market surveys before deciding what car to buy‰
„Kong Beng is surfing the internet doing research on the symptoms of HIV
AIDS‰
The word „research‰ has been used in many different ways and sometimes
rather loosely giving rise to confusion and sometimes with the intention to
deceive. The claim that the majority of dentists used the particular brand of
toothpaste is misleading as there is no mention as to how many dentists were
interviewed and whether they are representative of all dentists in the country.
With regards to the advertisement on slimming pills, research on the use of
various types of medicine and supplements by humans tend to be inconclusive. It
would be difficult to make a claim that it will reduce weight as there are many
other contributory factors such as gender, level of health, weight and so forth. In
the strictest sense of the term what Azlan and Kong Beng are doing is not
research. Looking up facts and writing them down is nothing more than fact
finding and fact transcribing (Leedy, 1974). There is even the tendency to call the
written work a „research report‰ which is not accurate. It may be a report but not
a research report. There needs to be a distinction between true research and the
accumulation of facts. Research is a way of thinking; it involves thinking what we
want to study, how we go about collecting data, analysing the data and deriving
conclusions.
ACTIVITY 1.1
Discuss the different ways in which the word „research‰ has been used
in the statements in the box.
Beliefs Intuition
Experience Science
(a) Beliefs
These are all superstitious beliefs people hold on to as though they are
fact. For example, wear your lucky shirt when making deals (the one you
strongly believe will bring you good fortune).
(b) Intuition
An approach to acquiring knowledge that is not based on reasoning
or inferring. Intuitive knowledge is not scientific but is knowledge that
originates from gut feeling or predictions by soothsayers, astrologers and
fortune-tellers.
(c) Authority
Knowledge that originates from persons or sources that are highly
respected. For example, various religions have a sacred text that represents
the facts, which are considered indisputable, final and cannot be challenged.
(d) Experience
This approach of acquiring knowledge is based on the statement which
says, „If I have experienced it, then it is valid and true‰. In other
words, only facts that are in agreement with experience are accepted, and
those that do not are rejected. However, reliance on experience has its
shortcomings because our perceptions of people, events and objects are
affected by many factors. For example, we constantly add, delete and
reconstruct our experiences.
(e) Rationalism
This approach uses reasoning to arrive at knowledge and assumes that
valid knowledge is acquired through correct reasoning.
(f) Science
It is a process that is followed in generating knowledge and has been
accepted as the best method of acquiring knowledge. It lists a series of steps
to be followed when acquiring knowledge using the scientific method.
However, it has been argued that strictly following the scientific method
prevents us from studying in depth human behaviour (we will discuss this
issue in Topics 7ă9 under qualitative research).
ACTIVITY 1.2
Scientific Controlled
Systematic process Theory, hypotheses
Inquiry or Investigation Presumed relations
Discovery Natural phenomena
The purpose of using the scientific method is to enable the researcher to describe
(the relationship between factors); to predict (given what is known we can we
predict what might happen); to control (when certain variables are manipulated,
does it lead to a particular condition), and to explain (can a theory be formulated
to explain the phenomena being investigated).
SELF-CHECK 1.1
ACTIVITY 1.3
Where does one find research problems in education? They are all around you!
There are abundant research problems or unresolved issues everywhere. Look at
the 14 year old who says, „I hate history‰. Do you know why? Do you want to
know why? In fact, whatever that arouses your interest for which there are as yet
no answers or are inconclusive have the potential of being a research problem.
Often one starts with a rather general, diffuse and even confused notion of the
problem. Do not worry, this is the nature and complexity of the research process.
It is the first step towards becoming a mature researcher. The following three
steps are to help you get a research problem.
Copyright © Open University Malaysia (OUM)
TOPIC 1 THE EDUCATIONAL RESEARCH PROCESS 9
(a) Step 1: Identify a broad problem in your area that is of interest to you
and related to your professional goals. Your research should ultimately
contribute to the corpus of knowledge in that area of interest. For example,
you may be interested in how young children learn mathematics; or
how to get teenagers interested in science; how to enhance the teaching
of moral education; how to improve training in the corporate sector using
e-Learning; or how a headmasterÊs leadership style affects teacher morale,
etc.
(b) Step 2: Systematic programme of reading within your broad area of study.
For example if you are interested in why children have difficulty in learning
mathematics, you could start with textbooks in the area or topics of
textbooks. Textbooks explain the basic concepts and facts related to the
issue and may cite research in the area which will be listed in the
„references‰ which you could further explore. Journals such as the Review
of Research in Education and Review of Educational Research provide
valuable information about a particular field as they review related
literature. Next, is to read articles in the relevant journals in the field. For
example if you are interested in reading research then you should read
„Journal of Reading‰ and the „Reading Research Quarterly‰. You should
look through the catalogue both in the library and from on-line resources
and identify the journals in your field.
Students do not adequately refer to journals in the field but instead tend to
cite from popular sources such as newspapers, magazines and speeches.
Journals report empirical evidence about the field you are interested in and
they indicate the current thinking about research and the trend of research
efforts in the field. This will give you a grasp of leading edge research in the
United States, Britain, Europe and Australia and how you might do the
same in Malaysia.
(c) Step 3: Relate your research problem to a theory in the field. A theory is an
explanation of events or phenomena or behaviour. For example, if you
are interested in finding out whether providing children with multimedia
presentations explaining science concepts will enhance understanding,
you may want to explore the underlying theories of visual learning.
Many phenomena in education are explained drawing upon theories from
cognitive psychology, sociology, psycholinguistics, management, computer
science and so forth. It should be remembered that theory provides the
direction for the research (we will discuss in more detail the role of theory
in Topic 2).
SELF-CHECK 1.2
SELF-CHECK 1.3
Your car will not start. You put forward the hypothesis that „the car does not
start because there is no petrol‰. You check the fuel gauge to either reject or
accept the hypothesis. If you find there is petrol, you reject the hypothesis.
Next, you hypothesise that „the car did not start because the spark plugs are
dirty‰. You check the spark plugs to determine if they are dirty and accept or
reject the hypothesis accordingly.
principals, than the survey would be the appropriate research design. If your
intention is to study inter-racial mixing among students in the school canteen,
than a qualitative approach using the observation technique might be more
appropriate.
Hence, the decision on which methodology to use will depend on the research
problem and the research questions or hypotheses. It is not good practice to
decide on a methodology and then work on the research questions. You have
to be able to state clearly what you intend to study and then decide on an
appropriate methodology. If you are clear about your research problem and
research questions, you will find it easy to get assistance from your supervisor
and other students in designing a study to find answers to your research
questions.
ACTIVITY 1.4
The following are research problems taken from research literature.
Study them carefully and construct one or two hypotheses or research
questions based on them.
(a) How do self-esteem and level of aspiration influence academic
achievement?
(b) Does providing learners with graphic organisers enhance their
understanding of science text materials?
(c) How does the organisational climate in schools affect teacher
satisfaction and morale?
ACTIVITY 1.5
You have stopped at a junction because the lights have just turned red.
But, on your left a motorcyclist and a car beat the lights. You are most
annoyed. Being a good social scientist, you ask yourself why people
beat traffic lights. If you were to convert the problem into a hypothesis,
which of the following would be the best hypothesis? Explain your
answer.
(a) Some people beat traffic lights more frequently than others.
(b) People who beat traffic lights are a danger to themselves and
other road users.
(c) Beating traffic lights is common in crowded cities.
(d) Road users are more likely to beat traffic lights when the traffic is
light than when it is heavy.
ACTIVITY 1.6
Put in another way, the DV is the variable predicted to, whereas the IV is
predicted from. The DV is the presumed effect, which varies with any change or
variation in the IV.
Now, let us look at operational definition of variables. „If you lead a good life,
you will not suffer‰. This is a specific prediction of the future, but it cannot be
scientifically tested. Such a prediction is not scientifically tested because we
cannot define it operationally. How do you define „good life‰ and how do you
define „suffer‰. According to Bridgman, 1927, operational definition means that
variables used in the study must be defined as it is used in the context of the
study and publicly observable. This is done to facilitate measurement and to
eliminate confusion. For example, when you state in your study that you are
studying „excellent principals‰, you should be able to explain what „excellent‰
means. Once the behaviours of an excellent principal have been identified the
operational definition will be unique to your study (see Figure 1.4).
Operational Definition
The person:
Excellent listens to teachers
Principal looks after the welfare of teachers
acknowledges effort
consults teachers
motivates teachers
Figure 1.4: Example of an operational definition of an excellent principal
However, it should be borne in mind that in education not all variables are
directly observable. For example, we cannot really observe learning, memory,
reasoning, and so forth. Though they cannot be observed they can be measured
to see their traces. With enough indirect evidence, researchers can make a
convincing case for the existence of these invisible variables (Mitchell & Jolley,
1988). For example, though we cannot observe learning directly, we can see its
effect on performance, i.e. we can operationally define learning as an increase in
performance. Thus, if we see students improve their performance after practicing
a task, we conclude that learning has occurred.
SELF-CHECK 1.4
1. What is a variable?
2. What is the difference between an Independent Variable and a
Dependent Variable? Give specific examples.
3. Why do you need to define variables operationally?
The word „research‰ has been used in many different ways and sometimes
rather loosely, giving rise to confusion and sometimes with the intention to
deceive.
There are six ways in acquiring knowledge and they are through: our beliefs,
intuition, authority, empiricism, rationalism and science.
The seven steps of the educational research process is a useful guide for
beginning researchers to follow.
Borg, W., & Borg, M. (1988). Educational research: An introduction. New York:
Longman.
Chapter 3: The research problem, research plan and pilot study. 71ă106.
Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart and Winston.
Chapter 2: Generating the research hypothesis. 14ă36.
Mikijanis, M., & Thom, D. (n.d.). The research question outline. Retrieved from:
http://kancrn.kckps.k12.ks.us/guide/question.html
INTRODUCTION
Without some viable theory to serve as a guide, many studies address trivial
questions or contribute nothing to the slow accumulation of knowledge
needed for advancement of the science of education.
Do you agree with the statement by W. Borg and M. Borg? Theory has not
been properly understood by some graduate students. Some students are of the
opinion that theory is not relevant to practice. Oftentimes, we hear students
remark that a particular course is „too theoretical‰ or they prefer courses that are
„practical-oriented and not too theoretical‰. Some go to the extent to denounce
theory as useless! Actually, this reflects a lack of understanding on what is theory
and what is practice or practical. According to W. Borg and M. Borg, theory
(a) According to Social Learning Theory by Albert Bandura, the observer will
imitate the modelÊs behaviour if the model possesses characteristics such as
talent, intelligence, power, good looks or popularity that the observer finds
attractive or desirable.
Note that each of these theories explain learning and the variables or factors that
determines learning. Assuming that each of these theories are true we can predict
that learning will take place when a student connects new information with
old information, is reinforced through a system of rewards and reproduces a
modelled behaviour if it brings pleasure. However, many areas of education
have virtually no theoretical foundation and have to rely on other behavioural
sciences, such as psychology, sociology, anthropology and social psychology.
Then, an experiment is set up in which young children are shown two identical
containers (A and B) with the same amount of water. Then, the water from
container B is poured into a flat container C (see Figure 2.2). Children are asked
whether the amount of water in container A is the same as container C (Is A =
C?). Based on these observations, the researcher concludes whether to confirm or
disconfirm the theory. If the theory is confirmed, children are unable to perform
conservation tasks. In other words, children replied that container C had more
water. If the theory is disconfirmed, then the majority of children answered that
the amount of water in container A and C is the same.
A B C
Figure 2.2: Containers A, B and C filled with water
SELF-CHECK 2.1
1. What is a theory?
2. Why are there few laws in the field of education?
3. What do you mean by confirming or disconfirming a theory?
4. Identify some theories in your field of interest.
5. What are the implications of PiagetÊs theory on childrenÊs cognition
in the classroom?
Achievement
Authority/ Power
Motivation (n-ach)
Motivation (n-pow)
Affiliation Motivation
(n-affil)
ACTIVITY 2.1
Until you know what others have done in your area and what has not been
done, you cannot convincingly carry out research that will contribute to
furthering knowledge in your field. Thus, the literature in any field forms the
foundation upon which all future work must be built. If you fail to build this
foundation of knowledge provided by the review of literature, your work is
likely to be shallow and naive, and will often duplicate what has already been
done better by someone else.
The review of literature is usually a standard topic of the research report, thesis
or dissertation. It is an account of the research done in the field of study. The
review forms an important topic in a thesis or dissertation where its purpose
is to provide the background to and justification for the research undertaken.
The review would usually consist of empirical studies done in the area being
investigated. It also includes theoretical positions or proposals related to the
study, which are not necessarily empirical in nature.
The aim of the literature review is to show what has been done in the field and
how your study relates to earlier research. It also indicates the approaches, the
samples used, the variables examined, the statistical procedure used and most
important of all, the findings obtained. The review gives an overview of the
findings of various previous studies. The review traces the general patterns of the
findings and the conclusions that can be made based on the findings.
It also provides an insight into how your study is similar or different from
previous studies. For example; Is your study an extension of what others have
done? Are you examining variables that have not been attended to in earlier
studies? Are you attempting to replicate earlier studies in a different cultural
context? Are you applying statistical procedures that have not been attempted by
others?
The review of literature requires you to locate, read and evaluate reports of
research as well as reports of opinions and proposals. The review must be
extensive and thorough because you are aiming to obtain a detailed account of
the topic being studied.
The review of literature helps the student in delimiting the research problem by
setting the parameters. By setting the limits of your study, you avoid being
questioned „why didnÊt you do this or do that?‰ You can confidently reply that
your study is confined to studying what you had set out to study. Delimiting the
research problem can be achieved if you read extensively and intensively the
problem you plan to investigate and from the literature specify clearly what is it
you want to study.
The literature also provides insight into the approaches and methodologies
adopted by different researchers. A common mistake made by students is to pay
attention to only the findings of studies. Besides findings, students should also
examine the methodologies used. There could be unique approaches adopted
which you might want to replicate in the Malaysian context. For example, in
most studies reviewed the sample used tended to be university students and
your study is an attempt to use secondary school students, which could be a
unique contribution to the field because it is different.
SELF-CHECK 2.2
(iii) Narrow your topic ă It is far more satisfying, to both the writer
and the reader, to restrict your topic and cover it in depth.
Comprehensiveness and narrowness of topic go hand in hand.
(iv) Write about what interests you ă If you are interested in the topic, you
are likely to already know something about it, which will make it
easier to gather information.
(ii) Search for helpful articles ă Some articles will contribute more than
others to your understanding of a topic. Sometimes you can find a
pivotal article that can serve as a foundation for your study (the
references will lead you to other similar studies).
(iv) „Read, Read, Read‰. That is the bottom line of doing a review. People
have different ways of doing a literature review. A common technique
used by many graduate students is to use a „Note Card‰ (see below).
It may be a rather old-fashioned technique, but has proven to be
most effective. Many graduate students will testify to this, despite
advancements in computer technology.
Read the easier articles first.
Identify (1) the problem statement, (2) the research questions or
hypotheses, (3) method used, (4) the findings, and (5) how the
findings were interpreted.
Jot down the contents of the article using the following Note Card
illustrated in Figure 2.4.
Title: …………………………………………………………………….
Author/s: ………………………………………………………………..
Source: …………………………………………………………………..
Methodology:…………………………………………………………….
Findings…………………………………………………………………
Conclusion:……………………………………………………………….
Comments:……………………………………………………………….
(v) Most important of all, avoid plagiarism. Give due recognition to the
works of other people. It does not cost anything to acknowledge
sources. In fact, it shows the breadth and depth of your review, and
the thoroughness of your work.
(b) The presentation is a mere listing of the studies without an attempt to show
how each study is similar or different. Use connectives such as however, on
the other hand, similarly, but and so forth.
(d) Hurriedly reviews the literature and relies too heavily upon secondary
sources.
(e) At times there is evidence to suggest that students have not read the
original works but instead have taken someoneÊs work and cited it as
though they had read the primary source.
(f) There is also evidence of „cut and paste‰ which SHOULD NOT be
encouraged. You must have read the original works and know in detail
every study that you cite.
(g) Articles or reports that are included are not critically evaluated. Critically
evaluate the research questions, the methodology used, the statistics used,
the conclusion arrived at and recommendations made by the researcher
(details about evaluation of articles is discussed in 2.7).
SELF-CHECK 2.3
2.7 SOURCES
A good literature review requires knowledge of the use of indexes and abstracts,
and the ability to conduct exhaustive bibliographic searches. You should be able
to organise the material meaningfully, describe, critique and relate each source to
the subject of the inquiry, and present the organised review logically and most
importantly to correctly cite all sources mentioned (Afolabi, 1992). Generally,
there are two main sources of materials:
How do you search for research articles, research reports, etc.? You can start by
referring to preliminary reference sources such as, indexes and abstracts intended
to help you identify and locate research articles and other primary sources of
information. The following are well-known indexes and abstracts in education:
Specialised Areas
(a) Child Development Abstracts and Bibliography
(b) Exceptional Child Education Resources (ECER)
(c) Education Administration Abstracts
(d) Physical Education Index
Other sources of information are theses and dissertations that have never been
published. The following are important sources that provide abstracts of mastersÊ
theses and doctoral dissertations in education:
ACTIVITY 2.2
What do you think are some of the problems graduate students face
when doing the Review of Literature for their theses or research projects?
(ii) Critique:
Was the reason for answering the research question or testing the
hypotheses convincing or just attempting to appeal to your emotion
or merely seeking endorsement from well-know authorities in the
field?
Did the research questions or hypotheses follow logically from
theory?
Was there a tendency to oversimplify the theories or studies
reviewed?
(ii) Critique:
Was it clear how the subjects were selected?
Was it adequately explained how the instrument or treatment was
administered?
Did the researcher discuss the issues of validity and reliability?
Was the design of the study appropriate? How did it reduce
different types of biases?
What were the independent and dependent variables?
Were the statistical procedures used appropriate?
Were the variables operationally defined?
If the study was an experiment, was the treatment explained in
sufficient detail? Could it have been done in another way?
(ii) Critique:
Were the results clearly reported and illustrated? (e.g. tables,
graphs).
Do the statistics test the predictions made in the „Introduction‰?
(ii) Critique:
Is the authorÊs way the only way to interpret the predicted results?
Can you explain any of the findings the author/s did not explain
or were unable to explain?
What are the weaknesses or limitations identified by the author or
which you found but were not mentioned?
SELF-CHECK 2.4
1. What are the major aspects of a study you would examine when
describing a research article?
2. When you critique the Methods section and the Results section,
what are you looking for?
Ć The review of literature shows what has been done in the field and how the
intended study relates to earlier research.
Ć The review of literature delimits the study; relates the methods used by
others; describes the recommendations of earlier works; and provides the
basis for the intended study.
Borg, W., & Borg, M. (1988). Educational research: An introduction. New York:
Longman.
Chapter 5: Reviewing the literature. 142ă191.
Tools for preparing literature review ă A web tutorial (n.d.). Retrieved from:
http://www.gwu.edu/~litrev/
Vaverka, K., & Fenn, S. (n.d.). Background research: The review of literature
Retrieved from: http://kancrn.kckps.k12.ks.us/guide/literature.html
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Describe what an experiment is;
2. Explain the components of an experiment in education;
3. Identify the threats to internal validity of experiments;
4. Explain how to control the extraneous variables that affect the
internal validity of experiments; and
5. Describe how random assignment is performed;
INTRODUCTION
„Remember more and think faster with BE SMART‰
„Rewarding pre-schoolers with chocolates has improved
attention in class‰
„The Mental Awareness Approach has proven to be an
effective way to help smokers give up the habit‰
„Cognitive therapy is an effective method for treating
drug addicts‰
„Enhancing self-esteem improves academic performance‰ Source:
faculty.washington.
edu/chudler
You may have come across these statements or somewhat similar statements
before. Note that each of them is making a claim that their proposed product,
technique or procedure is effective in enhancing human performance. Obviously,
you would like to know how they went about proving „effectiveness‰. Of
all available research methods, the experimental method is the best. You may
even have conducted science experiments in the laboratory or in the field!
The experimental method was originally used in the field of agriculture
where experiments were conducted to test the effectiveness of various kinds of
treatments such as fertilisers, water and sunlight on plant growth. The method is
also used in medical sciences especially in testing the effectiveness of various
kinds of drugs, procedures and therapy on patients. The experimental method is
widely used in education in which researchers observe the occurrence of a
phenomenon as a consequence of a particular action or intervention.
Hence, it is important that you use the word „effectiveness‰ carefully, as it only
applies if you are using the experimental method.
See Figure 3.1 which shows a simple experiment to determine whether teaching
young learners using analogies (e.g. blood circulation is like a river and its
tributaries) „causes‰ them to perform better academically in science („effect‰).
The experiment involves administering a treatment (Independent Variable) such
as teaching science using analogies. A pretest (Dependent Variable) is given
before the experiment and the same test or equivalent test is given after the
experiment. The differences between pretest scores and posttest scores will
determine whether teaching using analogies improves performance in science.
extraneous variable Y
Figure 3.1: A simple experiment
SELF-CHECK 3.1
Campbell and Stanley (1963) state that experiments are internally valid when the
obtained effect can be attributed to the manipulation of the independent variable.
In other words, if the effects (e.g. improved scores in mathematics) obtained in
the experiment are due only to the experimental conditions manipulated by the
researcher and not to any other variables (factors), the experiment has internal
validity. In any experiment there are always some other factors than the
independent variable (treatment) that can influence the observed effects
(dependent variable). These variables must be identified and dealt with, or at
least, held constant. Cook and Campbell (1979) list a number of factors that can
threaten the validity of experiments. It is important therefore, that you know
these threats in order to take the necessary steps to control their influence.
HISTORY
MATURATION
INTRUMENTATION
TESTING
Figure 3.2: Time interval between pretest and posttest and threats to internal validity
(a) The first is History which includes events that have occurred in the
subjectsÊ environment between the pre-test and the posttest that might
affect the scores. For example, the subjects may have experienced events
during the time lapse that affected their attitude and this is reflected in
the scores of the dependent measure. Generally, the longer the duration
between the pre- and the posttest, the greater the possibility of history
threatening internal validity. But even short time lapses can generate the
history effect.
(b) The second is Maturation in which subjects may change between the pre-
test and posttest resulting in subjects becoming more mature. The change
could be both biological and psychological such as age, learning, fatigue,
boredom and hunger that are not related to specific external events but
reside within the individual.
(d) The fourth is Testing whereby subjects remember the questions in the
pretest and if the same test is given as a posttest, the chances are they may
score higher in the posttest i.e. they have become „test-wise‰. The time
period between the pretest and the posttest should not be too short such
that subjects can recall.
SELF-CHECK 3.2
Subjects
Target selected
Population randomly
(N=70)
Group 2: Students taught by the
teacher (n=35)
23 34 26 91 73 93 83 59 50 51
76 79 54 45 65 13 11 56 91 27
68 57 37 38 45 45 04 85 66 12
45 25 98 63 52 23 03 36 06 08
89 3 39 34 91 94 12 39 13 31
90 26 83 26 21 34 82 07 34 67
23 61 64 65 37 06 54 26 29 75
87 82 51 02 95 64 62 35 96 49
90 71 25 86 62 39 53 49 48 52
12 38 67 09 67 31 45 40 28 31
SELF-CHECK 3.3
powerful way to ensure that subjects assigned to the various groups will have
more or less similar characteristics. Other techniques can be used to increase the
probability of subjects in two or more groups being equivalent.
3.4.1 Matching
Determine a particular factor, for example, academic performance, that can be
measurable and categorised as High and Low. From the sample, select two
High Ability subjects and randomly assign them to the control group and the
experimental group. Next, select two Low Ability subjects and assign them
randomly to the control group and the experimental. Continue doing this until
all subjects have been assigned and your two groups are matched in terms of
academic performance.
Another technique of matching is to give the pretest, and based on the scores
obtained, assign subjects to the control group and the experimental group.
However, you should ensure that the average score or mean score of the pretest
should be the same for the two groups. e.g. two subjects with mean of 23; two
subjects with mean 30; two subjects with mean 34 and so forth.
Sample of subjects
from low
socioeconomic status
Random Assignment
Group 1 Group 2
„You should keep in mind that ANCOVA is an imperfect statistical technique for
equating experimental groups prior to the treatment period. Only the variables
that are measured can be used as covariates. The groups may differ on other
variables, but if these variables have not been measured, they cannot be entered
into the ANCOVA‰ (Borg, W., & Borg, M. (1988). Educational research: An
introduction, p. 684).
SELF-CHECK 3.4
There are two types of hypothesis, the null hypothesis and the alternative
hypothesis; (the latter denoted by H1 the null hypothesis denoted by H0).
Ho: 1 = 2
OR
Ho: 1 ă 2 = 0
Where ø1 is the mean test score for the experimental group (Group 1);
and ø2 is the mean test score for the control group (Group 2)
In this example, since the researcher conjectures that the inductive approach
helps in improving creative thinking ability, the alternative hypothesis therefore
can be statistically expressed as follows:
Ho: 1 > 2
OR
Ho: 1 ă 2 > 0
Mean Standard
Deviation
Experimental Group 30.4 3.7
Control Broup 28.3 4.1
To the naive person, he or she might conclude that the experimental group
performed better than the control group because the mean score is higher by 2.1,
thus proving that the treatment is effective. This is misleading because it is likely
that the differences in the mean between the experimental group and control
group could have occurred by chance. In order for you to accept or reject the null
hypothesis, it is necessary that you analyse the data statistically because you
want to be sure that the treatment administered produced a real effect. How
do you determine that the difference between the two groups is caused by
the treatment and not some other extraneous variable? You could repeat the
experiment and see if you get the same results which will provide evidence of the
reliability of the obtained findings.
However, this is not an economical approach and for this reason statistical tests
are preferred. The test of significance enables one to determine whether the
amount of difference between the two groups is due to chance or due to the
treatment. Does a large difference between the mean score of the experimental
and control group indicate that the difference is real? Even large differences
could occur by chance, although the probability of this happening would be
very low. The most common practice is to state a significance level that must
Copyright © Open University Malaysia (OUM)
TOPIC 3 EXPERIMENTAL METHODOLOGY 53
If you decide from the onset of the experiment that the .05 significance level is to
be used, it means that you will accept as a real difference only one that is so large
that it could have occurred by chance only 5 times in 100 (i.e. 95% not due to
chance). If the .01 significance level is selected, then the difference can be
expected to occur only 1 time in 100 by chance (i.e. 99% not due to chance).
SELF-CHECK 3.5
ACTIVITY 3.1
The table above shows the pretest and posttest means on a critical
thinking skills test. The subjects were taught critical thinking skills for
one period (40 minutes) a week for six weeks.
1. Give a title for the study.
2. State TWO null hypotheses based on the data above.
3. State TWO conclusions based on the findings.
4. What are the independent and dependent variables?
5. Provide an operational definition for the treatment.
Random assignment means that each sampling unit has an equal chance of
being selected in the experiment.
ANCOVA Matching
Control & experimental groups Maturation
Directional hypothesis Null hypothesis
Equivalent groups Pretest & posttest
Experimenter effect Random assignment
History Selection
Holding a variable Subject effects
Hypothesis testing Table of random numbers
Including the variable Test of significance
Instrumentation Testing
Internal validity The experiment
Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart & Winston.
Chapter 4: Internal validity: Why researchers value experimental designs.
INTRODUCTION
What is meant by research design? According to Christensen (1988) the „research
design refers to the outline, plan, or strategy specifying the procedure to be
used in seeking an answer to the research question. It specifies „how to collect
and analyse the data‰ (p. 219). The design of an experiment will show how
extraneous variables are controlled or included in the study (refer to the control
techniques discussed in Topic 3). The design will determine the types of analysis
that can be done to answer your research questions and the conclusions that can
be drawn. To what extent your design is good or bad will depend on whether
you are able to get the answers to your research questions. If your design is
faulty, the results of the experiment will also be faulty. How do you go about
getting a good research design that will provide answers to the questions asked?
It is not easy and there is no fixed way to go about it. The logical thing to do
would be to examine different research designs, discover their strengths and
weaknesses, and make a decision.
X O
(praise) (scores on a
mathematics test)
You conclude that praising children increases their mathematics score. This
design is weak for the following reasons:
(a) Selection Bias: It is possible that the pupils you selected as subjects were
already good in mathematics.
(b) History: The school had organised a motivation course on mathematics for
year 4 pupils. So, it is possible it might influence their performance.
O1 X O2
(mathematics pretest) (praise) (mathematics posttest)
Figure 4.2: One-group pretest-posttest design
ACTIVITY 4.1
Twenty pupils who were weak in arithmetic were taught using the
Zandox method. Three weeks later when they were tested, their
arithmetic scores improved. Thus the Zandox method improves
arithmetic performance.
(a) Which type of research design is this study based on?
(b) What are some problems with this design?
EXPERIMENTAL GROUP X O
- - - - - - - - - -- - - - - - - - - - - - - - -- - - - -- - - - - - - - - -- - - - -- - - - - - - - - -
O
CONTROL GROUP (Praise) (mathematics posttest)
The three designs described are „weak‰ research designs because they do not
allow for the controlling of extraneous factors that might creep into the
experiment. These extraneous factors may affect the results of the dependent
measure. For example, if attitude towards mathematics and outside tuition in
mathematics are not controlled it may not be possible to conclude that „praise‰
(treatment) affects mathematics performance (dependent variable). Also, weak
research designs do not attempt to randomly assign subjects to the groups being
compared, which could introduce extraneous factors affecting the dependent
measure. Random assignment controls for both known and unknown extraneous
variables might affect the results of the experiment.
SELF-CHECK 4.1
ACTIVITY 4.2
In this topic, we will discuss two major types of true designs (see Figure 4.4):
(a) After-only design; and
(b) Before-after design.
What is the difference between the two designs? The after-only design relies only
a posttest while the before-after design (as the name suggests) relies on both a
pretest and a posttest.
Copyright © Open University Malaysia (OUM)
62 TOPIC 4 EXPERIMENTAL RESEARCH DESIGNS
After-Only Research
Design Factorial
Design
True Designs
Before-After Research
Design
Treatment Posttest
(inductive Method) (Science Problem
Solving Test Scores)
EXPERIMENTAL GROUP R X O
CONTROL GROUP R ă O
SELF-CHECK 4.2
Independent Variable A
METHODS
Inductive Deductive
(A1) (A2)
Independent ACADEMIC
High (B1) A1 B1 A2 B1
Variable B ABILITY
Low (B2) A1 B2 A2 B2
So, there are four possible combinations of the two independent variables. Each
of these treatment combinations are referred to as cells (i.e. A1B1; A2B1; A2B1
and A2B2). Subjects are randomly assigned to these four cells within the design.
For the experiment using this factorial design, you are looking for THREE
different kinds of effects: the main effect of the method, the main effect for ability
and the interaction between method and ability.
Figure 4.7: Factorial design showing means for ability and method
The results of the hypothetical experiment are shown in Figure 4.7. The main
effects for methods (variable A) showed that there was a significant difference in
History performance between students taught the inductive (Mean = 50.0) and
the deductive approach (M = 40.0). This means that the method had an „effect‰
on History performance. There was also a main effect for ability (variable B)
where a significant difference was observed between high (M = 55.0) and low
ability (M = 35.0) students on performance in history. Similarly, it means that
ability had an „effect‰ on History performance.
1. A main effect for method, no main effect for ability and No interaction.
2. No main effect for method, a main effect for ability and No interaction.
3. A main effect for method, a main affect for ability and No interaction.
4. A main effect for method, a main effect for ability and an interaction.
5. No main effect for either method or ability, but an interaction.
6. A main effect for method, no main effect for ability, and an interaction.
7. No main effect for method, a main effect for ability and an interaction.
8. No main effects (method & ability) or interaction.
(a) Main Effect for Method and Main Effect for Ability and No Interaction (No: 3)
Let us examine what this means with our hypothetical example. The data in
Figure 4.7 indicates that you have main effects for both method and ability.
Look at the first row. You can see that high ability learners treated with the
inductive method (M = 60.0) scored higher than high ability learners treated
with the deductive method (M = 50.0). Looking at the next row, you see that
low ability learners treated with the inductive method (M = 40.0) scored
higher than low ability learners treated with the deductive method (30.0).
You can see this in Figure 4.8.
60 High ability
Score
on the
History 50
Test
40
Low ability
30
Deductive Inductive
METHOD
Figure 4.9: Graph showing no interaction
Looking at the columns tells you about the effect of ability. You see
that high ability learners treated with the inductive method scored 60.0
compared to their low ability counterparts who scored 40.0 (i.e. 20 more) in
the same treatment group. Looking at the second column, you learn that
high ability learners (M = 50.0) treated with the deductive method scored
higher than low ability learners (M = 30.0) treated with the same method.
Thus, it appears that in addition to the method main effect, you have an
ability main effect.
Finally, you also know that there is no interaction because method has not
affected the ability level of students. As Figure 4.7 demonstrates, the effect
of method is independent of ability level and the effect of ability level is
independent of method of instruction. If you graph the means, your graph
should look something like Figure 4.8. The graph confirms what you saw in
Figure 4.7. The high ability line is above the low ability line. Similarly,
ability increases as shown by the fact that both lines slope upwards as they
go from deductive to inductive method. Finally, the graph tells you that
there is no interaction between method and ability on performance in
History because the lines are parallel.
(b) No Main Effect for Method and No Main Effect for Ability but an
Interaction (No: 7)
Let us examine what this means to our hypothetical experiment. According
to Figure 4.10, the means obtained on History performance according to
method reveals no significant difference between the inductive method
(M = 55.0) and the deductive method (M = 55.0). Similarly, for ability there
was no significance difference between high ability and low ability
students. However, there was an interaction and the interaction was
significant (see Figure 4.11). In this figure, you notice that the lines are not
parallel (as in Figure 4.9). Therefore, you have an interaction. What is the
meaning of this interaction since there was no effect for either method or
ability? You could say that method has an effect, but its effect depends on
ability level. Alternatively, you could say that ability has an effect but that
effect depends on the type of method students had been treated with.
Figure 4.10: Factorial design showing means for methods and ability
60 High ability
Score
on the
History 50 Low ability
Test
40
Deductive Inductive
METHOD
Figure 4.11: Graph showing an interaction
SELF-CHECK 4.3
ACTIVITY 4.3
Experimental Group R O X O
Control Group R O - O
For example, sometimes it is not possible to assign students to groups which are
a requirement of strong experimental research. Due to logistical reasons it is
difficult to randomly assign subjects to groups and so, intact groups such as a
class may have to be used. Is it still possible to do an experiment despite these
limitations? The answer is „yes‰; you can use a quasi-experimental design.
Experimental Group O X O
Control Group O - O
In the Nonequivalent control-group design both groups are given first a pretest
and then a posttest [after the treatment is given to the experimental group]. The
pretest score and the posttest score are compared to determine if there are
significant differences.
When you cannot randomly assign subjects, you can be sure that extraneous
variables or factors will creep into the experiment and threaten its internal
validity. (We have discussed in Topic 3, the factors that threaten the internal
validity of experiments). Do you leave it alone or do something about it?
Cook and Campbell (1979) proposed the following steps to enhance the internal
validity of the non-equivalent control-group design or quasi-experiments in
general:
(a) Selection: ensure that subjects in the experimental and control are
matched in terms of important variables that may affect the results of
the experiment. For example; match subjects in terms of academic ability,
IQ, attitudes, interests, gender, socioeconomic background and so forth).
(b) Testing: ensure that the time period between the pretest and posttest is not
too short such that subjects are able to remember the questions given to
them earlier.
(c) History: ensure that events outside the experiment do not affect the
experiment. The problem is most serious when only subjects from one of
the groups are exposed to such events (e.g. motivation talks, private tuition)
(d) Instrumentation: ensure that the pretest and the posttest are similar. If
a different test is used, you should make sure that the two tests are
equivalent in terms of what it is measuring (i.e. high reliability and
validity).
A hypothetical example may illustrate how the interrupted time series design is
used. Say that you want to determine whether positive reinforcement encourages
inattentive low ability learners to be more attentive. You identify a group of
11 year old low ability learners and get them to attend an experimental classroom
for at least one period each school day (see Figure 4.14). In this classroom,
subjects are taught reading skills in a positive environment where they are
praised and rewarded for attentive behaviour that is focussed on the given task
or activities. Before the students are sent to the positive treatment classroom their
behaviour is observed over three sessions in their regular classroom with regards
to their attentiveness. This is to obtain baseline data where their behaviour is
recorded in its freely occurring state. The treatment lasts for three weeks and
after the treatment, subjects are observed for their attentiveness and focussed
behaviour.
Multiple Multiple
PRETESTS TREATMENT POSTTESTS
O1 O2 O3 X O4 O5 O6
Percentage
of 60
students
who were
attentive
40
20
1 2 3 4 5 6
Sessions
Figure 4.15: Percentage of students observed to be attentive and focussed
SELF-CHECK 4.4
Any researcher conducting an experiment must ensure that the dignity and
welfare of the subjects are maintained. The American Psychological Association
published the Ethical Principles in the Conduct of Research with Human
Participants in 1982. The document listed the following principles:
(a) In planning a study, the researcher must take responsibility to ensure that
the study respects human values and protect the rights of human subjects.
(b) The researcher should determine the degree of risk imposed on subjects by
the study (e.g. stress on subjects, subjects required to take drugs).
(c) The principal researcher is responsible for the ethical conduct of the study
and be responsible for assistants or other researchers involved.
(d) The researcher should make it clear to the subjects before they participate in
the study regarding their obligations and responsibilities. The researcher
should inform subjects of all aspects of the research that might influence
their decision to participate.
(e) If the researcher cannot tell everything about the experiment because it is
too technical or it will affect the study, then the researcher must inform
subjects after the experiment.
(g) The researcher should protect subjects from physical and mental
discomfort, harm, and danger that may arise from the experiment. If there
are risks involved, the researcher must inform the subjects of that fact.
SELF-CHECK 4.5
Ć A quasi-experimental design is a design that does not provide for full control
of potential confounding variables.
Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart & Winston.
Chapter 7: Expanding the simple experiment: factorial designs.
INTRODUCTION
Do you have a curiosity as to what people are thinking, feeling or doing about
various aspects of life? If you do, than the survey would be the best research
method for finding out. You could survey them orally through an interview or
you could give them a questionnaire. By asking questions, you are tapping into
their attitudes, beliefs, and behaviour. The survey has proven to be a powerful
tool in gathering information about peopleÊs perceptions of and intentions
concerning different social, cultural, economic and political issues.
It is quite common for newspapers, the radio and television to report on the
opinions and perceptions of people obtained through surveys (at times using
survey data to sensationalise issues). Surveys are used by various government
agencies, non-governmental agencies, business and scientific organisations
probing into peopleÊs reactions to different issues and trends. For example:
(a) TV stations rely on surveys to profile the people who watch programmes
over their stations.
(b) Automobile companies use surveys to find out customer satisfaction with
their cars.
(c) Statistics departments conduct surveys to measure the consumer price
index with the purpose of advising policy.
(d) Engineering companies gather information about the habits of road users.
(e) Magazine companies conduct surveys to find out about the reading habits
and interests of their subscribers.
(f) Manufacturing companies conduct surveys to determine consumer
acceptance of their products.
The survey was designed to allow children to reflect on and rank their parentÊs
ability to show respect and love, have integrity and patience and be open-
minded, among a myriad of other skills and characteristics.
Although the Malaysian dad ranks lower than the Malaysian mum, a general
analysis of the survey results reveals fascinating details of Malaysian fathers as
seen through the eyes of young adults. For instance:
(a) Dad got a B for telling jokes and C for being fashionable.
(b) Dad does not really allow them much independence (Că) but respects their
privacy (B) and trusts them (B).
(c) Dad takes the trouble to teach them right from wrong (B+).
(d) Dad accepts their own taste in music (B).
(e) Dad talking about sex (D).
ACTIVITY 5.1
(a) What they know (Who was the first Prime Minister of Malaysia?)
(b) What they believe (Should students be given freedom to express
themselves?)
(c) What they expect (Do you expect to be a famous person?)
(d) What they feel (Do you think your school principal is fair?)
(e) What they have done (How often do you use the computer in a week?)
(f) What they plan (Do you intend to continue studying or start working?)
SELF-CHECK 5.1
5.3 SAMPLING
Unlike a census, where all members of the population are studied, surveys gather
information from only a portion of the population of interest ă the size of the
sample will depend on the purpose of the study we will discuss this issue later.
In a good survey the sample is not selected haphazardly or only from persons
who volunteer to participate. It is scientifically chosen so that each person in the
population will have a measurable chance of being selected. This procedure is
called sampling. This way the results can be reliably projected from the sample to
the larger population. Two key words involved in sampling are population and
sample. The word population is defined as all people, objects or events found in a
particular group the researcher is planning to generalise on (Borg & Borg, 1983).
For example, the population of primary school students; graduate teachers;
medical doctors in Malaysia; senior citizens in Selangor; secondary school
students in Sarawak; and so forth.
If the population is small (e.g. school principals in Kuala Lumpur), the researcher
may decide to study all the subjects. When the population is large, the researcher
is not able to study all individuals. It would be too expensive and time
consuming even for large research organisations. Only in a census is the whole
population studied involving enormous expenditure, time and many research
assistants. Just imagine having to administer a questionnaire to 100,000 sixteen
year olds in Malaysian schools!
So the researcher has to select a certain number of subjects or a sample from the
population to study. Regardless of the method used in selection of the sample,
the most important thing is to ensure that the sample is representative of the
population. See Figure 5.2 which shows a population of 10,000 individuals who
possess certain characteristics in terms of gender, socioeconomic status and
ethnicity. If you intend to select 5% of individuals from the population to form a
sample, you should ensure that the sample has similar characteristics to the
population. This is called population validity where the researcher is trying to
show that the sample is representative of the population according to the
variables specified. Using a sample is more economical and if the sample is
selected appropriately, the researcher can make conclusions about the population
based on the results from the sample. This is called generalisation.
POPULATION:
(Total = 10,000 persons)
Characteristics:
- Gender: 5100 males & 4900 females
- Socioeconomic status: (2500 high, 4000
middle, 3500 low)
- Ethnicity: 5000 Malays, 3000 Chinese,
1000 Indians, 1000 Ibans.
GENERALISATION
SAMPLING
ACTIVITY 5.2
Refer to Figure 5.2 and explain how the sample selected is representative
of the population.
Even though the sample is selected from the accessible population, you
may want to know the degree to which the results can be generalised to the
target population which requires two steps. In the first step, you generalise
from the results of the sample to the accessible population (teachers in
Perak). Second, you generalise from the accessible population to the target
population (graduate teachers in Malaysia). The leap from sample to the
accessible population presents no problem if a random sample of the
accessible population is obtained.
1 2 3 4 5 6 7 8 9 10
1 2345 3445 2678 9156 7332 9332 8345 5950 5023 5189
2 7612 7989 5456 4523 6516 1345 1123 5636 9189 27452
3 6823 5732 3702 3808 4553 4589 0467 8506 6612 12136
4 4598 2564 9860 6360 5245 2347 0391 3623 0620 0850
5 8956 357 3934 3495 9112 9472 1254 3998 1390 3194
6 9059 2691 8395 2634 2189 3465 8223 0745 3487 6709
7 2312 6120 6425 6556 3720 0639 5490 2614 2950 7556
8 8787 8236 5153 0202 9530 6490 6220 3523 9691 4917
9 9063 7190 2590 8696 6267 3923 5360 4937 4854 5223
10 1298 3820 6737 0932 6719 3154 4532 4046 2860 3191
After the required sample size has been calculated, every Nth record is
selected from a list of population members. As long as the list does not
contain any hidden order, this sampling method is as good as the random
sampling method. Its only advantage over the random sampling technique
is simplicity. Systematic sampling is frequently used to select a specified
number of records from a computer file.
Males Females
High achievers n=? n=?
Low achievers n=? n=?
ACTIVITY 5.3
You have been appointed to lead a research team assigned with the
task of finding the reasons teenagers smoke. The team has decided to
conduct a nation-wide survey involving students between 14ă16 years
of age in secondary schools.
1. Suggest THREE research questions that will guide data collection.
2. Explain how you plan to draw the sample of students using
stratified sampling. What subgroups would you include?
3. What further information do you need to draw a representative
sample?
SELF-CHECK 5.2
ACTIVITY 5.4
N=118
N=134 N=105 N=113
N=154
N=129 N=98
N=111 N=109
SELF-CHECK 5.3
The following are some guidelines you can use in deciding how large your
sample should be:
(a) When the sample selected has to be broken down into smaller groups
involving comparisons of groups, a larger sample is advisable. For
example, you want to compare the self-esteem of males from low
socioeconomic backgrounds from single-parent families against males from
high socioeconomic backgrounds from single-parent families.
(b) When attrition is expected, especially in longitudinal studies; the longer the
duration of a study, the higher will be the number of subjects who drop out.
To reduce attrition you should keep demands on subjects to the minimum,
fully inform subjects about the study, obtain a strong commitment from
subjects before the study and make frequent contacts with subjects to
maintain interest and rapport.
SELF-CHECK 5.4
What are the factors you will consider when making decisions about
sample size in a survey?
ACTIVITY 5.5
The seven steps for conducting a survey are (see Figure 5.4):
This initial planning requires some idea of the final analysis; and it may be
useful at the outset to outline the key tables for the final report (better still if
you are able to create dummy tables), and to consider the number of cases
expected in each major cell or subgroup. Conduct a review of the literature
to define terms and justify the theory or theories underlying research
questions.
(i) You have to decide the procedures for conducting the survey; how
many people you will survey (the sample size and how they will be
obtained)
(ii) Decide how you will survey your subjects (by phone, in class, mail
format, interview).
(iii) Decide how you intend to distribute and collect the questionnaires;
make follow-up contacts.
(iv) The pilot-test helps you find out if the content and form of the
questions are satisfactory. You can also get information about:
The length of time needed to complete the questionnaire;
The appropriate order of the questions;
Sufficient space needed for responses;
Whether the instructions or directions are understood;
The extent to which the questionnaire is reliable, e.g. if you gave
the questionnaire again four weeks later would you get the same
responses?; and
The validity of the questionnaire (does it measure what it is
supposed to measure?) We will discuss further these issues in
Topic 6.
(i) You should gain approval from the respondents you intend to survey.
For example, permission from the Ministry of Education (Educational,
Planning and Research Division), State Education Departments and
school principals should be sought.
(ii) In a small study you can administer the questionnaire yourself. But
in a large survey, you will need helpers. Your field assistants have
to be trained, especially if they are required to observe or conduct
interviews. Such training should be completed before the start of the
main study.
(v) Do not administer the questionnaire on a day when there are school
activities such as open day, celebration of festivals, sports events or
examinations. Students will be distracted with these events and may
not give the survey their full attention.
(vi) If you are surveying parents, you could send the questionnaire home
with the students. How can you ensure receiving enough returned
surveys? One suggestion is to reward students for returning
questionnaires, e.g. a coupon for a free soft drink.
(vii) Thank the groups that assisted in the survey process. An appreciation
goes a long way toward support and participation.
(i) Coding and entry of data. Check to ensure accuracy of data entry
and ensure that all codes are valid (for example, „1‰ is for male, „2‰
for female and there should not be a „3‰!) and look for any
inconsistencies.
(iii) If you are unclear of certain statistical procedures, obtain advice from
a person who is well-versed in statistics.
(i) The introduction (background of the study, rationale for the study,
the aims and research questions, limitations and significance of the
survey);
(ii) The review of literature (previous work done in the field, underlying
theory or core of theories);
(iv) Results (data presented in the form of tables and graphs; statistical
analysis; description of the findings);
SELF-CHECK 5.5
association. The disadvantage of the mail survey is the low response rate.
Also, since the researcher is not present, there is no way for the respondent
to seek clarification if questions are unclear.
(a) As far as possible use only number codes to link the respondent to a
questionnaire and store the name-to-code linkage information separately
from the questionnaires. For example 001 for respondent Azlina Darus, 002
for respondent Ong Mei Ling and so forth.
(b) The names and addresses of survey respondents should not be made
available to anyone outside those involved in the survey after the responses
have been entered into the computer (individuals and organisations have
been known to sell such databases to companies for marketing purposes
without the consent of individuals involved!).
(c) Omitting the names and addresses of survey respondents from computer
files used for analysis.
Respondents should be informed about the purpose of the survey and have the
option not to participate or not to divulge information that he or she feels not
comfortable with. For example, respondents may be reluctant to disclose their
incomes. To overcome this, you may want to use categories (e.g. RM1500ă
RM2000 per month) which may be less intrusive. You should determine in the
pilot-test which items respondents are uncomfortable with, so that you do not
have too many unanswered questions to the point that some research questions
cannot be answered. The questions asked should not in any way attempt to
deceive respondents. The integrity of a survey is enhanced if respondents are
clear about the purpose of the study.
Ć Surveys are useful in gathering data about what people are thinking, feeling
or doing.
Ć Since the entire population would be too costly and time consuming to
survey, a sample is drawn from the population.
Ć There are two methods of data collection using the questionnaire: self-
administered survey and investigator administered survey.
Ć The self administered survey is the mail survey and web survey while
investigator administered surveys include telephone interviews, face-to-face
interviews and group administered questionnaires.
Borg, W., & Borg, M. (1988). Educational research: An introduction. New York:
Longman.
Chapter 6: Population and sample.
Mitchell, M., & Jolley, J. (1998). Research design explained. New York: Holt,
Rinehart and Winston.
Chapter 12: An introduction to survey research. 284ă311.
6 and Data
Analysis
LEARNING OUTCOMES
By the end of this topic, you should be able to:
1. Differentiate between instrument and instrumentation;
2. Define what a questionnaire is;
3. Explain the construction of a questionnaire;
4. Design and develop an attitude scale;
5. Analyse data to show significant differences; and
6. Analyse data to show correlations or relationships.
INTRODUCTION
The most important and sometimes difficult aspect of educational research is
determining the instruments to be used in data collection (see the Self-Esteem
Test in Figure 6.1 which is an example of an instrument). There are numerous
tools that may be used to obtain information. In this topic we will discuss in
detail the process of instrumentation and look at some examples of widely
used types of instruments. We will also discuss two very important concepts
related to instrumentation, namely; reliability and validity. Fraenkel and
Wallen (2001) make a distinction between instrument and instrumentation. An
instrument is a device or procedure for systematically collecting information
while instrumentation includes both the instrument and „the conditions under
which it is used, when it is used, and by whom it is used‰ (p. 81). Examples of
instruments are tests, questionnaires, rating scales, inventories and checklists.
Self-Esteem Test
This test is designed to test your level of self-esteem. Read each item and state whether
you strongly agree, agree, disagree or strongly disagree with each statement by
referring to yourself.
Strongly Strongly
Agree Agree Disagree Disagree
4 3 2 1
1. I am an attractive person.
4. I like my hair.
Source: Reprinted with permission: © CoPS Project Malaysia (2005). John Arul Phillips
ACTIVITY 6.1
Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. There is greater standardisation in
questionnaires as each respondent receives the same set of questions which
allows interpreting from a large sample. The questionnaire if properly designed
can address many issues in a relatively efficient way. Also, there is assurance of
anonymity which increases the likelihood that the responses are genuine and
reflective of the opinions, perceptions and beliefs of respondents. However, the
quality of data obtained from questionnaires will depend on how well it is
designed. Fowler (1984) suggests the following important criteria when designing
a questionnaire:
(a) Is this a question that can be asked exactly the way it is written?
(b) Is this a question that will mean the same thing to everyone?
(c) Is this a question that people can answer?
(d) Is this a question that people will be willing to answer?
Designing a questionnaire is both an art and a science; and takes time and
precision. Before setting out to design a questionnaire from scratch, it is a good
idea to find out whether such questionnaires already exist that could be used or
modified to gather the information that you seek. A little bit of research could
save you a lot of work in creating an entirely new questionnaire.
In designing a questionnaire you have to determine the questions that you want
to ask; select the question response format for each question, decide on question
sequence and overall layout, pilot-test and implement the instrument. Deciding
what questions to ask will depend on the aims of your survey. For example, if the
aim of your study is to survey the reading habits and interests of teenagers, one
of the questions you will ask is: „What kinds of books do you like to read?‰ You
should ensure that there is a close link between the research aims and the
individual questions asked. You do not want a situation where the data collected
does not answer the research questions.
(a) Dichotomous questions: These items require two possible responses such as
Agree/Disagree or True/False or Yes/No.
(b) Check Only One Answer from a List: The respondent is required to check
or tick only ONE answer.
(c) Checking More than One Answer from a List: The respondent is required to
place a check or tick next to one or more answers.
(d) Ranked items: Respondents can also be asked their preferences by ranking
the items from 1, 2 and so forth. We want the respondent to put a 1, 2, 3, 4
or 5 next to the brand, where 1 is the respondentÊs first choice, 2 the second
choice and so forth.
Proton _____
Toyota _____
Honda _____
Nissan _____
Ford _____
(e) Rated responses: The Likert Scale has become a popular tool in getting people
to respond to questions (The Self-Esteem Test presented at the beginning of the
Topic consists of items using the Likert Scale). The Likert scale is a rating scale
that measures the strength of agreement with a statement.
AGREEMENT
Strongly Strongly
Disagree Disagree Undecided Agree Agree
1 2 3 4 5
FREQUENCY
Very
Rarely Very rarely Occasionally Frequently Frequently
1 2 3 4 5
IMPORTANCE
Of little Moderately Very
Unimportant importance important Important important
1 2 3 4 5
(f) Filter questions: This format is used when you want to obtain information
from people about the habits (whether they „do‰ or „do not‰ engage in
something), beliefs in or opinions of something (i.e. you are filtering).
Such questions would provide many kinds of responses. Some would be long
answers while others would be just a phrase. If you are interested in getting a
variety of reasons and also some in depth reasons, then the unstructured or open-
ended format would be useful. Unstructured or open questions should be used
when you feel that the particular question cannot be categorised to include all
possible responses. However, if you are concerned about the time-consuming
task of processing many different responses, than closed questions should be
used; but, you should be aware of the disadvantages of such questions.
SELF-CHECK 6.1
(c) Instructions:
(i) Instructions on how to answer the questions should be clear and
concise, using words that are not difficult to understand.
(ii) State the purpose of the survey and identify who is administering or
sponsoring the survey.
(iii) Indicate how confidentiality is protected. This assurance may increase
the likelihood of honest responses.
(iv) Indicate who the respondent can call or write to if they have
questions, concerns or want a copy of the survey results.
(ii) Group together questions that are similar in terms of format and the
specifics covered (e.g. group together questions asking respondents
about their background such as occupation, income, age, etc.).
(iii) The transition from one question to another should not be too abrupt.
Avoid questions that jump from one unrelated topic to another.
(e) Scales:
(i) When you want subjects to respond to a scale from lower to higher, it
is usually better to place the lower end of the scale on the left and the
higher end of the scale to the right.
1. Never 2. Seldom 3. Occasionally 4. Always
(ii) Avoid having too many divisions in your scale which strains the
respondent (see example below). Usually a 4 or 5 point scale would be
adequate.
1. Never 2. Seldom 3. Occasionally 4. Fairly often
5. Often 6. Very Often 7. Always
(ii) Watch out for words that are ambiguous or have more than one
meaning:
(iii) Avoid jargon, acronyms or terms that only a few people may know.
Acronyms should be expanded or stated in full unless the target
audience commonly knows them.
Do you favour inclusion? (Are you sure most of your subjects
know what „inclusion‰ is?).
What are your SPM results? (Foreigners may not know what SPM
stands for).
(iv) Use simple and direct language which is easily understood by the
respondent. Do not use words that will probably not be understood to
avoid misinterpretation. You could bold or underline words that you
want to emphasise to eliminate misunderstanding.
(v) Avoid double barrelled questions such as the following:
Do you think Malaysians should eat less and do more exercise?
Break up the question into two: one asking about „eating less‰ and the
other asking about „exercise‰.
(vi) Avoid questions involving negatives. Do not confuse respondents
with questions such as the following:
Are you against the ban on smoking?
Do you oppose the ban on public speaking?
SELF-CHECK 6.2
ACTIVITY 6.2
(a) Define the Construct: Define the attitude (or construct) that you wish to
measure. You should review the literature in the field to see how the
attitude has been defined. Examples of attitudes are; attitudes toward racial
integration, attitudes toward women bosses, attitudes toward smoking,
attitudes toward water conservation and so forth.
(b) Break down the Construct: Usually an attitude (or construct) needs to
broken down into a number of categories. In other words, you are
providing an operational definition of the attitude. For example, racial
attitudes can be broken down into: marriage, social interaction, cultural
heritage, workplace and so forth.
(c) Brainstorm: Discuss and come up with a list of about eight to ten statements
for each category of the attitude. Develop an equal number of positive and
negative statements about each category of the attitude object.
(d) Rating the Items: Next is to have a group of judges or panel of experts, rate
each statement on a 5 point rating scale to ensure content validity (or face
validity). Do the statements cover the breadth of the attitude being
assessed? Measure what it is supposed to measure. For example, is the
statement: I would be or have been in a romantic relationship with a person
of another race; describing a racial attitude.
1 = strongly does not describe the attitude
2 = somewhat does not describe the attitude
3 = undecided
4 = somewhat describes the attitude
5 = strongly describes the attitude
(e) Scale: Decide on an appropriate scale such as the following which has
five possible responses: 1 = strongly disagree, 2 = somewhat agree, 3 =
undecided, 4 = somewhat agree, and 5 = strongly agree. If you want to
avoid an „undecided‰ category, you can choose to use an even number of
choices, i.e. a 4 point Likert scale or a 6 point Likert scale.
(g) Reliability: You have to examine the test for reliability which means
whatever the instrument measures, it measures consistently. CronbachÊs
alpha measures how well a set of items measures a single construct (or
attitude). It measures consistency within the instrument where all items are
compared with each other. Alpha coefficient ranges in value from 0 to 1 and
the higher the score, the more reliable is the attitude scale. Nunnaly (1978)
has indicated 0.7 to be an acceptable reliability coefficient but lower
thresholds are sometimes used in the literature.
(h) Reliability Analysis: See Figure 6.2 which is a printout of the reliability
analysis done for a sample of items. The Cronbach alpha for the instrument
is 0.77 (rounded from 0.77102). The second column shows the correlation
between each item with the total or rest of the items. The third column
shows the correlation coefficient obtained if a particular item is deleted.
Note that if you were to delete Item 4, the Cronbach alpha will increase to
0.82 (rounded from 0.816080).
(i) Final Items: Based on the reliability analysis, final items to be included in
the instrument can be decided. Generally, the reliability of the scales tends
to increase with the number of items. However, as the number of items in a
scale increases, so the time taken to complete the attitude test will also
increase, and this may demotivate respondents. There is no hard and fast
rule to determine the final number of items in a scale and this will reflect
the nature and complexity of the attitude being assessed. Generally fewer
than 20 items may reduce reliability acceptably, but more than 30 will begin
to demotivate the respondent.
1 2 3 4
1. Sex education should be the main responsibility of
parents and not teachers.
2. Rather than have students learn about sex from their
peers or on their own, it is better if sex education is
taught in schools.
3. Since some students are likely to experiment with
sex, it is a good idea to have sex education taught in
the public schools.
4. Allowing sex education to be taught in the schools
would lead to an increase in teenage pregnancy.
5. Sex education should not be taught in public schools
because it will lead to students experimenting with
sex earlier than they might, otherwise.
6. The reproductive system taught in science and
biology is not sufficient for students to know about
sex.
ACTIVITY 6.3
Referring to the „Attitude towards Sex Education Scale‰, do the
following:
1. Identify the positive and negative statements.
2. Complete the attitude scale by adding three positive statements and
three negative statements.
3. Try out the complete attitude scale with some of your colleagues
and friends.
This section will discuss quantitative data analysis such as the t-test, ANOVA,
chi-square and correlation coefficient.
(a) Mean
The mean is what is commonly called the average. It is the sum of all the
scores, divided by the number of scores. The mean is the most commonly
used measure of central tendency or the location of the middle or the centre
of a distribution of scores. For example, the mean for the following set of
scores: 5, 3, 6, 4 & 7 = 5.
is steep (as in the case for Group A). When the scores are spread away from
the mean, the standard deviation is large and the bell curve is relatively flat
(as in the case of Group B).
(i) ONE standard deviation (SD = 5) from the mean in either direction on
the horizontal axis accounts for around 68% of the students in this
group. In other words, 68% of students obtained between 15 and
25 marks.
(ii) TWO standard deviations (5 + 5 = 10) away from the mean accounts
for roughly 95% of students. In other words, 95% of students obtained
between 10 and 30 marks.
In the following section we will focus in answering these two questions and
the more commonly used statistical tools. We will talk about these tools
without going into the mathematics involved. You should be aware that we
will discuss the use of these tools at a conceptual level and many issues
(such as assumptions) related to each tool will not be mentioned. (You are
advised to refer to a book on statistics for the details and the mathematics
involved).
H0: There is no significant difference between the experimental group and the
control group in terms of science achievement
Steps:
(a) You can use a statistical tool called the StudentÊs t-test to obtain the t-value
for independent means. („Independent‰ means that the two groups consist
of different subjects). The t-test gives the probability that the difference
between the two means is caused by chance. To test the significance, you
need to set a risk level called the alpha level. In most educational research,
the „rule of thumb‰ is to set the alpha level at .05.
This means that an obtained result that is significant at the .05 level could
occur by chance only 5 times in 100 trials.
(b) Table 6.1 shows that a t-value of 2.65 was obtained. If you are using a
statistical software, the probability level is given (i.e. p < 0.02). You could
also look it up in a table of critical values to find out whether the t-value is
large enough to say that the difference between the groups is not likely to
have been a chance finding.
(c) You determine the degrees of freedom (df) for the test which is the sum of
the persons in both groups minus 2 (i.e. nă2). Given the alpha level, the df,
and the t-value, you can look up the t-value in the table of critical values
(available as an appendix in the back of most statistics texts) to determine
whether the t-value is large enough to be significant.
(d) See Table 6.2. The obtained t-value (2.65) is bigger than the critical value
(2.1009) for 18 degree of freedom (20 ă 2 = 18). From this you can conclude
that the difference between the means for the two groups is significantly
different at the 0.05 level of significance.
df 0.05 0.01
17 2.1098 2.8982
18 2.1009 2.8784
19 2.1009 2.8609
(e) Note that the difference is NOT SIGNIFICANT at the 0.01 level of
significance because the t-value (2.65) is smaller than the critical value
(2.8784) for 18 degrees of freedom.
H0: There is no significant difference between the pretest mean and the
posttest mean in terms of science achievement.
Table 6.3: Means and Standard Deviations Obtained for the Pretest and Posttest Scores
Steps:
(a) Using the t-test for dependent groups (also called paired groups or
correlated groups), you obtain a t-value of 1.94. („Dependent‰ means that
the two means are obtained from the same groups of subjects).The degrees
of freedom (df) for the test is the persons in the one group minus 1 (nă1).
You set the alpha level at 0.05 and with the t-value you look up the table of
critical values
(b) From Table 6.4 you find that for 9 df the critical value is 2.2622 which is
larger than the t-value of 1.94. You conclude that the means is NOT
significantly different at the 0.05 level of significance.
df 0.05 0.01
8 2.3060 3.3554
9 2.2622 3.2498
10 2.2281 3.1693
ACTIVITY 6.4
Control group: (n = 10) 20, 23, 25, 22, 27, 19, 28, 24, 25 & 26.
Experimental group: (n = 10) 29, 30, 28, 32, 33, 37, 27, 36, 31 & 29.
You used the One-way ANOVA and obtained an F-value of 7.1811, which is
significant at 0.003. Hence, the null hypothesis of no differences between means
is rejected. However, you are not sure which differences contribute to the
significance. In other words, whether all three methods are different or whether
only two methods are significantly different.
So, you have to perform another statistical test called Pos Hoc Tests or Pos Hoc
Comparisons. Two common tests used are the Tukey Test and the Scheffe Test
which is usually applied after an analysis of variance. The Tukey Test in Table 6.5
shows that there is no significant difference between the performance of subjects
taught science using Method 1 and Method 2. Subjects taught with Method 3
performed significantly better than subjects taught with Method 2 at the
0.05 level of significance. Also, subjects taught with Method 3 significantly
outperformed subjects taught with Method 1 at the 0.01 level of significance.
The chi-square can be used to test whether or not two or more samples are
different and can be generalised to the population from which the samples were
drawn. The chi-square is widely used in analysing data obtained from surveys
using questionnaires. Say for example, you ask a group of 103 undergraduates
„Is there too much violence on TV?‰ The null hypothesis you want to test is:
The frequencies of their responses are shown in Table 6.6. The degrees of
freedom (df) are obtained by the formula df = (ră1)(că). That is, the df equals the
number of rows in the table minus one multiplied by the number of columns in
the table minus one. So the df for Table 6.6 is 1.
df 0.05 0.01
1 3.841 6.634
2 5.991 9.210
The chi-square (2) obtained is 10.5077 and with 1 degree of freedom, it is more
than the critical value of 3.84 at the 0.05 level of significance and 6.634 at the
0.01 level of significance (see Table 6.7). So, you can reject the null hypothesis
and affirm that male and female undergraduates differ in their opinion about
violence on TV.
ACTIVITY 6.5
Yes No
Final Year Undergraduate 54 83
First Year Undergraduates 75 55
You want to find out whether there is a correlation between self-esteem and
academic performance. A correlation has direction and can be either positive or
negative.
(a) With a positive correlation, individuals who score high (or low) on one
measure (or variable) tend to score similarly on the other variable.
(b) With a negative correlation, individuals who score high on one measure (or
variable) tend to score low on the other (or vice versa).
The Pearson product-moment correlation coefficient (represented by „r‰) is used
to show the degree or strength of the relationship between two variables. The
coefficient can range from r = +1.00 to ă1.00. See Figure 6.3 which illustrates what
the coefficient means.
(a) Figure 6.5(a) shows a perfect positive correlation (r = +1.00) which means
an increase in variable y is also followed by an increase in variable x.
(b) Figure 6.5(b) shows a perfect negative correlation (r = ă1.00) which means
an increase in variable y is followed by a decrease in variable x (or vice
versa).
(c) Figure 6.5(c) shows a zero correlation (r = 0.00) which means there is no
relationship between variable y and variable x.
ACTIVITY 6.6
Enter the following data in the Input data column and press Submit.
What is the correlation between attitude and performance?
Ć Questionnaires are widely used because they are cost effective and can be
administered to large groups of people. If properly designed, questionnaires
can address many issues in a relatively efficient way.
Ć Attitude scales may be designed to measure attitudes using the Likert scale.
Ć The t-test is used to test for significant differences between means for
independent and dependent groups.
Ć One-way ANOVA is used when comparing the means of more than two groups.
Muijs, D. (2004). Doing quantitative research in education with SPSS. London: Sage.
Chapter 7: Bivariate analysis: comparing two groups.
Chapter 10: Using analysis of variance to compare more than two groups.
Reynaldo, J., & Santos, A. (1999). CronbachÊs Alpha: A tool for assessing the
reliability scales. Journal of Extension, 37 (2). Retrieved from:
http://www.joe.org/joe/1999april/tt3.html
StatPac. Inc. (n.d.). Questionnaire design: A free web tutorial. Retrieved from:
http://www.statpac.com/surveys/
INTRODUCTION
The term qualitative research is a general definition that includes many different
methods used in understanding and explaining social phenomena with minimum
interference with the natural environment.
(c) According to Pope and May (1995), qualitative researchers study things in
their natural settings in an effort to discover the meanings seen by those
who are being researched rather than that of the researcher.
Qualitative research begins by accepting that there are many different ways of
understanding and of making sense of the world. You are not attempting to
predict what may happen in the future. You want to understand the people in
that setting. „What are their lives like? What is going on for them? What beliefs
do they hold about the world?‰ In short, qualitative research is concerned with
the social aspects of our world and seeks to find answers to the following
questions:
(a) Why do people behave the way they do?
(b) How are opinions and attitudes formed?
(c) How are people affected by the events that go on around them?
(d) How and why have cultures developed in the way they have?
(e) What are the differences between social groups?
At the same time, the Ê60s was a period of turmoil in the United States and
Europe. Society was going through radical changes, being most concerned about
issues such as racial integration, poverty, womenÊs rights and the cold war (the
threat of Russia). People began to question the use of quantitative methods (such
as experiments and surveys) in explaining social phenomena such as juvenile
delinquency, drug addiction, truancy and so forth).
In education, Carr and Kemmis (1986) differentiated the two types of research
as positivism and phenomenology. According to the positivist approach,
knowledge obtained using the scientific method is objective and measurable.
„Reality‰ according to this perspective is stable, observable and can be measured.
On the other hand, the phenomenological perspective in education focuses on the
processes and experiences one goes through. Literally, phenomenology is the
study of „phenomena‰ or the things as they appear in our experience or the ways
we experience things.
Example:
You are interested in investigating the experiences of a group of adults learning
how to use the computer for the first time. You get them to relate their
experiences and how they feel about touching the keyboard and looking at the
computer screen for the first time in their lives. You also study their experiences
which include their perceptions, their misconceptions, their emotions (feelings),
their desires, their actions and their thoughts. In conducting the study you are
seeking to understand the processes and experiences these adult learners are
going through.
SELF-CHECK 7.1
1. What is positivism?
2. What is phenomenology?
3. Identify FIVE common features of qualitative research in the
definitions given by scholars in the field.
If you are asked why you have chosen the qualitative perspective and your
answer is, „⁄. because no statistics is involved!‰; you do not know in depth the
philosophy and orientation of qualitative research. Your choice of using the
qualitative approach should be based on the basic question, „Is the quantitative
or the qualitative approach appropriate in answering your research question?‰
The decision to conduct research using the qualitative approach should be based
on your orientation as a researcher towards issues such as;
(a) Reality
You must accept the fact that when you use qualitative research methods,
you are interested in „multiple realities‰ or multiple interpretations and not
just one conception of reality or one interpretation.
(c) Knowledge
The knowledge produced from your study will „emerge‰ and you may be
interested in developing a theory rather confirming a theory.
(c) Qualitative research involves field work; that is, the researcher must himself
or herself „walk the factory floor‰ or go into the setting where there are
people, and observe or interview them in their natural setting.
Qualitative Quantitative
Focus: Quality (features) Quantity (how much, numbers)
Philosophy: Phenomenology Positivism
Method: Ethnography/Observation Experiments/Correlation
Goal: Understand, meaning Prediction, test hypothesis
Design: Flexible, emerging Structured, predetermined
Sample: Small, purposeful Large, random, representation
Data collection: Interviews, observation Questionnaire, scales, tests,
documents, artefacts inventories
Analysis: Inductive (by the researcher) Deductive (by statistical methods)
Findings: Comprehensive, description
Precise, numerical
detailed, holistic
Researcher: Immersed Detached
(d) Qualitative research adopts the inductive approach (see Figure 7.1). Such
research is conducted because of a lack of theory or, existing theories are
unable to explain a phenomenon convincingly. Because of this, no
hypotheses are put forward to guide research. The qualitative researcher
begins by observing phenomena and continues to find patterns in the form
of themes, categories, concepts and typologies that emerge. Tentative
hypothesis are introduced and additional information are collected to
explain the phenomenon.
Other than that, qualitative research is also described as emerging. However, this
is not characteristic of all qualitative research. Graduate students doing masters
or doctoral thesis do not have much time at their disposal and so they may not be
able to see the emergence of theories. The sampling in qualitative research is
small and not chosen randomly. Rather, the choice of a sample is purposeful. For
example, if you intend to study drug addicts at a drug rehabilitation centre you
will select a few addicts to study. A long time is required to enable the researcher
to interact with other samples in different situations.
SELF-CHECK 7.2
Ethnography Action
Research
Generic Qualitative
Case Study
Method
7.4 ETHNOGRAPHY
We will discuss the definition, the role of researcher fieldsites, fieldnotes and
some practical aspects of ethnography.
7.4.3 Fieldsites
Traditionally, anthropologists have undertaken ethnographic research in small
bounded villages while living among the inhabitants. These ethnographers may
have been one of few non-natives the villagers may have seen. Today, however,
fieldsites can be nearly anywhere. While research may still focus on villages, an
increasing number of ethnographic studies have been conducted in urban areas.
Sometimes the „group‰ does not live in one location. A fieldsite could be a bank,
a religious centre, a school, a faculty and so forth. Once a potential fieldsite has
been selected, ethnographers must negotiate entry or access.
Say for example you entered a primary school and proceeded to the teacherÊs
lounge, you will certainly be questioned by the school authorities. Obviously,
you will not do this but made an effort to seek permission to gain access.
According to Hammersley and Atkinson (1989), gaining access requires the
(a) WHEN to do your observation which may not be obvious when you enter
the setting. For example, if you are interested in what goes on in the
morning session, than you have to arrive before school begins in the
morning.
(b) HOW OFTEN and for HOW LONG you intend to observe. You may not be
able to determine precisely until you get into the field and find out what
will be needed in order to gain an adequate picture We will discuss this
issue in more detail in Topic 8 under 8.6 Sampling. The usual criterion used
is that it should not be too short (there is insufficient data) or too long (until
it becomes too costly).
(c) WHERE to POSITION yourself. In other words, you will have to find an
appropriate place to stand that is unobtrusive (un-noticeable as possible).
You have to position yourself in such a way so that people around you
feel comfortable with your presence and you can explain to them (when
necessary) that you are not planted by the management or those in
authority. Saying that you are „doing research‰ can be quite comforting
and puts most people at ease about your presence.
7.4.5 Fieldnotes
Deciding what to record and what to leave out when observing people in a
setting can be a difficult task. In any setting there are many things happening and
it is not possible, nor desirable to record everything. Imagine the variety of things
happening in the teacherÊs lounge when it is recess and about 60ă70 teachers
gather in the room! So, you have to be selective and what you observe and record
will depend on the questions you want answered.
To help you focus on what to record, Spradley (1980) suggests the following
checklist:
(a) Space: the physical place or places;
(b) Actors: the people involved;
(c) Activities: a set of related acts people do;
(d) Objects: the physical things which are present;
(e) Acts: single actions people do;
(f) Events: a set of related activities that people carry out;
(g) Time: the sequencing that takes place over time;
(h) Goal: the things people are trying to accomplish; and
(i) Feelings: the emotions felt and expressed.
SELF-CHECK 7.3
1. Define ethnography.
2. List what should be included in fieldnotes.
3. What do you mean by access or entry into the fieldsite?
Boundary
In other words, to qualify as a case study, you have to state the boundary or
delimit what you want to study. For example, you have to limit the number of
people you intend to interview, you have to limit the amount of time you intend
to spend. „If there is no end, actually or theoretically, to the number of people
who could be interviewed or to observations that could be conducted, then the
phenomenon is not bounded enough to qualify as a case (Merriam, 1998, p. 28).
(a) The first relates to the research questions which are most likely to be „how‰
and „why‰questions. For example, „How do students interact in the school
canteen?‰
(b) Second relates to the unit of analysis which could be an individual, a group
of individuals, or an organisation.
(c) Third relates to linking the data collected with the research questions.
(a) Interviews
The interview is an important technique for data collection and there are
two forms of interviews: Closed or Structured Interviews and Open-Ended
Interviews. Open-ended interviews allow subjects to express themselves
more freely and provide insights into events.
(b) Observations
This could be direct observation of events and behaviours as well as
participant-observation where the researcher is an active participant in the
events being studied.
(c) Documents
These could be letters, memos, agendas, administrative documents,
newspaper articles and any other relevant documents. Documents are
useful for making inferences about events. Documents are communications
between persons in the study.
SELF-CHECK 7.4
1. List the characteristics of a case study.
2. When would you use a case study?
In education, action research has also been called classroom research (Hopkins,
1985) and self-reflective inquiry (Kemmis, 1982). It refers to those activities that
are designed to improve the quality of education. It refers to;
(i) Plan: My students find geography lessons boring. How can I make the
lessons more interesting and improve their thinking skills? Bring real-
world situations to the classroom using computer simulations.
(ii) Act: Take students to the computer lab and show them how to use
the simulation software. Select topics that are appropriate for using
computer simulations.
(iii) Observe: Record student interactions and the kind of questions asked.
Videotape a few lessons to see what is happening. Keep notes of my
impressions in a notebook.
(iv) Reflect: The lesson is lively but not all students are asking
questions. They are not discussing with others. They are glued to their
individual computers.
(i) Revise Plan: Break up students into groups of three with each group
working on one computer on a simulation.
(iv) Reflect: Can I continue with this teaching method? Can I use the
Internet instead? I am worried about practical problems such as the
availability of computer labs; will the school purchase more such
computer software? If the Internet is used, will there be broadband
access?
The Generic Qualitative Method does not have a guiding set of philosophic
assumptions in the form of one established qualitative methodology. The
Generic Qualitative Method exhibits some or all of the characteristics of other
methodologies or approaches but makes no claim to any particular qualitative
method. In other words, the Generic Qualitative Method will use the techniques
of ethnography, the case study method, grounded theory and the techniques of
action research, but does not claim it is ethnography, case study, grounded
theory or action research.
(a) To answer specific research questions or begins the study with rather
general or broad questions;
(h) There is no specific length of data gathering but a rationale is given for the
time spent gathering data from observations/interviews;
(l) Fieldnotes are kept of all observations: jottings, description, analysis and
reflection.
The list of possible research techniques that may be adopted in the Generic
Qualitative Method is very much dependent on the objectives of the study and
the researcherÊs beliefs on what constitutes knowledge. What information will be
needed to explain the phenomenon examined? How can others be convinced that
the findings of the study are credible? The following are some examples of
phenomena that may be studied using the Generic Qualitative Method:
(a) Racial integration in the school canteen during recess in a primary school;
(f) Being in an academically weak class for three years: Perceptions and
attitudes of three students;
Ć A paradigm shift will lead to the introduction of new research methods and
tools, and a whole new way of how the researcher sees the world.
Ć Action research is based on the belief that the practitioner is the best judge of
his or her practice; it encourages the researcher to put personal theories into
practice.
Ć The Generic Qualitative Method does not have a guiding set of philosophic
assumptions in the form of one established qualitative methodology, but
instead employs the techniques of data collection of other qualitative
methods.
Massey, A. (n.d.). The way we do things around here: The culture of ethnography.
Retrieved from:
http://www.freeyourvoice.co.uk/htm/waywedo.htm#harg
Tellis, W. (July. 1997). Introduction to case study. The Qualitative Report, 3(2).
Retrieved from: http://www.nova.edu/sss/QR/QR3-2/tellis1.html
INTRODUCTION
In Topic 7 we discussed four main methods of qualitative research commonly
adopted in educational research, namely; ethnography, case study, action research
and the basic qualitative method. Irrespective of the research method adopted,
the techniques of data collection are more or less similar. In this topic, we will
discuss in detail three common data collection or evidence-gathering techniques
employed in qualitative research. For example, the experimental method in
quantitative research uses tests or attitude scales to collect data. Similarly, the
survey method use questionnaires and interview checklists to collect data. So
for ethnographies or case studies, the data collection techniques could involve
observations, interviews or examining documents, or a combination of three
techniques.
For example, you could use content analysis to examine teacher feedback in the
classroom based on phrases such as, „All right‰, „Good‰, „Why do you say
that?‰, „Can anyone give another factor?‰
(a) The first step in content analysis is to establish the objectives of the
investigation. For example, you may be interested in finding evidence for
critical thinking activities in science textbooks or the infusion of values in
economics textbooks or the treatment of world events in history textbooks.
(b) The second step in the content analysis process is to locate data that is
relevant to the objectives. For example, if the objective of your study is to
examine how women are treated in Malaysian novels for teenagers, you
have to locate these novels and decide on the number and variety of novels
to select.
(c) The third step is to establish an empirical link between the data selected
and the inferences to be made from the data (Borg & Borg, 1988). In other
words, you need to provide some theory or model based on reviewing
previous research or citing expert opinion that supports the relationship
between the data and the objectives upon which the study is based. For
example, a review of previous literature may reveal that novels for
teenagers tend to portray women as helpless, „waiting for their prince
charming‰ and so forth.
(d) The fourth step is to sort the content into themes or categories. For example,
if you are examining the minutes of meetings, you might have themes such
as issues, delegation and action. The next step is to devise a coding scheme
which could record frequency (the number of issues discussed and how
often they are discussed), direction (who is the content directed to),
intensity (power of content), and space (size of content). However, not
everything always fits into the themes or categories and there is some
leftover content to be accounted for. This content must be interpreted by a
knowledgeable researcher who knows something about the culture of his
or her subjects. The researcher should be careful about making inferences
about motivation or intent and as far as possible content analysis should
be confined to what is in the text. For example, you cannot use content
analysis to prove that newspapers mislead the public or that certain articles
in newspapers have a certain effect on public attitudes. However, you can
use concepts such as „unconscious bias‰ or „unintended consequences‰
when interpreting the text.
SELF-CHECK 8.1
8.2 OBSERVATION
Imagine that you want to find out what goes on
in the teachersÊ lounge or staff room. You could
interview those involved, or maybe even send
out a questionnaire. Using the interview or the
questionnaire, you would be getting what people
thought about what was going on. Sometimes,
the best way to gain a „rich picture‰ of a setting
such as the staff room, the school canteen, a staff
meeting, the playground or the classroom is to
see for yourself what is happening, rather than
depending on your respondents.
SELF-CHECK 8.2
ACTIVITY 8.1
8.3 INTERVIEWS
Interviewing is a technique for gathering data from humans
by asking them questions and getting them to react verbally.
There are many different ways of conducting interviews (see
Figure 8.1). Structured interviews use an interview schedule
that is similar to the survey questionnaire. You could phrase
the question in such a way that you have a limited range of
responses.
For example, „Do you think the image of teachers in society has gone down?‰
Strongly agree, agree, somewhat agree, disagree and strongly disagree.
Structured interviews are widely used in surveying opinions, beliefs and
perceptions of people. Individual interviews are expensive and you should
consider whether the same amount of data can be more efficiently collected using
written questionnaires.
Qualitative Research
and interviewee to discuss certain topics in more detail. If the interviewee has
difficulty answering a question or hesitates, the interviewer will probe. Three
types of probes commonly used by the interviewer are:
Face-to-face or personal interviews are labour intensive but can be the best way
of collecting high quality data, especially when the subject matter is very
sensitive, if the questions are very complex or if the interview is likely to be
lengthy (Mathers, Fox & Hunn, 2002).
8.4 TRIANGULATION
Triangulation is the process of strengthening the findings obtained from a
qualitative inquiry by cross-checking information. A researcher who argues that
his or her findings are derived from many different kinds of people across many
different situations will be more convincing than another researcher whose
conclusions are based on observations of one person in one setting (Potter, 1996).
Document
Examination
Interview Observation
Figure 8.2: Methods triangulation
SELF-CHECK 8.3
(a) First, as a qualitative researcher, you must have an enormous tolerance for
ambiguity. Tolerance for ambiguity means the ability to tolerate or accept
inconsistencies and uncertainties. In qualitative research, there are no set
procedures or guidelines to be followed step by step. From the design stage
to the data collection and data analysis stages the researcher has to be
prepared to face unforeseen events and change in direction. Merriam (1998)
compares the role of the qualitative researcher to that of a detective who
looks for clues, finds the missing clues and putting the pieces together.
If you are a person who likes structured situations and have no patience
with ambiguity, you should choose quantitative methods which are more
structured and have step by step guidelines.
(c) Third, the qualitative researcher should be able to detect personal biases. It
should be remembered that the primary instrument is the researcher and
being human there is a likelihood that his/her values might creep into the
observations and interviews conducted. The researcher will bring his/her
perceptions or interpretations into the phenomenon being studied. Some
scholars have argued that this is unavoidable and we will have to accept
this as part and parcel of qualitative research. However, if the findings of
any qualitative research are to be accepted by others, there is a need to
ensure that infiltration of the researcherÊs values is minimised.
SELF-CHECK 8.4
too superficial and there is a lack of in-depth explanation, then you may conclude
that more time should be spent in conducting the study. The length of time spent
in collecting can be viewed from two angles:
(a) Span of Time ă how long? 2 weeks, 1 month, 3 months and so forth.
(b) Degree of Contact ă The number of contact hours the researcher spends
with the person or persons.
For example, Researcher A who observes a group of low ability students at the
rate of 1 hour per week for a span of 6 months which comes to a total of 24 hours.
Researcher B who moves in with the family of a low ability student and observes
the learner during all his waking hours. Who spent more time collecting data?
Researcher A took more time gathering data if you are focusing on span of time.
If you are focusing on contact hours, than Researcher B took more time collecting
data. Both conceptions of time (span of time and contact hours) are important. A
long span of time enables you to see broad and overall patterns while long
contact hours allow you to detect micro patterns or details in the environment.
(a) Cox (1980) observed a kindergarten class for two hours a day (though
not every day), between September and June. Thus about 132 hours of
observation took place over a 66 day period.
(b) Bossert (1974) watched two classrooms over a six month period about
3ă4 hours each week. He observed for 40ă60 minutes each time, and
observations were rotated so that all classroom activities could be sampled.
(c) Baker (1985) observed children in the playground twice a day during each
of two recess periods. These observations took 30ă40 minutes each day over
a one year period.
SELF-CHECK 8.5
1. What is the difference between the span of time and the contact
hours in terms of length of time spent in data collection?
2. How should a researcher determine the length of time spent in
collecting data?
8.7 SAMPLING
In quantitative research, sampling is the selection of a group of persons from
a population with each person having an equal chance of being selected. The
objective is to draw a representative sample and the results obtained from the
sample can be generalised to the population. How is the issue of sampling dealt
with in qualitative research? In qualitative research the concern is with the issue
of „access‰. What is meant by access?
(a) First, the researcher might select a person or persons to investigate because
of efficiency or convenience.
(b) Evidence is collected from people who are easily available to support the
researcherÊs arguments. For example:
(i) A teacher might ask his or her students in the class to allow themselves
to be interviewed.
Lincoln and Guba (1985) emphasise that the guiding principle of sampling in
qualitative research is one of convenience. An important consideration in
sampling is whether there are people available who will allow the researcher to
collect data about them. For example, the researcher interested in preschoolers
interacting during recess may call up several kindergartens until he or she finds
kindergartens that will allow him or her to observe children during recess.
If you read research using qualitative methods, you will see a range of positions
on the issue of sampling. Some studies do not give much information about how
the people interviewed or observed were selected. Other studies give some
information about why they selected certain people and how they gained access.
The main issue with sampling is the extent to which readers trust the findings of
the research. If the researcher fails to provide sufficient information about how
he or she collected evidence especially in relation to how and why particular
persons were selected, it would be difficult for the reader to trust the findings.
On the other hand, if the researcher provides a detailed description about the
process of gaining access and selection of the persons interviewed or observed,
readers will be more inclined to trust the findings.
SELF-CHECK 8.6
How do you decide on who and how many subjects to include in your
sample?
"In fact, we have assumed that, if the purpose of the research is to generalise,
one should employ quantitative methodology."
(p. 54)
They are of the opinion that generalisation is not important and is not consistent
with the qualitative paradigm or perspective. If generalisation is the objective, than
quantitative methods should be used and not qualitative methods. However, there
are other researchers who believe that efforts should be made to generalise
findings of qualitative research. These researchers argue that the in-depth
description of a particular phenomenon is sufficient for the researcher to make
generalisations to other individuals or individuals. To enable the findings of
qualitative research to be generalised, researchers have proposed ways in which
validity can he enhanced. Benz and Newman (1998) proposed the following terms
while discussing the issue of generalisation of qualitative research findings:
(a) Applicability
Can the study be applied to another sample? It should be remembered that
there is no „significant difference‰ and it is difficult to generalise to the
population based on the findings of a sample. The deep description of the
characteristics of the subject being studied may allow one to conclude
the extent to which it is comparable to other subjects. If the subjects are
comparable, then one would be more comfortable to make generalisations.
For example, you observe three science teachers. Can you generalise what
you observed to all science teachers in the country? It has been argued that
the in-depth description of subjects or the sample studies enables the
researcher to decide the extent to which it is the same with other subjects or
another sample. The greater the similarity between subjects, the higher is
the possibility of making generalisations.
(c) Replicability
What is the likelihood that a particular product or event will recur given
similar conditions? It is difficult to replicate or repeat a qualitative research
because the natural setting is constantly changing. Unless you have data
showing the changes, you are advised to be cautious when making claims
that a study can be replicated.
SELF-CHECK 8.7
(a) Triangulation
This involves the cross-checking of information from different dimensions.
Data triangulation is when the researcher refers to different sources of data
in understanding a particular phenomenon. Methods triangulation is when
the researcher uses different methods to study a particular phenomenon.
(g) Reflexivity
One of the main issues with the validity of qualitative research is the
likelihood of the researcherÊs values creeping into the interpretation of data.
To minimise this happening you examine yourself critically to detect any
potential bias and inclination that may influence the conclusions you make
about the data. Though no data can be a hundred percent value free and
objective, the researcher should make serious effort to convince others that
the level of objectivity of your study has been maintained. If you had used
other investigators in your study, you should provide evidence that there
was high inter-rate reliability and consistency between investigators.
SELF-CHECK 8.8
Ć If you feel that the research study to is too superficial and there is a lack of
in-depth explanation, than it may be concluded that more time should be
spent in conducting the study.
Ć External validity is defined as the extent to which the findings of a study may
be generalised to another setting or another group of people.
Ć Triangulation, member checking, negative case sampling and audit trail are
some techniques to enhance qualitative research.
INTRODUCTION
The Leadership Behaviour of a Principal as Perceived by
Teachers in a School
The following are probable qualitative data collection methods that
could be used to investigate the above topic:
(a) Field notes from observations of staff meetings.
(b) Focus groups interview transcripts.
(c) Copies of diary entries teachers have been asked to complete
each day.
(d) Researcher memos and reflections.
(e) Audio recordings.
sitting on your desk waiting to be analysed. How do you go about making some
sense of qualitative data? Qualitative data is mostly in the form of words and
narratives, while some may include visual images, audio and video recordings.
Generally, since numbers are not used, the qualitative researcher looks for
categories or themes from the raw data to describe and explain phenomena.
He/she analyses the relationships and patterns between the categories or themes
that have been identified. These categories or themes may be derived using two
approaches:
Analysis
Grounded
theory
Report Coding
Writing Analysis
Framework
analysis
(a) Familiarisation
The first step of data analysis is familiarisation in which you listen to tapes
and watch video material, reading and re-reading the field notes, making
memos and summaries before formal analysis begins. The is especially
important when besides you, others were also involved in data collection.
You have got to be familiar with the field notes they made (perhaps trying
to decipher their handwriting!).
(b) Transcription
Almost all qualitative research studies involve some degree of transcription.
What is transcription? Transcription is the process of converting audio or
video-recorded data obtained from interviews and focus groups as well as
handwritten fieldnotes into verbatim form (i.e. written or printed) for easy
reading. Why do you have to do this? If you were to analyse directly from
an audio or video recording, there is the likelihood that you might include
those sections that seem relevant or interesting to you and ignore others.
With a transcript of everything that you observed and recorded (audio,
video or fieldnotes), you get the whole picture of what happened and the
chances of your analysis being biased is minimised.
You should not forget to include non-verbal cues in the transcript such as silence
(which may indicate embarrassment or emotional distress), pause for thought
(such as „well ... er ... I suppose ...) laughter, gestures (which may add meaning
to the spoken word) and so forth. If someone else is transcribing your material,
make sure to tell him or her how much of this non-verbal information to include.
If you have never transcribed material, it is useful to do a little yourself. Try
doing Activity 9.1.
ACTIVITY 9.1
(c) Organisation
After transcription, it is necessary to organise your data into sections that
are easy to retrieve. What does this mean? Say for example, in your study
you interviewed 10 teachers (30 minutes each) on their opinion about the
leadership style of their principal. It is advisable that you give each teacher
a pseudonym (e.g. Elvis, Jagger, Dina ⁄ not their real names) or referred to
by a code number (e.g. T1, T2 ... T10). You need to keep a file that links the
pseudonyms or code numbers to the original informants which have to be
kept confidential and destroyed after completion of the research. Names
and other identifiable material should be removed from the transcripts.
The narrative data you obtained from the 10 teachers needs to be numbered
depending on your unit of analysis. In other words, you have to determine
whether you intend to analyse at the word level, sentence level or paragraph
level and they have to be numbered accordingly. Make sure that the unit of
text you use can be traced back to its original context.
(i) If you are interested in conducting an exploratory study and are more
concerned with theory generation, than the grounded theory approach
should be your choice of analysis.
SELF-CHECK 9.1
(d) Coding
Coding is the process of examining the raw qualitative data in the
transcripts and extracting sections of text units (words, phrases, sentences
or paragraphs) and assigning different codes. This is done by marking
sections of the transcript and giving a numerical reference, symbol,
descriptive words or category words. Most of the text (or transcript) will be
marked and given different codes which will be later refined or combined
to form themes or categories. To help you with the practicalities of coding,
you could:
(i) Cut and Paste ă you can literally cut your transcripts into smaller
units of analysis which could be individual words, phrases, sentences
or paragraphs. You could paste these text units on to cards which you
could sort and re-sort easily. Keep in mind that each text unit needs to
be traceable to its original context. Sometimes, a text unit may have to
be sorted into two different categories or themes. So you will need
to make several copies of a text unit to be sorted into two or more
categories.
(ii) Colour code ă you could also use highlighting pens to highlight text
units or coloured pens to underline units of text. There could be a
problem when there are hundreds of text units and you will need
hundreds of colours which could pose a problem differentiating the
colours. The advantage of using coloured pens or highlighters is that
you do not need to cut up the transcripts. Colour coding would be the
choice if you do not have too many categories or text units.
Grounded theory evolved from the work of sociologists Glaser and Strauss
(1967). Grounded theory is a method and approach in doing qualitative
research. It is an inductive method of qualitative research in which theory is
systematically generated from data. However, many studies in education,
business, management and in the health field (especially in nursing), have
adopted grounded theory as a procedure for conceptualising and analysing
data without taking on the whole methodology. The appeal of grounded
theory analysis is that it allows for the theory to „emerge‰ from the data
through a process of rigorous analysis (see Figure 9.1). The word „theory‰
is used to mean the relationships that exist among concepts that come from
the data and help us understand our social world more clearly (Strauss and
Corbin, 1998).
The main feature of the grounded theory procedure is the use of the
constant comparison technique. Using this technique, categories or concepts
that emerge from one stage of analysis are compared with categories
or concepts that emerge from the previous stage (see Figure 9.2). The
researcher continues with this technique until what is called „theoretical
saturation‰ is reached or no new significant categories or concepts emerge.
The grounded theory procedure is cyclical involving frequent revisiting of
data in the light of emergence of new categories or concepts as data analysis
progresses. The theory that develops is best seen as provisional until
proven by the data and validation from others.
Strauss and Corbin (1998) suggest what is called open coding. Open coding
is where you „sweep‰ through the data, marking the text. It is a good idea
to leave a column at the side of your data so you can write your codes next
to the segments you are coding. LetÊs assume you are interested in how a
group of teachers view the behaviour of their principal in staff meetings.
Refer to an extract of an interview with a teacher and the key phrases
extracted as show in the right margin.
Extraction of key
phrases
R: How long have you been a teacher in this school?
T: For about 10 years.
R: Your principle, how would you describe him?
T: Quite a hot-tempered guy. hot-tempered
R. What do you mean hot-tempered?
T: Well, in the last staff meeting, I objected to his idea of cutting down the number
of fieldtrips for students. He argued that that it was too much of a responsibility
for the school. Also, it was getting more and more expensive for the school. lost his
R: What happened than? cool
T: Before I could say anything, he lost his cool and came for me. refused to
He refused to listen to what I had to say….he just went on and on. listen
R: What do you think? just went on
T: Personally, I think it was not fair of him to scold me. After all this is a democracy and on
and he should at least listen to what I had to say. It was very unpleasant and
many of my colleagues were very disturbed over the incident.
R: How do the others feel? not fair
T. Many of us prefer to keep quite and suffer in silence. You know, he is quite scold
close with the higher-ups. Anyone who questions his decisions are ridiculed
You know he determines whether we get promoted or not. You know, it’s the
usual thing!
R: How often does this happen? ridiculed
T: Almost always…..all meetings becomes a one man show …it’s all …talk….talk. for
You have uncovered eight descriptions of behaviour and the questioning
ll i d
Copyright © Open University Malaysia (OUM)
178 TOPIC 9 QUALITATIVE DATA ANALYSIS
Next you may want to recode the eight descriptions into one or two
categories or themes. In other words, the category or theme emerges from
the data. You may have to assign a name for the category or theme. In this
example, B3 and B8 could be recoded to A1 and assigned the category or
theme „self-centred‰. You go on doing this until you have exhausted the
data in terms of developing any new codes.
Using the headings, you can create charts of your data so that you can
easily read across the whole dataset. Charts can be either thematic for each
theme or category across all respondents (cases) or by case for each
respondent across all themes:
In the chart boxes, you could put line and page references to relevant
passages in the interview transcript. You might also want to include some
text; e.g. key words or quotations as a reminder of what is being referred to
(see the charts above). For example, under the theme Psychological Causes,
Case 2 talked about „stress in the workplace‰ while Case 3 talked about
„business failure‰.
SELF-CHECK 9.2
What is the difference between the grounded theory approach and the
framework analysis approach in the analysis of qualitative data?
(i) Begin with something interesting, e.g. a quote or story, to capture the
readerÊs interest.
(iii) Tell why thereÊs a need for the study. Cite relevant literature that
calls for the need for the research in this area, or demonstrate the
lack of attention to the topic. In your own words, describe how you
think this study will be useful.
(iv) Describe the intended audience for your research (e.g., the public,
family therapists).
(ii) Cite the major authors who have described your research method.
(iii) Explain how you selected your subjects and gained entry into the
research context (if relevant).
(iv) Describe the procedures you took to protect the rights of your
subjects (e.g. informed consent, human subjectÊs approval,
debriefing, etc).
(v) Describe the kind of relationship you had with the subjects. Will you
be neutral, collaborative or objective?
(vi) Describe the kind of data you collected (e.g. field notes from
memory, audio tapes, video tapes, transcripts of conversations,
examination of existing documents, etc).
(viii) Describe the procedures you used to keep track of the research
process. E.g. your audit trail.
(xi) Summarise and reference all of the relevant literature that you have
reviewed.
(xii) Describe how you reviewed the literature and how it has influenced
the way you approached the research.
(xiii) Discuss how your previous experience with your topic has
influenced the way you have conceptualised this research.
An adequate research report not only explains but also persuades. Being
persuasive is very much an issue of good clear writing. The way you write should
help readers to „see for themselves‰ what you claim to find in and make of the
data. The evidence is the data you collected and from which you choose carefully
an excerpt or excerpts to illustrate „points‰ in your report. It must be the right and
sufficient data to illustrate clearly and logically what is being claimed. Also, the
relevant evidence must be presented within a description that displays in narrative
form „the point‰ being made. Successful qualitative analysis tells a good,
absorbing, and understandable story. The story makes sense because you have
made an effort to make it so and you have communicated this to your reader.
Copyright © Open University Malaysia (OUM)
182 TOPIC 9 QUALITATIVE DATA ANALYSIS
Ć Familiarisation is when you listen to tapes and watch video material, reading
and re-reading the field notes, making memos and summaries before formal
analysis begins.
Ć After transcription, it is necessary to organise your data into sections that are
easy to retrieve.
Ć Coding is the process of examining the raw qualitative data in the transcripts
and extracting sections of text units (words, phrases, sentences or paragraphs)
and assigning different codes to them.
Ć The framework analysis approach allows the researcher to set the categories
and themes from the beginning of the study.
1. When would you use the grounded theory approach instead of the
framework analysis approach when analysing qualitative data?
2. What are some of the elements you would include when writing a
qualitative research report?
3. Conduct a 20 minute observation of a classroom (primary or
secondary) and jot down in a notebook whatever you see. Analyse
the data using either the grounded theory approach or the
framework analysis approach.
Ratcliff, D. (n. d.). 15 methods of data analysis in qualitative research. Retrieve from:
http://www.vanguard.edu/uploadedFiles/faculty/dratcliff/qualresource
s/15methods.pdf
INTRODUCTION
From Topic 1 to Topic 9, we discussed the methods
and techniques of conducting research, both from
the quantitative and qualitative perspective. This
topic will focus on the writing process and will
address two important tasks that most graduate
students will have to do. First, the writing of the
research proposal and second the writing of the
research report.
Chapter 1 Introduction
1.9 Background to the study
1.10 Problem statement
1.11 Objectives of the study
1.12 Research questions
1.13 Research hypotheses (if any)
1.14 Significance of the study
1.15 Limitations of the study
1.16 Definitions of terms
Chapter 2 Review of Literature
2.1 Previous studies
2.2 Theoretical framework
2.3 Methodological issues
Chapter 3 Methodology
3.1 Sample
3.4 Instrumentation
3.5 Data collection procedures
3.6 Proposed framework for data
analysis
References
Appendices
Note: The headings in each chapter may vary in
different studies.
(a) Introduction
(iii) Remember, the problem statement may only be tentative at this stage
as the research has not been carried out yet. It is not expected in a
proposal that you have an answer to your research question. It helps
if you have a tentative answer, however. A hypothesis is useful for
this purpose, though this might only be necessary for more empirical
subjects.
(iv) You should use simple and jargon-free language. The introduction
must actually narrow down; not get wider. You must demonstrate
that you understand well the issues in the area and that you are
focussing on a particular issue.
(i) Briefly tell the reader the significance of the study (justify doing the
study).
(ii) You can argue the significance of your study based on the following
criteria:
You are studying certain variables that have not been given
attention in previous studies.
(i) This is where you provide more details on what others have done in
the area, and what you propose to do. Refer to Topic 2: Theory and
Review of Literature for more details
The gaps in the literature (in more detail than that provided in the
introduction).
Definition of key terms (this can be done when you introduce each
idea, or in a definition sub-section).
Questions arising from the gaps that can be the focus of data
collection or analysis.
(iii) The theoretical framework usually forms the final part of the Literature
Review section. It describes the model that you are using in the thesis
to demonstrate your point.
(iv) Read a thesis in a similar area to get a feel for what is required in this
section.
(i) You do not have to describe the methodology used in great detail (this
will be done in the thesis) but you should justify its use over other
similar methodologies.
Why you have chosen a sampling frame and the size of a certain
sample.
(i) Provide a project timetable specifying how long you will take to
complete the project paper, thesis or dissertation. For example,
indicate how long you will take to collect data, analyse the data and
write up the final report.
(ii) Use the citation style proposed by the Manual of the American
Psychological Association (APA style for short).
Supporting Materials
References
Note: The headings in each chapter may vary in different studies.
2008
(ii) Acknowledgement
An acknowledgment page is included if you have received unusual
assistance in the conduct of the study. The acknowledgement should
be simple and restrained. Do not indulge in flattery and excessive
recognition for routine participation of family members, lecturers,
supervisors, librarians and clerical helpers.
Note: All pages in the preliminary section are numbered at the centre of the
bottom margin with lower-case Roman numerals (i, ii, iii, iv).
Ć Sampling: It is here that you explain the size of the samples and
how you selected them. Indicate the extent to which the sample is
representative of the population. Did you use random sampling?
Did you use stratified sampling?
You should keep in mind that this topic is the most used part of the
research report by other readers. Readers who scan research literature
to find significant studies examine this topic before deciding whether
or not further examination of the report is worthwhile.
(i) You had cited an author in the report but it is not listed in the
References and vice-versa.
(i) Tables and data ă important, but not essential to the understanding of
the report.
(ii) Copies of cover letters used, and printed forms of the questionnaire,
tests and other dataăgathering devices.
OR
Thank you.