Page 1 of 17

RESEARCH METHODOLOGY Formatted: Font: (Default) +Body (Calibri), 12 pt

Formatted: Bottom: 3.81 cm, Height: 35.56 cm
A. Research Design
B. Sampling Method
C. Data Gathering Tool
D. Data Gathering Procedure
E. Methods of Analysis
Formatted: Font: (Default) +Body (Calibri), 12 pt
RESEARCH DESIGNS: Formatted: Tab stops: 13.7 cm, Left

- research experts have different views about the definition of research design
- appears to be the schema that maps out the following:
 sources of data
 type of data to be collected
 how the data will be collected
 methods to be used in the data analysis
NOTE:
“A good research design must also set time constraints within which the research problem
should be answered.”

TYPES OF RESEARCH DESIGN

Descriptive Research
- illustrate / determine patterns or characteristics of variable
OBJECTIVES:
describe, analyze, explain, define, illustrate
Formatted: Font: (Default) +Body (Calibri)
Exploratory Research Formatted: Font: (Default) +Body (Calibri), 12 pt
- gather initial data patterns or characteristics of variables
- studies have not been done or minimal data are available
- hypothesis is not usually present, recommend possible hypothesis which can be
tested in future studies
OBJECTIVES:
explore, find out, know, examine, investigate

Causal/Explanatory Research
- occurrence of or change in the independent variable leads to changes in the
independent variable
OBJECTIVES:
analyze, explain, identify

Correlational Research
- patterns of relationship without suggesting that one variable causes the other
variable to change
- positive or negative correlation
OBJECTIVES:
correlate, interrelate

Comparative Research
- confirm if two or more variables reveal similar or different patterns of
characteristics when compared
OBJECTIVES:
compare, determine similarities,
Find out differences, contrast
 Page 2 of 17

Evaluative Research
- assess performance outcome or impact of a set of variables on another one
OBJECTIVES:
evaluate, assess, appraise, measure,
Quantify

Experimental Research
- cause-and-effect relationship through experimental and control groups
OBJECTIVES:
determine the effects, examine the differences, compare the results, analyze the influence

Action Research
- conducted to develop or recommend fresh approaches, policies, programs,
strategies or skills to solve problems in an actual organizational setting
OBJECTIVES:
develop, propose, suggest, recommend, determine alternative approaches, identify
solutions, formulate policies

SAMPLING PROCESS

1. Define the population from which the sample is to be drawn.

2. Specify the population frame from which the sample will be taken
3. Choose the sampling method of selecting samples.
4. Determine the sample size requirement for the study.
5. Select the actual samples.

DATA GATHERING TOOLS

- face to face interview
- telephone / text messages
- computerized assisted
- questionnaire (personally administered) (mail)
(electronically administered)
- observations of individuals
- laboratories

PRIMARY DATA
- individuals, focus groups, panel of respondents

SECONDARY DATA
- records, publications, industry analysis (ex, layout of the plant), maps, etc.

NOTE: “The choice depends on the facilities available, accuracy required, time, cost and
resources.”
 Page 3 of 17

TYPES OF SAMPLING DESIGNS:

There are two major types of sampling designs: probability and nonprobability sampling.
Each of these two major designs has different sampling strategies. Depending on the extent of
generalizability desired, the availability of time and other research, and the purpose of which the
study is done.

1. Probability Sampling
the elements in the population have some known chance or probability of being
selected as sample subjects

for purposes of wider generalizability

2. Non-Probability Sampling
elements do not have a known predetermined chance of being selected as
subjects

when time and other factors rather than generalizability become critical

PROBABILITY SAMPLING

1. Simple Random Sampling

every element in the population has a known and equal chance of being selected as a
subject.

2. Systematic Sampling
involves drawing every nth element in the population starting with a randomly chosen
element between 1 and n.

3. Stratified Random Sampling

involves a process of stratification or segregation, followed by random selection of
subjects from each stratum. In stratified random sampling the population is first divided
into mutually exclusive groups that are relevant, appropriate, and meaningful in the
context of the study.

4. Proportionate / Disproportionate
once the stratum has been stratified in some meaningful way, a sample of members from
each stratum can be drawn using either a simple random sampling or a systematic
sampling procedure. The subjects drawn from each stratum can be either proportionate
or disproportionate to the number of elements in the stratum. This sampling design
because, for the same sample size, we get more representatives from each important
segment of the population and obtain more valuable and differentiated information with
respect to each group.

5. Cluster Sampling
groups of chunks of elements that, ideally, would have heterogeneity among the
members within each group are chosen for study in cluster sampling. When several
groups with intragroup heterogeneity and intergroup homogeneity are found, then
random sampling of the clusters or groups can ideally be done with information gathered
from each of the clusters or groups can ideally be done with information gathered from
each of the members in the randomly chosen clusters.

6. Area Sampling
 Page 4 of 17

when the research pertains to populations within identifiable geographical areas such as
counties, city blocks or particular boundaries within a locality, area sampling can be done.

7. Double Sampling
when a sample is used in a study to collect some preliminary information of interest, and
later a subsample of this primary sample is used to examine the matter in more detail

8. Multi Stage Sampling

involves a probability sampling of the primary sampling units, from each of these primary
units, a probability sample of the secondary sampling units is then drawn; in turn, a third
level of probability sampling is done from each of these secondary units, and so on, until
we have reached the final stage of breakdown for the sample units when we will sample
every member in those units.

NON PROBABILITY SAMPLING

The elements in the population do not have any probabilities attached to their being
chooses as sample subjects.

1. Convenience Sampling
as its name implies, convenience sampling involves collecting information from members
of the population who are conveniently available to provide this information

2. Purposive Sampling
instead of obtaining information from those who are most conveniently available, it might
sometimes become necessary to obtain information from specific targets - that is, specific
types of people who will be able to provide the desired information, either because they
are the only ones who can give the needed information or because they conform to some
criteria set by the researcher.

3. Judgment Sampling
involves the choice of subjects who are in the best position to provide the information
required

4. Quota Sampling
form of proportionate sampling in which a predetermined proportion of people are
sampled from different groups, but on a convenience basis

ISSUES OF PRECISION AND CONFIDENCE

 Page 5 of 17

IN DETERMINING SAMPLE SIZE

PRECISION denotes how close we are in estimating the population parameters based on the
sample statistic.

CONFIDENCE denotes how certain we are that our estimates will really hold true for the
population

NOTE:
A 95 percent confidence is the accepted level for most business research.

DETERMINING THE SAMPLE SIZE

1. Sample sizes larger than 30 and less than 500 are appropriate for most research.

2. Where samples are to be broken into subsamples, a minimum sample size of 30 for each
category is necessary.

3. In multivariate research, the sample size should be several times, (preferably 10 times or
more) as large as the number of variables in the study.

4. For simple experimental research with tight experimental controls, successful research is
possible with samples as small as 10 to 30 in size.

NOTE:
Sampling decisions should consider both the sampling design and the sample size.

N
n = --------------------------
1 + Ne2
 Page 6 of 17

Table for Determining Sample Size

N S N S N S
10 10 220 140 1200 291
15 14 230 144 1300 297
20 19 240 148 1400 302
25 24 250 152 1500 306
30 28 260 155 1600 310
35 32 270 159 1700 313
40 36 280 162 1800 317
45 40 290 165 1900 320
50 44 300 169 2000 322
55 48 320 175 2200 327
60 52 340 181 2400 331
65 56 360 186 2600 335
70 59 380 191 2800 338
75 63 400 196 3000 341
80 66 420 201 3500 346
85 70 440 205 4000 351
90 73 460 210 4500 354
95 76 480 214 5000 357
100 80 500 217 6000 361
110 86 550 226 7000 364
120 92 600 234 8000 367
130 97 650 242 9000 368
140 103 700 248 10000 370
150 108 750 254 15000 375
160 113 800 260 20000 377
170 118 850 265 30000 379
180 123 900 269 40000 380
190 127 950 274 50000 381
200 132 1000 278 75000 382
210 136 1100 285 1000000 384
 Page 7 of 17

Formula recommended by the

Philippine Social Science Council Survey

N Z/22 p (1 – p)
n = --------------------------
N E2 + Z/22 p (1 – p)
where:
at 95% confidence level Z/2 = 1.96
at 90% confidence level Z/2 = 1.65
at 99% confidence level Z/2 = 2.58
assume proportion p = 0.5 (as preliminary estimate)
sampling error E normally assumes the values: 0.025; 0.05; 0.10

SAMPLE COMPUTATION:

1000 * 1.962 * 0.5 * (1–0.5)

n = -----------------------------------------
1000 * 0.052 + 1.962 * 0.5 * (1–0.5)

960.4
n = ---------------
2.5 + 0.9604

= 277.54  278
 Page 8 of 17

Types of Research Design Formatted: Font: (Default) +Body (Calibri), 12 pt

Quantitative Research Designs Formatted: Font: (Default) +Body (Calibri)

Descriptive  Describe phenomena as they exist. Descriptive studies Formatted: Font: (Default) +Body (Calibri)
generally take raw data and summarize it in a useable form.
 Can also be qualitative in nature if the sample size is small and
data are collected from questionnaires, interviews or
observations.

Experimental  The art of planning and implementing an experiment in which Formatted: Font: (Default) +Body (Calibri)
the research has control over some of the conditions where
the study takes place and control over some aspects of the
independent variable(s) (presumed cause or variable used to
predict another variable)

Quasi-experimental  A form of experimental research. One in which the researcher Formatted: Font: (Default) +Body (Calibri)
cannot control at least one of the three elements of an
experimental design:

 Environment
 Intervention (program or practice)
 Assignment to experimental and control groups

Qualitative Research Designs Formatted: Font: (Default) +Body (Calibri)

Historical  Collection and evaluation of data related to past events that Formatted: Font: (Default) +Body (Calibri)
are used to describe causes, effects and trends that may
explain present or future events. Data are often archival.
 Data includes interviews.

Ethnographic  The collection of extensive narrative data over an extended Formatted: Font: (Default) +Body (Calibri)
period of time in natural settings to gain insights about other
types of research.
 Data are collected through observations at particular points of
time over a sustained period.
 Data include observations, records and interpretations of what
is seen.

Case Studies  An in-depth study of an individual group, institution, Formatted: Font: (Default) +Body (Calibri)
organization or program.
 Data include interviews, field notes of observations, archival
data and biographical data.

Source: Formatted: Font: (Default) +Body (Calibri)

The materials offered were developed by The National Clearinghouse for Comprehensive
School Reform, staff from Regional Educational Laboratories, and Comprehensive School
Reform Coordinators from select state education agencies.
The content was presented at the former Region VII Comprehensive Center's Schoolwide
Institute by Belinda Biscoe, Ph.D., Director, April 2003.
 Page 9 of 17

Historical Research Design - The purpose is to collect, verify, synthesize evidence to establish
facts that defend or refute your hypothesis. It uses primary sources, secondary sources, and
lots of qualitative data sources such as logs, diaries, official records, reports, etc. The limitation
is that the sources must be both authentic and valid.

Case and Field Research Design - Also called ethnographic research, it uses direct observation
to give a complete snapshot of a case that is being studied. It is useful when not much is known
about a phenomenon. Uses few subjects.

Descriptive or Survey Research Design - It attempts to describe and explain conditions of the
present by using many subjects and questionnaires to fully describe a phenomenon. Survey
research design /survey methodology is one of the most popular for dissertation research.
There are many advantages.

Correlational or Prospective Research Design - It attempts to explore relationships to make

predictions. It uses one set of subjects with two or more variables for each.

Causal Comparative or Ex Post Facto Research Design - This research design attempts to
explore cause and affect relationships where causes already exist and cannot be manipulated. It
uses what already exists and looks backward to explain why.

Developmental or Time Series Research Design - Data are collected at certain points in time
going forward. There is an emphasis on time patterns and longitudinal growth or change.

Experimental Research Design - This design is most appropriate in controlled settings such as
laboratories. The design assumes random assignment of subjects and random assignment to
groups (E and C). It attempts to explore cause and affect relationships where causes can be
manipulated to produce different kinds of effects. Because of the requirement of random
assignment, this design can be difficult to execute in the real world (non laboratory) setting.

Quasi Experimental Research Design - This research design approximates the experimental
design but does not have a control group. There is more error possible in the results.

EXPERIMENTAL RESEARCH METHODS

The first method is the straightforward experiment, involving the standard practice of manipulating
quantitative, independent variables to generate statistically analyzable data.

Generally, the system of scientific measurements is interval or ratio based. When we talk about
‘scientific research methods’, this is what most people immediately think of, because it passes all of
the definitions of ‘true science’. The researcher is accepting or refuting the null hypothesis.

The results generated are analyzable and are used to test hypotheses, with statistics giving a clear
and unambiguous picture.

This research method is one of the most difficult, requiring rigorous design and a great deal of
expense, especially for larger experiments. The other problem, where real life organisms are used, is
that taking something out of its natural environment can seriously affect its behavior.

It is often argued that, in some fields of research, experimental research is ‘too’ accurate. It is also
the biggest drain on time and resources, and is often impossible to perform for some fields, because
of ethical considerations.

The Tuskegee Syphilis Study was a prime example of experimental research that was fixated on
results, and failed to take into account moral considerations.
 Page 10 of 17

In other fields of study, which do not always have the luxury of definable and quantifiable variables -
you need to use different research methods. These should attempt to fit all of the definitions of
repeatability or falsifiability, although this is not always feasible.

OPINION BASED RESEARCH METHODS

Opinion based research methods generally involve designing an experiment and collecting quantitative
data. For this type of research, the measurements are usually arbitrary, following the ordinal or
interval type.

Questionnaires are an effective way of quantifying data from a sample group, and testing emotions or
preferences. This method is very cheap and easy, where budget is a problem, and gives an element of
scale to opinion and emotion. These figures are arbitrary, but at least give a directional method of
measuring intensity.

Quantifying behavior is another way of performing this research, with researchers often applying a
‘numerical scale’ to the type, or intensity, of behavior. The Bandura Bobo Doll experiment and the
Asch Experiment were examples of opinion based research.

By definition, this experiment method must be used where emotions or behaviors are measured, as
there is no other way of defining the variables.

Whilst not as robust as experimental research, the methods can be replicated and the results falsified.

OBSERVATIONAL RESEARCH METHODS

Observational research is a group of different research methods where the research try to observe a
phenomenon without interfering too much.

Observational research methods, such as the case study, are probably the furthest removed from the
established scientific method. This type is looked down upon, by many scientists, as ‘quasi-
experimental’ research, although this is usually an unfair criticism. Observational research tends to
use nominal or ordinal scales of measurement.

Observational research often has no clearly defined research problem, and questions may arise during
the course of the study. For example, a researcher may notice unusual behavior and ask, ‘What is
happening?’ or ‘Why?’

Observation is heavily used in social sciences, behavioral studies and anthropology, as a way of
studying a group without affecting their behavior. Whilst the experiment cannot be replicated or
falsified, it still offers unique insights, and will advance human knowledge.

Case studies are often used as a pre-cursor to more rigorous methods, and avoid the problem of the
experiment environment affecting the behavior of an organism. Observational research methods are
useful when ethics are a problem.

Questionnaires
These are sets of questions and attitudinal statements administered to individuals for self-
completion. They can be posted to respondents, sent by email or filled in by the researcher
during face-to-face enquiries. They can be:

 wholly closed-ended, with every question having a fixed range of alternative

responses, or
 open-ended, with very broad questions designed to elicit the sample's own views
rather than their responses to a re-specified range of answers, or
 a mixture of the two.

Potential advantages
 Page 11 of 17

1. Questionnaires are a useful means of getting data from a relatively large number of
people or from a representative sample of that population.
2. Therefore they are very efficient in terms of the researcher's use of time and effort.
That is, you can distribute hundreds of them to students, colleagues or clients and
get them completed and returned in the time it takes to set up and do half a dozen
interviews. Questionnaires are also, usually, quicker to code and analyse that semi-
structured or unstructured interviews.
3. Questionnaires are a means of standardising the data collection process, i.e.
everyone who completes it is being asked exactly the same questions in the same
sequence [but of course this does not mean that they will choose to answer every
question].
4. The self-completion questionnaire may provide the respondent with time to reflect on
the questions before answering them [especially if they can complete them in their
own time].
5. Respondents may feel that they can say what they really think if the questionnaire
can be completed in privacy and anonymously [especially if the researcher is known
to them or might be thought to have a vested interest in their answers].

Potential disadvantages

1. There are hidden costs in the use of questionnaires. It may be a quicker method of
collecting data and the format may facilitate data analysed but the design of a good
questionnaire with clear instructions and unambiguous questions can take a long
time. Semi-structured and unstructured interview schedules can be developed more
quickly because the researcher gets immediate feedback from the respondents if
they do not understand the questions.
2. You may not always know that your carefully constructed questionnaire is not asking
the 'right' questions until you start analysing the data, i.e. when it is too late to do
anything about it. If they have sufficient time researchers try to get round this
problem by piloting the questionnaire with a small group drawn from the population
they are interested in before they make the final adjustments to it. But this is time
consuming and practitioners doing research on a part-time basis do not always feel
that they can spare the time to do this. Pilot your questionnaire if you can. At the
very least send it to some colleagues and your tutor for comment.
3. Once you have sent out the questionnaire you have little control over the situation.
Suppose only a few completed ones are returned, i.e. what researchers refer to as a
low response rate. What can you do? You can try several things to increase the
likelihood of a good response rate:
i. Get a 'captive' population or sample to complete it, i.e. ask students to complete it
during a seminar or clients to complete it in your presence;
ii. Explain at the beginning of the questionnaire (or in an accompanying letter if
it is a postal questionnaire) why their responses are important and what use
will be made of the data.
iii. Make the questionnaire as user friendly and attractive as possible by using
different font sizes, colour printing, attractive layout and provide as many
opportunities as possible to tick boxes or ring code numbers so that it can be
filled in as quickly as possible.
iv. Distribute reminders perhaps with a second copy of the questionnaire.
Avoid a lot of open-ended questions. As we have seen, one of the advantages of a
self-completion questionnaire is that it pre-codes people's responses into a standardised set
of answers to facilitate data analysis. Why lose that advantage? If you feel the need to have
a lot of open-ended questions perhaps you have opted for the wrong method.

Interviews
A face-to-face conversation (or by telephone or video conference) which is structured
around a set of questions, topics or themes. The degree to which it is structured can vary
greatly. It can be highly structured (and then usually it looks like a questionnaire or checklist
but administered by the researcher rather than self-completed). It can be semi-structured,
mainly comprising a set of open-ended questions but often with follow-up probes and
 Page 12 of 17

prompts; or it can be relatively unstructured - a list of themes or topics or headings which

can be adjusted to each individual interviewee.

Potential advantages

1. Provides an opportunity for the interviewee to give a more detailed response than in
a questionnaire.
2. The data will usually be richer and more full of contextual information than the data
provided by a questionnaire.
3. The data will be collected in a natural setting. For example, interviewing a lecturer
immediately after a seminar will probably generate more data than a questionnaire
which could be completed at any time. There is scope for the interviewer to cross-
reference to the seminar, particularly if he or she also observed it [an example of
triangulation at work].
4. An interview is a particularly useful tool if you are trying to understand the
experiences and actions of each individual respondent. The more unique each
respondent's experiences are the more important it is to use a research technique
which acknowledges and elicits that uniqueness.
5. An interview is also a useful method if the respondents are not very literate or not
used to expressing their views on paper.
6. It provides an opportunity to probe respondents' views in ways that might be difficult
to plan for in advance.
7. It provides the researcher with an opportunity to also observe and record the non-
verbal behaviour of the respondent as well.

Potential disadvantages

1. The interview is not necessarily a cost effective method for a researcher who does
not have any help in collecting data from a relatively large sample or population of
respondents.
2. The full transcription of interviews takes a lot of time. A one-hour tape recorded
interview takes about 8-10 hours to transcribe.
3. If you want to it is possible to quantify some of the answers (e.g. the number of
respondents who answered this question positively or negatively) but other methods
might be more appropriate for this purpose.
4. Good interviewing requires expertise and experience. It is not an easy option for the
inexperienced researcher. It requires, for example, good listening skills; body
language that encourages the interviewee to relax and talk; a capacity to ask
questions, perhaps take notes and yet maintain eye contact; an ability to prompt
people who are not very responsive; knowing just how long to allow a silence to
continue before intervening; an ability to probe sensitive areas and issues; being
able to 'think on your feet' and be flexible in your questioning e.g. to recognise that
someone has just answered 3 of your questions in one response and therefore being
able to instantly adjust your schedule of questions accordingly to being able to
decide on-the-spot whether an interviewee is leading you down a blind alley and
needs to be brought back to the sequence of questions on your schedule or is
raising interesting issues and should be encouraged to continue. In other words,
interviewing is not a soft option. It requires planning (including planning for
unexpected contingencies) and it requires skill.
5. There is always a high risk with interviews that the researcher will unduly influence
the responses of the interviewee: through the way the questions are asked, the
researcher's body language and manner, the interviewer's position of authority in the
interviewee's institution or vice versa (a potential problem in practitioner research).

Administrative issues

1. Once you have explained what the interview is all about and why you are
interviewing the respondent try one or two 'warm-up' questions to establish a good
rapport.
2. Listen, listen, listen! You can lose rapport if you ask someone a question which they
have already answered when replying to an earlier question. Also you may want to
 Page 13 of 17

cross-reference the answer to one question again the answers given to earlier
questions, e.g. 'Earlier in the interview you said that...'
3. Do not be afraid to leave out a question if you think it is not relevant to a particular
interviewee.
4. If your interviewee is talkative and giving you a lot of useful information do not
hesitate to change the sequence of your questions if you think it would be
appropriate (e.g. if the respondent has mentioned something that you were not
originally intending to ask about until later in the interview) but remember to go back
to the questions you have temporarily skipped when the time is right.
5. Think about how you are going to end the interview. Don't let it simply 'trail off'
because you have both run out of time or things to say.
6. Even when you have switched off the tape recorder or put away your notebook you
need to stay 'switched on'. A common phenomenon in interviewing is that people
often give you very interesting and useful information once the formal interview is
over. Write it down as soon as you can but don't foreclose the situation by either
asking if you can switch on your recorder again or by saying 'I'm afraid that I'm in a
bit of a hurry!'

Focused group discussions

These are widely used in market research, particularly for testing reactions to new products,
and in politics for testing reactions to new policies. The method is also sometimes used in
health promotion research and social work research. As an approach it tends to be used in
combination with other research techniques.

Potential advantages

1. A good medium for exploring a group of people's views without imposing your own
agenda on them too strongly.
2. Provides opportunities to explore the thinking behind the kinds of responses which
might have been given to a questionnaire but without opting for the more expensive
option of follow-up one-to-one interviews.
3. In the early stages of a project it can be a useful means of identifying issues or areas
of interest that could be followed-up using more quantitative research methods.
4. A useful method of working with a group of people that operate as a group (a class
of students, a department of colleagues, an interest or pressure group, etc)
5. It provides the researcher with an opportunity to cross-check responses, e.g. 'Does
everyone agree with that?'...Does that reflect your experience as well?...etc.

Potential disadvantages

1. Difficult to follow-up the views of individuals during the group discussion, especially
on topics which may be sensitive.
2. Difficult to follow-up the views of individuals once the group discussion has been
completed. If it was easy to do this why not opt for face-to-face interviews in the first
place?
3. This method can be heavily affected by the dynamics of the group, especially if
some members of the group are senior to others. One or two people can easily
dominate the proceedings if they have clear views and are articulate. It is possible as
a facilitator to counter this tendency but it takes experience and self-confidence.

Design Issues

1. Keep the size of the group fairly small (8-12).

2. An hour is probably the minimum time to allow for the discussion to develop. It takes
a while for the group to settle and for the right atmosphere to develop.
3. Draw up a list of topics or themes to be covered. These should loosely guide the
discussion. You want a balance between allowing the discussion to develop its own
momentum and making sure your issues are addressed.
4. Introduce yourself (if they do not already know you) and explain the purpose of the
session, how long it will take and what you will do with the information.
 Page 14 of 17

5. Get them to briefly introduce themselves (if they do not know each other).
6. Decide how you will record the discussion. Will you tape it? Take notes? Get a
colleague to take notes?
7. Good chairing skills are essential (a) to ensure that the discussion is dominated by
one or two and (b) to find out if there is a consensus within the group on a particular
issue or theme and (c) to judge when to move on to the next theme or issue.

Observation
The systematic description of events and behaviours in their actual social setting. This
tends to be classified along three dimensions. Firstly, whether the observer is a participant
in what he or she is observing or is detached. Secondly, on whether or not those who are
being observed are aware of this. Thirdly, the extent to which the observation is structured.
The degree of structure can range from the observer taking notes or keeping a diary at one
end of the spectrum to using coding schemes, checklists and category systems at the other.

Potential advantages

1. The biggest advantage of collecting data through direct observation is that it is one
of the most direct research techniques. You are not asking people what they would
do or think you are watching what they do and listening to what they say.
2. Used in combination with questionnaires or interviews, observation can therefore
provide useful insights into the extent to which there is a correspondence or
discrepancy between what people say and what they actually do. As such
observation makes a useful contribution to the process of triangulating evidence.
3. The researcher observing his or her own practice has an advantage because the
situation to be observed is familiar (which in turn reduces preparation time).
4. It allows the researcher to describe the full complexity of the situation, including the
dynamics of the group being observed.

Potential disadvantages

1. It is very time consuming. How many times do you need to observe a situation or a
group before you can be sure that you are describing with confidence what is really
happening.
2. When the observer is also a practitioner observing his or her own practice then there
is a potential problem of how to ensure that te observer can detach himself or herself
sufficiently.
3. It is difficult to estimate the effect of the observer on the situation being observed.
4. The analysis of observational data is time consuming and often open to very
different interpretations.

Design Issues

Here is a brief summary of some of the key issues when designing an observation
schedule:

1. The first key question is: 'Are you going to be a participant or detached observer?
What are the advantages to either role in this particular setting?
2. Linked to that question is: 'Are you going to observe your own practice or the
practice of your colleagues and are you going to observe your students or a
colleague's?'
3. Are you going to tell the group that you will be observing them?
4. Do you want a narrative description of what is happening or do you want to design a
specific observation schedule. The former approach is often described as
'naturalistic' and used by social anthropologists and some sociologists. Some form of
observation schedule is more often used by psychologists and educational
researchers. Look at the different kinds of schedule in the recommended reading
before making up your mind on this.
5. What specifically do you want to observe:
 Page 15 of 17

i. Do you need to describe the setting? [the layout of the room, where people
are sitting, the equipment used, etc)
ii. Do you need to describe the people involved?
iii. Do you need to describe every event and activity over a specific time period
or just a sample of events, or select the ones you are interested in?
iv. Are you just interested in the interactions between the tutor and the students
or are you also interested in the interactions between the students?
v. Will it be useful to time each event or activity? What will this tell you? How will
you use this data?
When will you record your observations: at the time or immediately afterwards or
much later? [this can be a real problem when you are a participant observer].
How will you record your observations? Pen and paper, video recording, audio
recording?
If you are using an observation schedule can you use a coding system to make the
data collection process easier?
Is observation the only source of data for your research or will it be complemented
by interviews, questionnaires or other methods? If the latter how will you relate one kind of
data to another?

Documentary Analysis
Practitioners can make use of a wide range of documents, including course and module
outlines, course materials, staff development documents, inspection reports, examiners'
reports, minutes of meetings, memoranda, letters, diaries, students' journals, administrative
records, management information systems.

Potential advantages

1. Unlike most research instruments and data sources, documents are not so prone to
being influenced by the researcher.
2. Documents enable the researcher to investigate the background and context of the
situation and the specific problem they are interested in.
3. A useful means of evaluating the extent to which the rhetoric (or the policy) is
actually put into practice [but of course you also need other means and other
sources for evaluating practice].
4. Documentary analysis is a useful means of analysing the 'official' view and
accessing the 'official' record of events, decisions and plans.
5. Some documents [e.g. student assessments] can provide a measure of the impact of
changes introduced during the Action Research process.

Potential disadvantages

1. Research bias may be restricted in one sense because the researcher has not
produced the documents [in the same way as he or she has devised the
questionnaire or interview schedule] but bias can still be present in the selection of
the documents.
2. Documentary analysis - if it is to be systematic - is time consuming. Analysis can
range from reading and commenting on it to undertaking a full-scale content analysis
or from quoting a mission statement to undertaking a statistical analysis of student
assessment results for the last five years.
3. There is little guidance available from experienced researchers on how to analyse
some kinds of documents [e.g. students' diaries or reflective journals].
4. Documents require a critical reading similar to the skills employed by the historian
when analysing primary sources. Each document was written for a particular
purpose and within a specific context which may not necessarily be obvious from the
document itself. Documents have ot be interpreted asd well as read and this calls for
expertise and experience.

Design Issues
 Page 16 of 17

Historians, political scientists and economists may design their research around the
analysis of documents, records and statistics but for most action researchers investigating
their own and others' practice documentary analysis is mainly a supplementary method
employed for collecting background information and for triangulating with the main methods
used.

Experiments
The key feature of any experiment is that the researcher deliberately controls and
manipulates the conditions under which the effects of a change or intervention can be
measured. Cohen, Manion & Morrison (2000) provide a useful brief description of
experimental research in the natural and physical sciences:

Imagine that we have been transported to a laboratory to investigate the properties of a new
wonder fertiliser that farmers could use on their cereal crops, let us say wheat. The scientist
would take the bag of wheat seed and randomly split it into two equal parts. One part would
be the grain under normal existing conditions - controlled and measured amount s of soil,
warmth, water and light and no other factors. This would be called the control group. The
other part would be grown under the same conditions - the same controlled and measured
amounts of soil, warmth and light as the control group but, additionally, the new wonder
fertiliser. Then, four months later, the two groups are examined and their growth measured.
The control group has grown half a metre and each ear of wheat is in place but the seeds
are small. The experimental group, by contrast has grown half a metre as well but has
significantly more seeds on each ear, the seeds are larger, fuller and more robust." (p.211)

The key features of the experiment are:

 An experimental group and a control group;

 A random allocation to each group to eliminate the possibility that any variables not
thought to be crucial to the experiment might have any unintended effects;
 The identification of key variables that will have some effect;
 The control of these key variables
 The application of the special treatment to the experimental group but not the control
group;
 Measurement of the effect of the treatment and comparison of the outcomes for the
two groups.

Will this classical experimental design still work when the subjects of the experiment are
people rather than wheat seeds?

The experiment is still the norm in medical research and is widely used in all forms of
psychology, research into health care and, to a lesser extent, research into social care. One
particular form of the experimental design, the randomised controlled trial (through which,
for instance, new drugs and forms of medical treatment are tested) is still generally
regarded in those disciplines as the 'gold standard' of research. It is used to test new drugs
and forms of medical treatment and to evaluate the effectiveness of new interventions in
health and social care [Gomm & Davies (2000) p.xi].

In each case the experiment is designed in such a way that it reduces the likelihood that the
prior knowledge of the subjects, the practitioners and the researchers taking part in the trial
might unduly influence the results of the experiment.

However, in many social situations it is simply not practical (or sensible) to try to control all
of the possible variables that might influence the outcomes of a specific change or
intervention. Indeed, in some instances it would also be unethical to use a controlled
experiment if, for instance, the subjects were not in a position to give their informed consent
to participation in the experiment or if participation meant that they might suffer or be treated
unfairly or if the experiment required them to do something illegal or immoral. Also, in the
 Page 17 of 17

real world, it may not be possible to assign people randomly to either the experimental or
the control group.

In such circumstances some researchers have introduced the idea of a quasi-experiment.

Perhaps the most common kinds of quasi-experiment employed in social research are:
Where the researchers collect data that enable them to compare the same subjects before-
and-after an intervention or change has been introduced;

Where the researchers establish a pilot or study group (the subjects of their research) who
experience the change or intervention and a reference or comparator group from whom
data are collected using the same research instruments even though they are not
experiencing the change or intervention.

Therefore the quasi-experiment retains the element of comparison which is so central to the
experimental research design but subjects are seldom allocated to their groups and, if they
are, this is rarely done at ransom. In practice, in most quasi-experiments the researcher
does not have any control at all over the so-called 'control group' (or reference or
comparator group).

Ultimately the central question for any researcher opting for a quasi-experimental design will
be: 'Am I comparing like with like?' If the answer is: 'I believe so' then the follow-up question
will inevitably be: 'How do I know?'.

There are a variety of different kinds of quasi-experimental research design and each has
its own advantages and disadvantages. Rather than outline all of them here it is
recommended that you read the relevant section in Colin Robson's book, Real World
Research (1993 Oxford) Chapter Four: Experimental Design Outside the Laboratory,
especially pp.96-108.