11/6/2013

TOPIC: Data Collection

(Experimental)
DR. CHITRAREKHA KABRE

Prof (Dr) Chitrarekha Kabre

The research process

Review the literature
Reviewtheliterature

Define
research
problem

Review
concepts
and
theories

Formulate
hypotheses

Design
research
(including
sample
design)

Collect
data

Analyse
data
(Test
hypothes
is if any )

Interpret
and
report

Review
previous
research
finding

Prof (Dr) Chitrarekha Kabre

11/6/2013

Experiments offer the best known scientific way to establish causality

or causal relationship between variables.
Three conditions have to be met for p
proving
g that and event A
(presumed cause) actually causes event B (presumed effect):
1. Co-occurrence (covariation): Both A and B must be observable and
measurable, and that some sort of relationship exists between the
cause and the effect (if A then B, if not A then not B)
2. Sequence (temporal precedence): Cause must come first then the
effect second (A precedes B)
3. Elimination of alternative explanations: Eliminate other possible or
plausible causes.
causes Alternative hypotheses are tested.
tested
Traditionally, an experiment involves (1) taking an action, and (2)
observing its effect.
The experiment examines the effect of an independent variable
(cause - an experimental stimulus or treatment) ON a dependent
variable (effect). I.e. examine cause on effect.

Prof (Dr) Chitrarekha Kabre

Research

on the performance of various

b ilding components constitutes
building
constit tes a
significant and longstanding domain within
architectural research as a whole.
Much of this research has focused on
improving building technologies in the
industrialized world.

11/6/2013

Givoni,

Gulich and Gomez radiant

cooling by metal roofs corrugated metal
roofs are effective for cooling in the
evening, they tend to overheat houses in the
day.
Hypothesis-the installation of operable
hinged interior insulating panels under the
roof would reduce daytime heating without
interfering with the nighttime cooling
function of the metal roofs.

11/6/2013

A small scale mock-up model of the typical house

((~test cell)) to measure the heating/cooling
g
g effect of
various test conditions.
Givoni et al tested three conditions:
1. With the insulation panels closed both day and
night
2. With the insulation panels open at night and
closed during day
3. With the insulation positioned as in condition 2 but
with the addition of a small ventilating fan from
midnight to 5 am. In addition, two levels of thermal
mass (water filled bottles) were also tested.

Ann

Sloan Devlin sought to discover the

e tent to which
extent
hich gender might ha
have
e an
effect on how job applicants are evaluated
in architectural practice.
Hypothesis- women architects would be
less favourably rated than male architects,
especially at the more senior level.

11/6/2013

To test this hypothesis, Devlin created both a juniorlevel (four years of experience) and senior-level
( hi
(thirteen
years off experience)
i
) resume
Copies of each resume type (junior and senior)
were created using a fictitious female name; an
equal number of copies carried a fictitious male
name.
Each resume included a career objective,
professional experience, affiliation, registration,
education, skills, honors and awards.
Respondents in the study were more than 200
architects (156 men and 48 women) Connecticut,
but representing all regions of the country.

The respondents were told the study was about the

perception architects have of the characteristics
possessed
db
by those
h
practicing
i i architecture.
hi

The respondents received one of the four fictitious

resumes and were asked to evaluate the candidates
on a seven point scale in the following areas:
i) technical aspects of the job, ii) administrative
aspects, iii) interpersonal aspects, iv) contribution
to growth of firms client base, v) creative
contribution, vi) advancement, and vii) overall
ability.
Respondents were asked whether they would
accept or reject the candidate for hire.

11/6/2013

Salient

result: Male architect respondents

were
ere more likely
likel to hire male applicants
than female applicants as senior architects.
She concludes that women in architecture
may indeed experience discrimination as
they advance through the ranks.

Givonis experiment

Devlins experiment

Tackles an aspect of
environmental technology

Clarify the dynamics of

gender discrimination in
architectural practice

Context: laboratory setting

Context: real life or field

setting

Variables: physical

Variables: behavioural and

social conditions

11/6/2013

1. The use of a treatment or independent

2.
3.
4.
5.

variable
ariable
The measurement of outcome, or
dependent variables
A clear unit of assignment (to the
treatment)
The use of comparison (or control) group
A focus on causality

The

researchers are seeking to study the

impact of one or more specific,
specific identifiable
variables on the phenomena under study.
Givoni et al are seeking to test the thermal
impact of several conditions
Devlin is seeking to clarify the impact of
gender designations on how architects
evaluate
l t job
j b applicants.
li
t
These variables are manipulated or controlled
by the researchers in some specified way and
so they are considered to be treatments

11/6/2013

The

impact of the experimental treatment by

measuring certain outcome measures or
dependent variables
Givoni et al.s study, the dependent variables
were the temperature readings for indoor areas
of the test cell environment.
Devlin was able to assess the impact of gender
d i
designations
ti
through
th
h two
t
measures: a
questionnaire instrument evaluation on a 1to -7 rating scale, and a hiring decision to
accept or reject.

Givoni

et al.s research, the treatment

conditions are all applied to an inanimate
object test cell- a 1-meter cube with metal
gabled roof.
Devlins study the unit of assignment is the
individual architects who were asked to
evaluate the fictitious job applicants.
Each of these units whether test cells or
individual architects-received a treatment
manipulated by the researcher(s).

11/6/2013

The

control condition is defined as one to

which the treatment is not applied
Givoni et al.s experiment control conditioninsulation panels closed both day and night so
that no heating or cooling occurs.
In Devlins study, all architect respondents
received some treatment,
treatment one of four
combinations of male or female applicant at a
junior or senior level. In this case, the different
treatments are compared against each other.

The

experimental research design is to

enable the researcher to credibl
credibly establish
a cause-effect relationship.
The experimental researcher is seeking to
ascertain and measure the extent to which a
treatment causes a clearly measured
outcome within a specified research setting,
whether in a laboratory or in the field.

11/6/2013

1. In the gender discrimination study Devlin

could
co
ld randomly
randoml assign the architects to
various resume conditions
2. In Givoni et al.s experiment based on
inert materials does not necessarily
require such randomization measures.

1. To test the effect of four lighting systems on

employee
p y p
productivity
y in four separate
p
office
areas.
2. A small gallery area near the school offices
that had been created to function as both an
exhibit space and a lounge area for faculty
and students.
Hypothesis: the gallery would be used more
if the
th arrangementt off furniture
f
it
were less
l
formal and if small screening elements were
used to block the view through the glass wall
along the doorway side of the space.

10

11/6/2013

Observations were made on

Monday (studio day) and Tuesday
(nonstudio day) of two successive
weeks, starting at 8:30 in the
morning and continuing to 7:30 at
night.
The experimental treatment
condition used in the second twoday observation inviting
ambience
Furniture arrangement
Lighting levels
Ambient sound
Addition of screening elements
Reallocation of furniture-privacy
and grouping
Reading material on the tables
Introduction of plants.

Observations:
A count of number of people using
the space during the observation
period
A plan of the gallery including
furniture arrangement
g
mapping
pp g
peoples movement and activities

Observations:
A coding system to describe
peoples specific activities
(speaking, writing/reading,
sleeping)

11

11/6/2013

12

11/6/2013

Study

Setting

Treatment

Outcome
measures

1. Radiant
Lab
cooling (Givoni
et al)

Environmental
modifications
Modification
insulation
venting mass

Instrumented
measures air
temperature

2. Gender
issues (Devlin)

Field

Resumes
gender and
seniority

Attitudinal
response
applicant
evaluation
hiring decision

3. Gallery
3
behaviour
(Barnes et al.)

Field

Environmental
modifications
furniture
lighting
ambient sound

Behavioural
change
staying/movin
g screens.

The basic experiment involves three pairs of

components:
Independent
I d
d
and
d dependent
d
d
variables
i bl
Pretesting and posttesting
Experimental and control groups
The true experiment could do without the
pretesting if subjects are randomly assigned to
groups to ensure equivalency between groups.
Notations of experiments
1. O = observation (measurement of dependent
variable - effect) eg. Pretest or posttest
2. T = treatment or experimental stimulus
(introduction of independent variable - cause)

13

11/6/2013

Notations
1.
2.

3.

1.
2.

3.

of experiments
R = Random assignment
X = Experimental treatment stimulus
(introduction of independent variable cause)
O = observation (measurement of
dependent variable - effect) eg. Pretest or
posttest

The study of radiant cooling is represented below. Each

row represents,
p
from left to right,
g the sequence
q
entailed in
each treatment condition
The notation conveys three essential points:
There is no explicit attention paid to random assignment
There are three different treatment conditions in addition to the
control condition.
Only posttest (i.e. no pretest) observations are made.
O

{Ob
{Observation
ti only,
l with
ith no prior
i ttreatment}
t
t}

X1

{Treatment 1, and subsequent observation}

X2

{Treatment 2, and subsequent observation}

X3

{Treatment 3, and subsequent observation}

14

11/6/2013

The study of gender issues in architectural practice

presents a slightly
p
g y different research design
g
Random assignment is an explicit and important consideration for
establishing comparability across treatment groups;
There is no explicit control condition.
Only posttest (i.e. no pretest) observations are made.

1.
2.
3.

X1 O

{R d
{Random
assignment,
i
t ffollowed
ll
db
by ttreatment
t
t 1,
1 observation}
b
ti }

X2 O

{Random assignment, followed by treatment 1, observation}

X3 O

{Random assignment, followed by treatment 1, observation}

X4 O

{Random assignment, followed by treatment 1, observation}

The student study of behavioural patterns in a gallery

space
p
p
presents a slightly
g y more ambiguous
g
research
design. If the gallery users had been substantially the
same group, then the notation of the research design:

O O X O O

{Two observations, treatment, followed by two observations}

This design is known as a single-group interrupted timeseries design. Two pretest observations were made, after
which the treatment (physical modification) was applied,
followed by two posttest observations.
If two sets of users were substantially different, diagram
X

O O

{No treatment, two observations only}

{Treatment, followed by two observations}

15

11/6/2013

Experimental Research
Issues

Strengths

Weaknesses

Efficacy and
Accuracy

Potential for
establishing
causality
y

Reduction of
complex causality
reality
y to identify
y
causal or
independent
variable

Misapplication

Potential for
generalizing results
to other settings and
phenomena

Misuse by
overgeneralization
to different ethnic,
gender populations

Ethical concern

Ability to control all

aspects of
experimental
design enables
attribution of
causality

Overemphasis on
control yields
ethical problems,
dehumanization

In

a two-level experiment (two levels or

conditions of the independent variable) and
when trying to decide upon how subjects are
assigned to both experimental conditions, one
is faced with two options.
Either randomly assign a group of subjects to
one condition and another group of subjects to
th other
the
th condition
diti (b
(between-subjects
t
bj t design),
d i )
or assign the same group of subjects for both
conditions of the experiment (within-subjects
design).

16

11/6/2013

Possible

effect of the experiment itself

rather than of the experimental
e perimental treatment
Need control group or Placebo (e.g. sugar
pill in medical research)
Benefits of the control group
Prevent the Hawthorne effect
Prevent effects of factors outside the experiment

Experimental

research designs may differ in

several ways.
y The following
g are some of those
variations:
1. One group pretest-posttest
2. Control group pretest-posttest (two group)
3. Randomized Solomon four-group design
4. Randomized control-group posttest only
5. Nonrandomized control-group pretestposttest
tt t
6. Counterbalanced treatments
7. One-group time-series
8. Control-group time-series

17

11/6/2013

1.
2.
3
3.
4.

History (effect of historical outside event)

Maturation (subjects growing older, tired, hungry, etc.)
Testing (Pretesting or posttesting influencing behavior)
Instrumentation (measurement instrument between pretest &
posttest, comparable)
5. Statistical regression (extreme measures of dependent
variable)
6. Selection biases (equivalent, comparable groups)
7. Experimental mortality (subjects dropping off)
8. Causal time-order (did stimulus cause dependent var. or
dependent var. caused changes in stimulus; rare)
9 Diffusion
9.
Diff i or imitation
i i i off treatments (passing
(
i iinfo
f from
f
experimental group to control group)
10. Compensation (effect of compensation on control group)
11. Compensatory rivalry (deprived control group try to beat
special experimental group)
12. Demoralization (deprived control group give up)

Interaction

between the testing situation &

the experimental
e perimental stim
stimulus
l s (testing
interaction with the stimulus) groups took
pretest in experiment, groups will not take
pretest in real life.
Solving Threats to External Validity
Use Solomon 4-group design with random
assignment of subjects into the groups.

18

11/6/2013

Variables come in different types depending upon

their role in the experiment.
Since
Si
the
h independent
i d
d t variable
i bl is
i the
h variable
i bl that
h
influences another variable (the dependent
variable), in this case it is the variable that is
manipulated or systematically altered by the
experimenter (also called experimental variable or
predictor variable).
Since the dependent variable is the one influenced
by another variable (the independent variable), in
this case it is the variable that is affected or is the
outcome of the manipulation (also called outcome
variable, response variable, criterion variable).

Any variable that may have an influence on the

effect of the experimental manipulation. Sommer
and
d Sommer
S
(1997) explain
l i three
h
ways to controll
the effects of extraneous variables (read more on
these three techniques in the required reading):
eliminate or hold extraneous variables constant
measure the extraneous variables to take them into
account in the analysis
use a control group or control conditions
The true experiment or simply the experiment has
to achieve two conditions:
Random assignment is possible
Control over extraneous variables is possible

19

11/6/2013

The

one-shot case study (no pretest & no

control group
gro p misleading results)
res lts)
One-group pretest-posttest design (no
control group factor other than
independent var. may be cause)
Static group comparison (no pretest initial
status of groups unknown)

20