EVALUATING DATA IN

PSYCHOLOGY
-using Validity and
Reliability
& Sample
Introduction to Psychology
EVALUATING DATA
An important goal in psychology is to obtain results that are both reliable
and valid.

Need to determine if data is high in validity (accuracy) and reliability

(consistency).
 VALIDITY
The degree to which a research study has produced results that accurately
measure what it claimed to measure.
Example:
Weighing yourself with a faulty bathroom scale.

IQ test- is it measuring intelligence or something else?

External Validity
Internal Validity: Research study has produced results that accurately measure the
behaviour or event it claims to have measured
– are the conclusions drawn from the results valid, or may the data have been
interpreted incorrectly?

External Validity: whether the results can be generalised to the real world
– Eg – research conducted in a sleep lab-does this relate to sleep at home?

Measuring things in a laboratory does not always accurately predict what would
happen in the real world.
Extraneous Variables & Validity
EV’s- other variables that are not the Independent Variable that can change your
results (dependent variable).

The more EV’s the less validity.

RELIABILITY-
The extent to which an experiment yields the same results on repeated trails (so the
degree of stability exhibited when a measure is repeated under identical conditions.)

Example: measuring height with a ruler and then double checked it, should get same result =
high reliability.

**A measure can be reliable without being valid, but a measure cannot be valid unless it is
reliable.

Biggest threat to validity and reliability are extraneous variables.

Reliable but not valid means that you are consistently testing the same thing over and
over again, but it’s not testing what you want to test.

Valid but not reliable means that the average scores align with the goals of the test, but
individual scores are inconsistent.

Both reliable and valid means that the test will consistently measure what it is supposed
to over a period of time – it’s consistently hitting the bullseye.
 Exam Question 2015
An educational psychologist investigated the development of
numeracy skills in children.
Letters were sent to all parents/guardians of Year 3 children in
three schools, asking for volunteers to participate in the
investigation. Twenty parents/guardians replied and their
children were accepted.

All participants in the investigation completed a numeracy test

(Test 1). The results were used to place the participants into ten
matched pairs, based on the similarity of their scores.
Each participant in a pair then had an equal chance of being
assigned to Group A or Group B.

Participants in Group A were required to watch numeracy

programs on television for 30 minutes every day for 4 weeks.

Participants in Group B were required to watch cartoons for 30

minutes every day for 4 weeks. After the 4 weeks, all participants
completed a second numeracy test (Test 2).
Answers
(e)(i) two marks for the following:
Watching numeracy programs improves numeracy skills

(ii) Two marks for the following (reduce vaildity)

Only three students from selected schools; data unrepresentative
Only volunteers; data unrepresentative
No random selection; data unrepresentative
Small samples size reduce external validity

(iii) Two marks for definition and two marks for the evaluation: (define reliability and use example)
Reliability: The consistency and stability of the results obtained under repeated conditions. Outliers present. Therefore
low reliability in the results.
The larger the sample size, the more accurately the results represent the population
Subjects must be assigned to groups randomly
Groups should be roughly equal in a number of variables, so that each is a good
representation of the whole population
Should be similar in terms of age, sex, background, experiences, etc…
 We are unable to generalise the results to the population

To avoid bias  random selection of participants

- otherwise the sample may be biased in terms of:
-gender, age, socio-economic status, background knowledge, school, geographic location

For greater confidence in results  more participants

For good generalisation of results  participants must have similar balance of characteristics as
original population
Limitations of the
Small or unrepresentative sample
Results
To avoid bias therefore random selection of participants
For greater confidence in results means you need to have
more participants
For good generalisation of results participants must have
similar balance of characteristics as original population

Can you think of any other limitations in a design?

Exam Question 2014
Answers
1. Two marks for one of the following:
Sample is biased because there could be demographic factors which are unique to the
school, therefore no external validity.
Conclusions cannot generalised to the population because the sample is small or comes only
from one school.

2. Two marks EACH for BOTH of the following:

Measurements are reliable because HR monitors consistently gives reading 5 beats higher
than it should.
Measurements are not valid because the readings are not accurate (not measuring what it
should be measuring).