Evaluation Research

GROUP 3
 Evaluation Research
• Definitions:
– “Evaluationis the systematic
assessment of the worth or merit
of some object”
– “Evaluation is the systematic
acquisition and assessment of
information to provide useful
feedback about some object”
(Trochim, 2006)
 Evaluation Research
• Basically is used to provide feedback on an
event, organization, program, policy,
technology, person, activity, etc.
– All of these are called “object(s)”
• It assesses information about the “object”
• Used heavily in social sciences and government
agencies
• Usually outcomes influence
decision-making process
Research Designs

• 4 Major Types:
1. Survey Research
2. Case Study
3. Field Experiment
4. Secondary Data Analysis
(Garson, n.d.)
Types of Evaluations:
• 2 Important Types:
1. Formative Evaluation
2. Summative Evaluation
 1. Formative Evaluation
A formative evaluation (sometimes
referred to as internal) is a method for
judging the worth of a program while the
program activities are forming (in
progress). They can be conducted
during any phase of the ADDIE process.
This part of the evaluation focuses on
the process.
 Purpose of Formative Evaluation
• Purpose:
 To strengthen or improve the object being
evaluated
• In other words, to provide useful feedback to for
the greater good of the “object”

• Questions generally asked:

– What is the definition and scope of the problem or
issue, or what’s the question?
– Where is the problem and how big or serious is it?
– How should the program or technology be delivered
to address the problem?
– How well is the program or technology delivered?
2. Summative Evaluation

• A summative evaluation (sometimes

referred to as external) is a method
of judging the worth of a program at
the end of the program activities
(summation). The focus is on the
outcome.
 Purpose of Summative Evaluation
• Purpose:
– To examine the effects or outcomes of some
object.
• In other words, to review the
results/conclusions of an “object”
• Questions generally asked:
– What type of evaluation is feasible?
– What was the effectiveness of the program
or technology?
– What is the net impact of the program?
 What is the Purpose of Evaluation
Research
• The purpose of the study was to “examine
subject matter topics researched in
agricultural education over a ten year
period”. Its purpose is to make judgments
about a program, to improve its
effectiveness, and/or to inform
programming decisions.
Summary of Evaluation Research Method
• The outcomes of this study will
provide useful feedback to those in
the profession.
What is in a name: The impact of job titles on job
evaluation results
Authors and affiliations
Brien N. Smith, Jeffrey S. Hornsby, Philip G. Benson, Mark Wesolowski
Journal of Business and Psychology
March 1989, Volume 3, Issue 3, pp 341–351|
Abstract
The effect of job title status on job evaluation ratings was examined.
Eighty-six personnel management students used the Factor Evaluation
System (FES) to evaluate two job descriptions. One of three different
forms of a secretary and accountant job description, differing only on
the status of the job title, was randomly assigned to the subjects. The
results showed that job title status significantly influenced job
evaluation ratings for both the accounting and secretarial jobs. The
implications of these findings are discussed and recommendations are
made to avoid the contamination of job evaluation results by job title
status.
Effect of Female-Linked Job Titles on Job Evaluation Ratings
Thomas J. Naughton
First Published December 1, 1988 Research
Articlehttps://doi.org/10.1177/014920638801400407
Abstract
In this study an experiment was performed to test whether sex role
stereotyping evoked through female-linked job titles might influence job
evaluation ratings. One hundred and ten subjects who had received training in
job evaluation procedures rated two jobs on nine factors commonly used in job
evaluation plans. For one half of the subjects the job titles for the two jobs
were changed to reflect a female sex linkage. The results indicate that one of
the two jobs with a femalelinked job title received 5.6% fewer total points. A
key finding was that, in comparison to the sex-linkage effects, a large
proportion of variance in ratings reflected between job differences. Where bias
occurred, however, it appeared to be based on job content factors (effort and
responsibility required) rather than skill or human capital (education and
experience)factors. Potential implications for managers as well as limitations of
the research are discussed. Suggestions for future research are also noted.
Thank You