OPERATIONS RESEARCH DESIGN

Virgilio C.Aganon IR 299.1 School of Labor & Industrial Relations 2nd Sem 2007-2008

PRESENTATION
Introduction Research Problem Strategy Selection Objectives & Hypotheses Intervention Description Operational

Definition Study Design Sampling Data Collection Tabulation & Analysis of Data

INTRODUCTION
Definition:

A process, a way of identifying and solving program problems: 1) Problem identification & diagnosis, 2) Strategy selection , 3) Strategy experimentation & evaluation, 4) Information dissemination, 5) Information utilization.

Introduction,Contd
Focus & Objectives:

Focuses on the day-to-day activities of programs. These are under the control of mgrs & adm such as trng, log, info & educ activities and delivery systems. Looks at problems affecting the supply side of programs. OR yield answers to perceived prog problems. Provide info to improve existing delivery activities and plan future ones.

Introduction,Contd
Categories of OR Studies

1. Exploratory or Diagnostic Studies: Needed whenever there is a perceived program problem but the nature of the problem is simply not known. 2. Field Intervention Studies: These test on an experimental basis, new approaches to overcoming a program problem. 3. Evaluative Studies: These examine retrospectively or cross-sectionally the effect of program activities.

Introduction, Contd
Methods

and Study Design 1.Methods: From Qualitative to Quantitative. 2. Study Design: From Non-experimental to True Experimental

Introduction, Contd
Illustrative

Topics: 1. Training Programs 2. IEC 3. Mgt & Info Systems 4. Program Impact 5. Adm & Mgt 6. Quality of Service

RESEARCH PROBLEM
A. Problem Identification A potential research situation arises when 3 conditions exist: 1. A perceived discrepancy bet. What is and should be. 2. A question about why the discrepancy exists. 3. At least 2 possible and plausible answers to the question.

Research Problem, Contd

B. Problem Definition: Determine all that is currently known about the problem and why it exists. Review relevant literature, examine current service statistics, seek educated opinions and obtain probable reasons for the problem from social,economic, or health theory. A careful review of existing sources of info and analysis of data help the researcher determine:

Research Problem,Contd
1. 2. 3. 4. 5. 6.

Incidence and prevalence. Geographic areas affected. Characteristics of population groups. Probable reasons for the problem. Possible solutions. Unanswered questions.

Research Problem, Contd

C. Problem Justification 1. Is the problem a current and timely one? Does it exist now ? 2. How widespread is the problem? Are many areas and people affected? 3. Does the problem affect special groups , such as mothers, or minorities? 4. Does it relate to ongoing program activities? 5. Does it relate to broad social, economic and health issues such as unemployment, income distn ? 6. Who else is concerned with the problem ?

STRATEGY SELECTION
A.

Guidelines:

1. 2. 3. 4. 5. 6. 7. 8.

Review the strategies other people have used to solve similar problem. Meet with the people who are most affected or most concerned with it. Look for strategies that can be implemented without overburdening the implementing agencies. Select strategies that can be sustained over time. Seek strategies that are simple to implement. Develop strategies where the solution is under the control of the program. Avoid strategies where the cost of the field test is higher than the expected benefits. Avoid strategies that are not consistent with the implementing agencys goals, objectives, & dev plans.

Strategy Selection, Contd

B . What to do: 1. Identify potential strategies that could be used to to solve the problem. 2. Clearly indicate which one or more of the potential strategies as the most appropriate. 3. Justify your selection by indicating: a. Past success with applying the strategy to similar problem situations. b. The simplicity of implementing the strategy over other strategies that are likely to produce the same impact. c. The potential for sustaining the strategy once the OR study is completed.

OBJECTIVES & HYPOTHESES

A.

B.

Ultimate Objectives: Relate the proposed research to broad social, economic, and other concerns. These are expected and hoped for contributions. Immediate Objectives: State what will happen.They indicate the variables that will be examined and measured. Whenever possible it should be stated in behavioral terms: who will do, how much of what, to whom, when, where, & for what purpose.

Objectives & Ho:, Contd

C.Hypotheses: A statement about an expected relationship between two or more that permits empirical testing.

INTERVENTION DESCRIPTION
Who will be responsible for implementing the intervention. B. Where will the intervention activities take place. C. What activities will be initiated.
A.

OPERATIONAL DEFINITIONS
Two Essential Purposes: 1. They establish the rules and procedures to measure variables; 2. They provide unambiguous meaning to terms that otherwise can be interpreted in different ways.

Operational Definitions, Contd

A.

Operational Definitions of Variables: The variables must be defined in terms of events that are observable by the senses.The observable events serve as an indicator of the variable. Usually,such an indicator is based on a series of questions. One can also establish categories or levels ofn a variable. This requires an operational rule that tells us how to asign any respondent to a category.

Operational Definitions, Contd

B. Operational Definitions of Terms: Just as it is necessary to define variables operationally, so too is it necessary to define the terms that indicate the nature of the relationship between variables: more than, greater than, less than, larger than, higher than, bigger than, lower than, smaller than, safer, significant, acceptable, expanded, improved, increased.

Operational Definitions, Contd

Each of these terms can have a variety of meanings, and thus each requires an operational definition for the research proposal. The basic problem with such terms as more than or increased is that they denote a comparison but do not indicate the standard of comparison. We need to know how much more and increased by how much.

STUDY DESIGN
A.

Reliability and Validity: 1. Reliability: Consistency, stability, or dependability of data. A reliable measurement is one that if repeated a second time will give the same results as it did the first time. 2. Validity: Refers to data that are not only reliable but also true and accurate.

Study Design, Contd

B. Validity Threats 1. History. Sometimes events occur during the life of the project that tend to either to increase or to decrease the expected outcomes of the project. 2. Selection. Occurs whenever people selected for the control group differ greatly from the people selected for the experimental group. 3.Testing. Whenever a pretest is given, it tends to have an effect on the posttest.

Study Design, Contd

B. Validity Threats, contd 4. Instrumentation. Whenever a measurement instrument (such as a questionnaire) is changed between the pretest and posttest, the change is likely to result in an effect that is independent of any effect due to a program or project. 5. Maturation: Over time, people (and things) change. In longitudinal studies, it is not unusual to find that respondents become more experienced , more knowledgeable, wiser, sometimes more resistant and older. People mature over time.

Study Design, contd

B.Validity Threats, contd 6. Mortality. In cohort studies (panel studies) where the same group is followed over time, there is always some dropout or lossof cases.

Study Design, Contd

C. Types of Study Design 1. Non-experimental Designs: a. Posttest Only Design Time Experimental Group X O1

Study Design, Contd

b. Pretest- Posttest Design Time Experimental Group O1 X O2 c. Static Group Comparison Experimental Group Control Group Time X O1 O2

Study Design, Contd

2. Experimental Designs a. Pretest-Posttest Control Group Design Time Experimental Group O1 X O2 RA Control Group O3 O4

Study Design, Contd

2. Experimental Design, contd b. Posttest Only Control Group Design Time Experimental Group X O1 RA Control Group O2

Study Design, Contd

3. Quasi-Experimental Design a. Time Series Design T i m e Experimental Group O1 O2 O3 X O4 O5 O6

Study Design, Contd

3. Quasi-Experimental Design b. Nonequivalent Control Group Design Time Experimental Group O1 X O2 Nonequivalent Control Group O3 O4

Study Design, Contd

3. Quasi-Experimental Design c. Separate Sample Pretest-Posttest Design T i m e RA Pretest Group O1 X RA Posttest Group X O2

Study Design, Contd

D. Selecting a Study Design 1. Ethical Issues 2. Practical and Administrative Issues 3. Technical Issues

Study Design, Contd

E. Guidelines for a Good Rsch Design 1. An ethical research design 2. One that is capable of obtaining the most reliable and valid data possible given the constraints of funds, time, personnel & eqpt. 3. One that is capable of measuring whatever it is that happens in the field setting , whether the impact of planned intervention activities or the impact of unplanned and possibly invalidating events. 4. One that helps an investigator avoid making mistaken conclusions.

Study Design, Contd

F. The Principle of Three Multiples 1. Seek multiple data sources to obtain information on the same variables. 2. Seek multiple measurements over time of the same variables. 3. Seek multiple replications of the study intervention in different field settings.

SAMPLING
Selecting Area & Population Information refers commonly to individuals Referred to subjects, respondents, or cases Cases need not be individual persons;they may be aggregations or group of persons.
A.

Sampling,Contd
B. Selecting Cases Population: Composed of elements each of which is a potential case. Sample: A subset of the population from information will be collected. Basic Types of Samples: Probability and Non-probability samples.

Sampling, Contd
B. Selecting Cases, contd 1. Probability Sample: a. Simple Random Sampling b. Systematic Sampling c. Stratified Sampling d. Cluster Sampling e. Multistage Sampling f. Probability Proportional to Size

Sampling, contd
B. Selecting Cases, contd 2. Non-probability Sample a. Accidental b. Purposive c. Quota sampling

Sampling, contd
C. Sample Size Determination Determined primarily by: -Availability of resources, which sets the upper limit of the sample size; - Req. of the proposed plan of analysis, which sets the lower limit. Must be large : - To allow for reliable analysis of x-tabs; - To provide for desired levels of accuracy in estimates of proportions; - To test for the significance of differences of proportions.

Sampling, Contd
C. Sample Size Determination, contd 1. What are the reasonable estimates of key proportions to be measured in the study ? 2. What degree of accuracy do you want to have in your study? 3. What confidence level do you want to use? 4. What is the population size? 5. What is the minimum difference you expect to find statistically significant?

Sampling, contd
C. Sample Size Determination, contd n = (z^2) pq (d^2) n = ( n ) 1 + n/N

Sampling, contd
n = (1.96)^2 (.50) (.50) = 384 (.05)^2 n = ( 384 ) 1 + (384/1,000) = 277

DATA COLLECTION
A. 1.

Quantitative Data Structured Interviews a. Use simple language; b. Whenever possible, precode responses to the questions; c. Avoid embarrassing or painful questions; d. Avoid asking for more than one item of information in a single question; e. Watch out for ambiguous wording; f. Do not overload your interview schedule; g. Include all questions necessary to provide sufficient information;

Data Collection,Contd
1.Structured Interviews, contd h. Start with easier questions; i. Ask all respondents each question in exactly the same way; j. Pretest the Q in an actual field situation; k. Provide complete training for all interviews; l. An appt should be made for a call-back visit; m. Interviewers must be given clear instructions for obtaining substitutes; n. Isolate the respondent during the interview; o. Check all completed interview schedules for errors, omissions, and discrepanciesas soon after interviewing as possible.

Data Collection, Contd

2. Service Statistics 3. Self-Administered Questionnaires - Likely to be useful in situations where literate respondents are already gathered together in a setting where they can write.

Data Collection, Contd

B. Qualitative Data 1. Unstructured Interviews: The interviewer has an outline of topics or a set of general questions to serve as a guide to the kind of information required. Probe for details; 2. Focus Group Discussion; 3. Direct Observation of Operations; 4. Content Analysis of Written Materials

Data Collection, Contd

C. Data Quality Checks: 1. Include repeat questions that can be used to check consistency of response; 2. Have supv monitor the work of the interviewers in the field; 3. Reinterview a % of respondents & look for inconsistencies; 4. Recode a % of the Q to be sure that there are no coding errors; 5. Examine the frequency distn on all variables to see if there are odd codes or items that are not logical.

Data Collection, Contd

D. Confidentiality of Information 1. Describe in detail how you plan to maintain the confidentiality of info collected. 2. Include a sample of the Informed Consent Form in your study proposal.

TABULATION & ANALYSIS OF DATA

A.

Preparing Tabulations - Data Coding - Editing of Coded Data 1) Illegal Codes 2) Omissions 3) Logical Inconsistencies 4) Improbabilities

Tabulation & Analysis of Data

B. Plan for Analysis of Data 1. Attributes of the data a. Central Tendency b.Variance in the Data c. Differences within the Data d. Relationship within the Data

Tabulation & Analysis of Data

2. Types of Analytical Procedures a. Variable Transformations: Recodes, Counts, Scaling, Conditional Transformations,Others b. Univariate Analysis: Freq Distn * Nominal Measurement * Ordinal Measurement * Interval Measurement

Tabulation & Analysis of Data

c.Time Series Analysis d. Comparisons e. Bivariate Relationship f. Multivariate Analysis g. Cost-Effectiveness Analysis h. Use-Effectivenes C. Dummy Tables