Developing and Testing Self-

Report Scales

Adopted by
Ibrahim R. Ayasreh
RN, MSN

Supervised by
Prof. Lubna Abu-Shaikhah
 Objectives

At the end of this presentation, student will be able to:

- Identify the major steps of developing and testing self-report scales.

- Identify the preliminary evaluation measures of items.
- Identify the major principles of scale development data analysis.
- Identify the major principles of translating scales to other languages.
- Evaluate and critique scale development studies.
 Beginning Steps: Conceptualization And Item
Generation

Conceptualizing the Construct:

- Become an expert on the construct.
- Identify and understand all dimensions of the construct.
- Differentiate the construct from other related ones.
- Conceptualization of the population for whom the scale is intended.
 Beginning Steps: Conceptualization And Item
Generation

Deciding on the Type of Scale:

- Two types of scales used which are traditional likert scale, and
latent trait scales. ( what is the difference ??? )

Developing an Item Pool: Getting Started:

- To develop a possible items for the scale from possible sources
include:
1) Existing instruments. 2) The literature.
3) Concept analysis. 4) In-depth qualitative research.
5) Clinical observations.
 Beginning Steps: Conceptualization And Item
Generation

Making Decisions about Item Features:

- Number of items:

Domain sampling model.

Starting with a large number of items

(at a minimum there should be 50% more at beginning).
 Beginning Steps: Conceptualization And Item
Generation

Making Decisions about Item Features:

- Response options:

Scale items involve a stem, and a set of response options.

Likert scales often involve response options on a continuum of agreement,

frequency (never/always), importance (very important/unimportant),
quality (excellent/very poor), and likelihood.

Most likert scales have 5 to 7 options.

Midpoint option allows them to express neutrality

 Beginning Steps: Conceptualization And Item
Generation

Making Decisions about Item Features:

- Positive and negative stems:

Including both positively and negatively worded statements

and to reverse-score negative items is advised. WHY ????

Acquiescence can be minimized by putting the most positive

response options (e.g., strongly agree) at the end of the list rather
than at the beginning
 Beginning Steps: Conceptualization And Item
Generation

Making Decisions about Item Features:

- Item intensity:

The intensity of the stems should be similar and fairly strongly

worded.

Statements should not be so extremely worded as to result in

universal rejection
Beginning Steps: Conceptualization And Item
Generation

Making Decisions about Item Features:

- Item time frames.

- Wording the Items:

1) Clarity. 2) Jargon.
3) Double negatives. 4) Length.
5) Double-Barreled Items.
 Preliminary Evaluation Of Items

Internal Review:

- Do items capture the construct ?

- Are items grammatical and well worded ?

- Readability.
 Preliminary Evaluation Of Items

Input from the Target Population:

Pretest initial items with a sample of 10 -20 from the

target population.
Are there statements that confused you?
Did you understand the meaning of each question?
Were the directions clear?)

Convention with focus groups

 Preliminary Evaluation Of Items

External Review by Experts:

Revision of items by a panel of experts.

- Criteria for selecting and recruiting the experts:

1) Have clinical or personal experience regarding construct.
2) Have published papers in refereed journals regarding construct.
3) Should be knowledgeable about the key construct and population.
 Preliminary Evaluation Of Items

External Review by Experts:

- It is advisable to undertake two rounds of review.

- In the initial phase of a two-part review, we advise having an expert panel

of 8 to 12 members, with a good mix in terms of roles.

- The second panel (3 to 5 experts in the content area) for formally

assessing the content validity of a more refined set of items should consist
Preliminary Expert Review: Content
Validation of Items
 How to evaluate

S-CVI & I-CVI

???????
 Administration To A Development Sample

Developing a Sampling Plan:

- The sample for testing the scale should be representative of the population.

- It is advantageous to recruit a sample from multiple sites.

- Some suggest that 300 is an adequate number to support a factor analysis.

- Recommendations range from 3 or 4 people per item to 40 or 50

per item, with 10 per item being the number most often
recommended.
 Administration To A Development Sample

Developing a Data Collection Plan:

- How to administer the instrument (e.g., by mailed or distributed

questionnaires, over the Internet) ?

- What to include in the instrument ?

- The instrument should include the scale items and basic

demographic information.
 Administration To A Development Sample

Preparing for Data Collection:

- Friends, colleagues, mentors, or family members evaluate the

appearance of the instrument.

- Instructions for completing the instrument should :

1) be clear, readable.
2) encourage candor.
3) State that there are no right or wrong answers.
Administration To A Development Sample

Proximity Effect

?????
 Analysis Of Scale Development Data

Inter-item correlation:

- It is the extent to which scores on one item are related to

scores on all other items in a scale.

- Values between .30 and .70 are often recommended.

Itemscale correlations.
 Analysis Of Scale Development Data

Exploratory Factor Analysis:

- Factor Extraction.

- Factor Rotation.
Final Steps: Scale Refinement And Validation

Revising the Scale.

Scoring and Transforming the Scale.
Conducting a Validation Study.
Establishing Cutoff Points.
Establishing Norms.
Translating Scales Into Other Languages

Centered versus decentered translations/Conceptual

equivalence.

Semantic equivalence (back translations):

Selecting and preparing translators
Undertaking an iterative process
Testing the translated version
 Evidence for Semantic Equivalence and
Psychometric Soundness

Pretests of original and translated scale

Sample of bilingual people
Comparison:
- Reliabilities
- Factor structures
- Validity estimates between two scales