Validation of the Paediatric Hearing Impairment Caregiver Experience

Questionnaire in Singapore Population

Hsueh Yee LynneLIM
1
, MBBS, FRCS, MPH, Ling XIANG
2
, MBBS, MMed, MSc, Ruijie Li
3
, MSc, Naomi Wong
1
, BSpPath (Hons), Chi Pun Kevin Yuen
4
, PhD
1
Otolaryngology - Head Neck Surgery Department, National University Health System, Singapore,
2
National University of Singapore,
3
Health Services & Outcomes Research, National Healthcare Group, Singapore,
4
Department of Special Education and Counselling, The Hong Kong Institute of Education
Corresponding author:
LI Ruijie (Ruijie_li@nhg.com.sg)
Research Analyst,
Health Services & Outcomes Research,
National Healthcare Group,
Singapore
Background
The stress experienced by parents of children with hearing loss has been studied over the past 2 decades. Increased parenting
stress is often associated with developmental problems such as delay in acquisition of language, social and emotional
problems. The stress that parents endure is also subjected to fluctuation as they go through the process of meeting and
overcoming various challenges associated with their childs hearing impairment. An instrument, the Paediatric Hearing
Impairment Caregiver Experience Questionnaire (PHICE) was therefore developed to document the levels of stress as
experienced by parents of children with hearing impairment
1
. While this instrument has been validated in the United States, it
use would not be appropriate for use in a Singapore context due to the varying infrastructures and culture of the hearing
impaired in the 2 countries. This study therefore aims to modify and validate the PHICE questionnaire to make it relevant for
use in Singapore.
The PHICE questionnaire is a 68 item instrument spanning across the domains: communication, education, emotional well-
being, equipment, financial, healthcare, social, and support. The item is scored on an 8 point Likert scale with the following
descriptors: "Not applicable", "No stress", "Very low stress", "Low stress", "Moderate stress", "High stress", "Very high stress"
and "Extremely high stress". Given its length, it takes a considerable amount of time to complete the questionnaire. Therefore,
another aim of this study is to create a shorter version of the questionnaire.
Methods & Analysis
Expert panel review
Prior to administering the questionnaire on the local population, an expert panel was convened to assess each item on the
questionnaire for its suitability for use in Singapore. The expert panel consisted of an otolaryngology surgeon, an audiologist
and two speech-language therapists. After a round of review, 7 questions were removed.
These 7 questions were removed primarily because of the focus on sign language. These items were considered irrelevant
because in the Singapore, only a limited number of children with hearing loss are attending the Singapore School for the Deaf,
the only school whose primary medium of instruction is sign language in Singapore.
Administering of the questionnaire
125 caregivers of children with permanent hearing loss for at least 6 months attending the otolaryngology, audiology and
aural rehabilitation clinic were recruited into the study. Informed consent was sought and a total of 125 completed PHICE
questionnaires were collected between January 2006 and December 2008. The questionnaire was self-administered.
Scaling
The items were recoded such that the Not applicable entries were mapped to No stress. This is on the assumption that if it
is not applicable to a caregiver, they should not face stress in that particular area. This recoding is necessary in order to
preserve the inherent assumption of equidistant between points on the Likert scale.
Missing data
Missing data was filled by cross validation imputation using the package missMDA
2
through R
3
. The amount of missing
data was deemed to be small at 0.72%. The imputed data was rounded off to the nearest integer to maintain the ordinal
nature of the scale and to make the data interpretable
4
.
Exploratory factor analysis
Factor analysis using principal axis factoring was conducted the using the package psych
5
through R
3
. Oblique rotation was
used as the latent factors are expected to correlate with each other
6
. Oblimin was selected as the rotation of choice.
Number of factors to retain
Non-graphical solutions to the scree test
7
including parallel analysis, optimal coordinates and acceleration factor was used to
help decide on the appropriate number of factors to retain. The number of factors suggested to retain is 3, 3 and 1
respectively.
The suggested factor solutions were studied but none of them yield any interpretable solution. A more thorough search for an
interpretable solution was conducted for a 4 13 factor structure. A 6 factor solution was eventually adopted as the most
interpretable factor solution yielding the factors Adapting to hearing loss, Support, Education, Healthcare, Policy and
Expectation.
Cross-loadings
Cross-loadings for each item were assessed. A cross-loading difference threshold of less than 0.1 between the 2 highest
factors loading was set as the criteria for removal of items. The solution however had a factor, Expectation, with only 2 items
left after the removal. A decision was then made to move the items to the next factor on which it loaded heaviest on. This
resulted in a new 5 factor solution.
Adjustment of factors to improve interpretability and clinical relevance
Using the new 5 factor solution, changes were further made to it. 6 questions were reassigned to other factors. This was done
to ensure congruence between the meanings of the factors and the questions contained within. This would also improve
interpretability of the factors.
Health and Education
Both the health and education subscales have more than half of the items in the new factors belonging to the old factor. This
implies that these 2 subscales have changed little qualitatively. The new items added to these 2 subscales were most likely due
to the probable attribution to 1 or more of the subscales at the inception of the questionnaire. Factor analysis in this instance
has helped to clarify under which subscale the items would more appropriately belong. Furthermore, the new items are
coherent with the implied meaning of the subscale.
Non-overlapping factors
5 other subscales from the old factor structure have been regrouped into 2 subscales. The reassignments of the subscales help
better define the meaning of each subscale by the new composition of the items and are supported by the confirmatory
analysis.
Conclusion
In conclusion, the PHICE has been revised, reorganised in terms of the subscales composition and the resulting instrument is
deemed to be structurally valid and internally consistent.
References
1. Meinzen-Derr J, Lim LHY, Choo DI, Buyniski S, Wiley S. Pediatric hearing impairment caregiver experience: Impact of
duration of hearing loss on parental stress. International Journal of Pediatric Otorhinolaryngology. 2008;72(11):16931703.
2. Husson F, Josse J. missMDA: Handling missing values with/in multivariate data analysis principal component methods). 2010.
3. R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for
Statistical Computing; 2011. Available at: http://www.R-project.org. Accessed May 1, 2010.
4. Schafer JL. Analysis of Incomplete Multivariate Data. Chapman and Hall/CRC; 1997.
5. Revelle W. psych: Procedures for Psychological, Psychometric, and Personality Research. Evanston, Illinois: Northwestern
University; 2011. Available at: http://personality-project.org/r/psych.manual.pdf.
6. Tabachnick BG, Fidell LS. Using Multivariate Statistics. 5th ed. Allyn & Bacon; 2006.
7. Rache G, Riopel M, Blais J-G. Non Graphical Solutions for the Cattells Scree Test. In: Montral, Canada: Psychometric
Society; 2006.
8. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives.
Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):155.
Confirmatory factor analysis
Confirmatory factor analysis was conducted on the
original 8 factor model, the 5 factor model suggested
in the exploratory factor analysis and the modified 5
factor model. Table 3 lists the various goodness-of-fit
indices proposed by Hu & Bentler
8
for assessing
goodness-of-fit indices.
From the various indices, we can see that the original 5
factor solution is a superior fit for the data and fulfils
the criteria set out by Hu & Bentler for a good fit. The
changes made to the 5 factor model increased the
levels of misfits as indicated by the indices. However,
this is a trade-off that has to be made to improve
interpretability. Furthermore, the change between the
original and the modified 5 factor model is deemed to
be minimal.
The confirmatory factor analysis suggests that after
refactoring and reducing the number of items, the new
factor structure is able to better explain the underlying
phenomenon as expressed by the data.
Internal consistency
Cronbachs was computed for the new subscales
within the 5 factor structure to determine its internal
consistency. Both the value for the original and the
modified 5 factor solution was computed as presented
in Table 4. All subscales had an alpha value of more
than 0.73 suggesting good internal consistency. The
change from the original 5 factor solution to the
modified 5 factor solution is also minimal.
Discussion
The factor structure of the original questionnaire has
been changed significantly from an 8 factor structure
to a 5 factor structure with only 3 overlapping factors.
Given the large reassignment of items, it may be
suggested that the new structure could an artefact of
factor analysis and may not be meaningful in clinical
usage. This discussion would look at the overlapping
and non-overlapping factors (Table 6) and explore the
qualitative changes in the reassignment.
Overlapping factors
The overlapping factors are as shown in Table 6. The
table is formulated after removal of items removed in
the new factor structure from the old factor structure.
This allows for a fairer comparison of the changes
made to the new factor structure.
The number of shared items between these shared items
varies between 2 5 items suggesting that qualitatively, the
factors share some similarity.
Support
The support factor expanded from the original 6 items
to the current 12 items. The original support subscale
consisted of items that suggests as direct need for
support such as 10.
3 questions that were removed due to high
cross loadings were reintroduced. This
reintroduction was a result of feedback from
clinicians that these questions were pertinent
to the care of the patient. While these
questions could be scored separately, a
decision was made to group them under
existing factors based on their content after
examining the change in model fit statistics
as presented in the subsequent portions. The
results that were reassigned and reintroduced
are as presented in Table 1. The factor
solution derived is presented in Table 2.
Inadequate support or understanding
from an employer.. The new subscale
support contains items that are less
direct in nature such as 15. Obtaining
special learning materials for my child
(e.g., books, captioned videos).. A close
look at the various items under the new
subscale reveals this difference. This
suggests that support in the context of
caregiver stress may be homogenous,
whether they are direct or indirect.