1

Qualitative Rigor or Research Validity in Qualitative Research

Journal for Specialists in Pediatric Nursing
Volume 16, Issue 2, pages 151155, April 2011
1. Eileen Thomas PhD, RN1, 2. Joan Kathy Magilvy PhD, RN, FAAN2

Keywords:

Credibility; confirmability; dependability; novice; qualitative rigor; reliability and validity; transferability

Abstract
Column Editor: Lauren Clark Scientific Inquiry provides a forum to facilitate the ongoing process of questioning and evaluating practice, presents informed practice based on available data, and innovates new practices through research and experimental learning. Both novice and experienced qualitative researchers often struggle with the term qualitative rigor. While frequently debated among the community of qualitative scholars, this concept may be one of the most critical aspects of qualitative research. Rigor, in qualitative terms, and reliability/validity, in quantitative terms, are ways to establish trust or confidence in the findings or results of a research study. Rigor is useful for establishing consistency of the study methods over time and provides an accurate representation of the population studied. In other words, rigor provides details as a means to replicate a study with a different research sample. The focus of this article is to address the unique attributes of qualitative research relative to qualitative rigor, which is similar to reliability and validity used in quantitative research. Oxford College Dictionary (2007) defines rigor as the quality of being extremely thorough, exhaustive, or accurate. The term rigor literally means stiffness, from the Latin word rigere to

be stiff, and implies rigidity, harshness, strict precision, unyielding, or inflexible. The term qualitative rigor itself is an oxymoron, considering that qualitative research is a journey of explanation and discovery that does not lend to stiff boundaries. Lincoln and Guba (1985) were the first to address rigor in their model of trustworthiness of qualitative research. We will discuss qualitative rigor and highlight the ways rigor or trustworthiness is typically described among qualitative scholars.

QUANTITATIVE RESEARCH IN CONTRAST TO QUALITATIVE RESEARCH

The first step toward gaining an understanding of rigor is a foundational understanding of the differences between qualitative and quantitative research. The fox knows many things, but the hedgehog knows one big thing, a verse from the seventh-century Greek poet Archilochus, is a metaphor used to provide an accurate but simplistic description of the major differences between quantitative and qualitative research (Eisner, 1998). Referring to the Archilochus metaphor, foxes' (quantitative researchers) attention is on breadth by gathering a variety of information on which to build knowledge, which typically results in generalizable numeric results or outcomes. Breadth, for the purpose of this article, refers to the ability to generalize quantitative results across a large number of cases or subjects. Foxes prefer questions that can be answered in many different ways. Quantitative research and analysis is like the fox: The focus is to gather information quickly from a variety of data points. Foxes tend to cover a lot of ground (generalizability), meaning the research findings or conclusions from a sample population can be extended to apply to the population at large. In contrast, hedgehogs (qualitative researchers) tend to focus their attention on depth by identifying a single phenomenon while burrowing deep. Depth, for the purpose of this article, refers to the ability or need to gain a deep understanding of a specific phenomena/experience with a limited number of participants. Hedgehogs are interested in a holistic, close-up view of many features (variables in quantitative terms) of a single phenomenon. The perspective of qualitative research and analysis is like that of the hedgehog, staying focused on a single spot (phenomena/experience). Qualitative researchers typically collect a lot more information on one topic, phenomena, or experience to enrich their desired understanding. The purpose of qualitative research is not to generalize to other subjects or settings, but to explore deeply a specific phenomenon or experience on which to build further knowledge or to develop a more patientfocused practice that is sensitive to the research participants. Not to negate the value of quantitative research, as both have value and can provide support and balance to a study, the intent of qualitative research is to provide a close-up view, a deeper and richer understanding within a specific context, which can be missed in quantitative research. Researchers may use both quantitative and qualitative methods within a single study to discover something that would have been missed if only a quantitative approach had been used, to use findings from one method to inform the other method, or to expand the breadth and depth of a single study. Researchers may use a qualitative approach for one phase of a study, then a quantitative approach for the second phase of the same study or the reverse. This is called a mixed-method design, where both numerical and text data are collected. Mixed-method research is like conducting two mini-studies within one overall research study. Two types of data are

collected sequentially or concurrently. One type of data provides a basis for collection of another type of data. For instance, some researchers may conduct an experiment (quantitative study) then, after the experiment, conduct an interview or focus group (qualitative) with the participants to see how the participants viewed the experiment or to see if they agreed with the results. One of the authors (Thomas, in press) used qualitative methods to gain an in-depth understanding of women's breast cancer screening behaviors and then used the qualitative data to develop an instrument that will be evaluated using traditional quantitative methods. For a program evaluation, one might develop an instrument that can be used to measure the effectiveness of a parent support intervention in a neonatal intensive care unit by first holding focus groups or interviews to explore the parent perceptions of or meanings inherent in the neonatal intensive care unit experience. Often used to test existing theories or models, with smaller amounts of data collected with a large number of subjects, for example, surveys; the focus of qualitative research is breadth and precision. While qualitative research is focused on depth, richness, and context, which can result in the emergence of a new theory, model, or development of a valid instrument, and a larger amount of data is collected with a smaller number of participants, for example, focus groups or interviews.

QUALITATIVE RIGOR
Lincoln and Guba (1985), in their classic work on naturalistic inquiry, explained the basic question of qualitative research rigor, How can an inquirer persuade his or her audiences (including self) that the findings of an inquiry are worth paying attention to, worth taking account of? (p. 290). Philosophically, we know that a single, generalizable, external truth held and perceived by all would be impossible. Each person has her/his own personal perspective, seen through the lens of cultural, experiential, environmental, and other contextual influences. However, researchers and their audiences, such as nurses in practice settings who hope to build practice on the best evidence, need to have confidence and trust in the research findings presented. Different models are available that address how to build trust in qualitative research, such as the model of trustworthiness of qualitative research proposed by Lincoln and Guba. This model addresses four components of trustworthiness that are relevant to qualitative research: (a) truthvalue (credibility); (b) applicability (transferability); (c) consistency (dependability); and (d) neutrality (confirmability). Credibility Credibility, similar to internal validity in quantitative research, is the element that allows others to recognize the experiences contained within the study through the interpretation of participants' experiences. Achievement of credibility occurs by checking for the representativeness of the data as a whole. To establish credibility, a researcher will review the individual transcripts, looking for similarities within and across study participants. As stated in a classic article by Krefting (1991, p. 218), A qualitative study is considered credible when it presents an accurate description or interpretation of human experience that people who also share the same experience would immediately recognize. Examples of strategies used to establish credibility include reflexivity, member checking, and peer debriefing or peer examination. Member

checking (also known as informant feedback) involves returning to the persons from whom data were generated (a qualitative term for data collection) to ensure that the interpretations (reported as categories and themes) of the researcher are recognized by the participants as accurate representations of their experiences. For example, the researcher may ask all participants involved in the study, or select two or three articulate participants from a focus group session to review the focus group transcripts and interpretations of the focus group data. The researcher will ask peers or consultants experienced in the qualitative analysis process to review and discuss the coding process (Holloway, 1997). In addition, prolonged and varied time spent with the participants, interview techniques, and the transcripts, while writing the final report and using the words of the participants, are strategies used to strengthen the credibility of a study. When quantitative researchers speak of research validity and reliability, they are usually referring to a research that is credible while the credibility of a qualitative research depends on the ability and effort of the researcher (Golafshani, 2003, p. 600). Transferability The ability to transfer research findings or methods from one group to another, or how one determines the extent to which the findings of a particular inquiry have applicability in other contexts or with other subjects/participants, is called transferability in qualitative language, equivalent to external validity in qualitative research (Lincoln & Guba, 1985, p. 290). One strategy to establish transferability is to provide a dense description of the population studied by providing descriptions of demographics and geographic boundaries of the study. The first author (Thomas & Usher, 2009) replicated her 2009 study by using the same data collection methods first with a group of African American women, then with a group of Hispanic women. Recruitment inclusion criteria were the same for both groups, and study findings yielded similar results. Likewise, a nurse caring for families with a child living with autism might read qualitative research studies that might not exactly describe the experience of each family in the caseload but, rather, would give a range of experiences on which to build interventions and understanding. This understanding informs the fit or applicability of the research to practice. Maloni et al. (2010) used a qualitative descriptive approach to explore Bangladesh mothers' perceptions of their children's. A nurse might consider using the same methods with mothers in the United States. Meert, Briller, Schim, Thurston, and Kabel (2009) used qualitative methods (individual interviews and focus groups) to explore the needs of bereaved parents in a Midwestern urban children's hospital pediatric intensive care unit. Other researchers might consider using similar methods at other children's hospitals in different geographic locations to see if results are similar to the results in the Meert et al. study. Clark, Bunik, and Johnson (2010) used participant observation and semi-structured interviews in Colorado and northern New Mexico to explore curanderos' practices related to childhood obesity prevention in Latino families. These researchers found working with the curanderos problematic, first, because it was difficult to locate the number of curanderos eligible to participate in the study and, second, because the researchers perceived that these curanderos may have seen the researchers as representing an opposing and even antagonistic worldview (Clark et al., 2010, p. 10). Repeating the study methods with Latino researchers or research assistants, and/or a different group of curanderos in a different geographic location might yield different results. Dependability

Dependability, related to reliability in quantitative terms, occurs when another researcher can follow the decision trail used by the researcher. An audit trail is achieved by (a) describing the specific purpose of the study; (b) discussing how and why the participants were selected for the study; (c) describing how the data were collected and how long the data collection lasted; (d) explaining how the data were reduced or transformed for analysis; (e) discussing the interpretation and presentation of the research findings; and (f) communicating the specific techniques used to determine the credibility of the data. Strategies used to establish dependability include having peers participate in the analysis process, providing a detailed description of the research methods, or conducting a step-by-step repeat of the study to see if results might be similar or to enhance the original findings (replication is not a term, as a rule, used in qualitative research because like a river, the water is not the same even if one's stance and perspective from the bank is from the same spot). Confirmability Confirmability, similar to objectivity in quantitative terms, occurs when credibility, transferability, and dependability have been established. The qualitative research must be reflective, maintaining a sense of awareness and openness to the study and unfolding results. The term reflexivity, similar to construct validity in quantitative research, requires a self-critical attitude on the part of the researcher about how one's own preconceptions affect the research. Immediately following each individual and focus group interview, the researcher will write or audiotape record field notes regarding personal feelings, biases, and insights. In addition, the researcher should make a conscious effort to follow, rather than lead, the direction of the interviews by asking the participants for clarification of definitions, slang words, and metaphors. Like reflective practice (Johns, 2009), reflective research allows a big picture view with interpretations that produce new insights, allowing for developing confirmability of the research and, overall, leading the reader or consumer of the research to have a sense of trust in the conduct credibility of findings and applicability of the study.

IMPLICATIONS FOR THE NOVICE QUALITATIVE RESEARCHER

Qualitative research is not intended to be scary or beyond the grasp of a novice. Rather, nurses practice elements of qualitative research every day in practice when they use their skills of keen observation, clinical reasoning, patient centeredness, and exploration of how the context and meaning of an experience of care may differ across children, parents, families, nurses, and others. Therefore, to develop competence as a qualitative researcher, a nurse builds upon her/himself as an instrument of the research. We bring all of our past experiences and knowledge into the qualitative research setting but learn, over time, to set aside our own strongly held perceptions and truly listen to the participants/subjects of our research to learn their stories, experiences, and meanings. Qualitative descriptive studies are a good design to begin the qualitative journey. A novice researcher might want to be a coinvestigator with a more experienced qualitative researcher as the techniques described above and in the previous article by the authors (Magilvy & Thomas, 2009) are often learned best in a mentored setting. Having a shared research project not only

promotes mentoring but also expands the understandings gained, as each researcher's perspective is included through the two different lenses. Paying attention to the qualitative rigor and model of trustworthiness from the moment of conceptualization of the research is essential. Researchers who use interviewing often plan for a second interview for each or some of the participants, and write this activity into the proposal. A second interview allows both the participant and the researcher to reflect on the original conversation, fills in missing pieces or new information, and provides assurance that the participant's words and experiences were accurately described. A different setting for the second interview also may expand the description. For example, an interview conducted with an adolescent during hospitalization about the experience of recovering from a major surgical procedure may be different from the interview conducted later in the home. While encouraging examination for accuracy of the original interview text, the adolescent may reflect on the hospital experience more holistically after returning to the home and school environment, and may add or fill in missing pieces. The researcher may also review some emerging findings from other participants, which, while not this participant's direct experience, the participant may recognize this experience as plausible for people like themselves. This type of member checking enhances the credibility of a study. Novice and experienced qualitative researchers will agree that qualitative research is an experience of discovery and understanding that transcends one's own experience and enriches the practice experience and evaluations of the quality of care. Attending to the rigor of qualitative research is an essential part of the qualitative research journey and provides an opportunity for critique and further development of the science.

References

Clark, L., Bunik, M., & Johnson, S. L. (2010). Research opportunities with curanderos to address childhood overweight in Latino families. Qualitative Health Research, 20(1), 4 14. doi:10.1177/1049732309355285 o CrossRef,
o o

PubMed, Web of Science

Go here for SFX

Eisner, E. W. (1998). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. Upper Saddle River, NJ: Merrill. Go here for SFX Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597607. Retrieved from http://www.nova.edu/ssss/QR/QR84/golafshani.pdfGo here for SFX Holloway, I. (1997). Basic concepts for qualitative research. Oxford, UK: Blackwell Publications. Go here for SFX

Johns, C. (2009). Becoming a reflective practitioner (3rd ed.). San Francisco, CA: Wiley. Go here for SFX Krefting, L. (1991). Rigor in qualitative research: The assessment of trustworthiness. American Journal of Occupational Therapy, 45(3), 214222.
o o o

PubMed, ChemPort, Web of Science Times Cited: 201

Go here for SFX

Lincoln, Y., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. Go here for SFX Magilvy, J. K., & Thomas, E. (2009). A first qualitative project: Qualitative descriptive design for novice researchers. Journal for Specialists in Pediatric Nursing, 14(4), 298 300. doi:10.1111/j.1744-6155.2009.00212.x Direct Link:
o o o o

Abstract Full Article (HTML) PDF(48K) References

Go here for SFX

Maloni, P. K., Despres, E. R., Habbous, J., Primmer, A. R., Slatten, J. B., Gibson, B. E., & Landry, M. D. (2010). Perceptions of disability among mothers of children with disability in Bangladesh: Implications for rehabilitation service delivery. Disability and Rehabilitation, 32(10), 845854. doi:10.3109/096382809033260633 o CrossRef,
o o

PubMed, Web of Science Times Cited: 1

Go here for SFX

Meert, K. L., Briller, S. H., Schim, S. M., Thurston, C., & Kabel, A. (2009). Examining the needs of bereaved parents in the pediatric intensive care unit: A qualitative study. Death Studies, 33(8), 712740. doi:10.1080/07481180903070434 o CrossRef,

8
o o

PubMed, Web of Science Times Cited: 1

Go here for SFX

Oxford College Dictionary. (2007). Rigor (2nd ed., p. 1174). New York: Spark. Go here for SFX Thomas, E. (in press). From qualitative data to the women's breast conflict scale: A predictive instrument. The Qualitative Report.
o

PubMed

Go here for SFX

Thomas, E., & Usher, L. (2009). One more hurdle to increasing mammography screening: Pubescent, adolescent, and prior mammography screening experiences. Women's Health Issues, 19(6), 425433. doi:10.1080/07481180903070434 o CrossRef,
o o

PubMed, Web of Science

Go here for SFX