Critical appraisal for speech and language therapists: questionnaire studies

Cross-sectional surveys using questionnaires. A representative sample of patients, practitioners, carers etc. are invited to respond to a questionnaire in order to gain information on a clinical question, such as, What types of community support do people with condition X receive? or What influences speech and language therapy decision-making for ? or How accessible are speech and language therapy services for people in geographical area X? The data for surveys are collected at a single time point in time but may refer to retrospective information, such as asking whether an adult received speech and language therapy services during childhood. NB There is also a separate tool for observational studies in general, i.e. studies using cohort study, case-controlled study and cross-sectional survey methods. Use the following questions to appraise a report of questionnaire research with respect to the study results, their validity and whether they will help with clinical practice or decision-making. There is space for you to add your own notes. The prompts are there to highlight why an issue is important but they shouldnt limit your thinking. There is quite a lot of overlap between questions.

Study reference:

What was the research question? Is a questionnaire survey an appropriate way of answering the question(s) under the circumstances?

Aims

Was the questionnaire used VALID (measures what it says it measures) and RELIABLE (gives consistent results from repeated use)? If an off the peg questionnaire (i.e. a previously published and validated one) was available, did the researchers use it, and, if not, why not?

Measurement

Adapted from Trisha Greenhalgh (2006) How to read a paper: the basics of evidence-based medicine, 3 edition, Oxford: Blackwell Publishing Ltd. by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal club 5: observational studies, Speech & Language Therapy in Practice Autumn, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

rd

What did the questionnaire look like? Check quality of both format and content.

Was it appropriate for its target population?

Appropriateness

Were the instructions clear? Check they included explanations of purpose, confidentiality, source as well as how to complete.

Instructions

Was the questionnaire adequately piloted? Characteristics of piloting sample How was it piloted? What changes were made as a result of piloting?

Testing

Adapted from Trisha Greenhalgh (2006) How to read a paper: the basics of evidence-based medicine, 3 edition, Oxford: Blackwell Publishing Ltd. by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal club 5: observational studies, Speech & Language Therapy in Practice Autumn, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

rd

What was the sample? (opportunity, random, stratified, quota, snowball) Look for selection bias which might compromise validity of the findings.

Sampling

How was the questionnaire administered? Have they considered ways to increase response rate?

Method

Was the response rate adequate? Check characteristics of those who were invited but did not respond could there be systematic differences between respondents and non-respondents (source of bias in results)?

How were the data analysed? Appropriate descriptive and/or inferential stats for quantitative responses? Recognised method of qualitative analysis for open-ended responses?

Results
What are the results of this study? In a nutshell

Adapted from Trisha Greenhalgh (2006) How to read a paper: the basics of evidence-based medicine, 3 edition, Oxford: Blackwell Publishing Ltd. by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal club 5: observational studies, Speech & Language Therapy in Practice Autumn, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.

rd

Key conclusions and do you believe them? Consider: What do the results actually mean? Have they drawn an appropriate link between the data and their conclusions? Findings placed in context (wider body of knowledge)? Acknowledgement of study limitations?

Findings

Does this study have implications for my practice for my colleagues or care group for the service as a whole? Is there a further question to be asked? NB One questionnaire study is unlikely to provide sufficiently robust evidence to recommend changes to clinical practice or decision-making. However, for certain questions, good quality surveys may provide some preliminary evidence. You should be cautious about generalising from a small, local study to a national context. Recommendations from questionnaire studies are always stronger when supported by other evidence.

Impact Summary

Adapted from Trisha Greenhalgh (2006) How to read a paper: the basics of evidence-based medicine, 3 edition, Oxford: Blackwell Publishing Ltd. by Jennifer Reid, Speech and Language Therapy Service, NHS Fife. Cartoons by Fran commissioned by Speech and Language Therapy in Practice. Accompanies Reid, J. (2011) Journal club 5: observational studies, Speech & Language Therapy in Practice Autumn, pp.18-21. Please attribute these sources if you use, distribute or adapt any of this material.
rd