Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Edited by
Vandana Desai and Robert B. Potter
SAGE Publications
London ● Thousand Oaks ● New Delhi
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 115
13
Quantitative, Qualitative or Participatory?
Which Method, for What and When?
Linda Mayoux
understanding of complex realities and process may involve small focus groups, larger
processes where even the questions and hypo- participatory workshops or individual diaries
theses emerge cumulatively as the investigation and diagrams which are then collated into a
progresses. The possibility of ‘objectivity’ is plenary discussion. Participation (and hence
questioned and instead the aim is to under- sampling) may be open or carefully targeted
stand differing and often competing ‘subjec- to particular social groups. Larger meetings
tivities’ in terms of very different accounts may be subdivided into what are assumed to
of ‘facts’, different meanings and different be more ‘homogeneous groups’ or groups
perceptions. with complementary information.
Qualitative research typically focuses on Participatory research typically uses and
compiling a selection of micro-level case stud- adapts diagram tools from farmer-led research,
ies which are investigated using a combination systems analysis and also oral and visual tools
of informal interviews, participant observation from anthropology, though many commonly
and more recently visual media like photogra- used tools have also been developed by
phy and video. Questions are broad and open- NGOs and participants in the field.The use of
ended, and change and develop over time to fill diagram and oral tools makes both discussion
in a ‘jigsaw’ of differing accounts of ‘reality’, and analysis accessible to non-literate partici-
unravelling which may be said to be generally pants and across language groups. Through
‘true’ and which are specific and subjective, sharing their different sources of information,
and why. Different sampling methods are com- participants themselves may increase their
bined depending on the particular dimension understanding of development issues and the
of the issue being considered: different purpo- problems they face, and develop solutions, as
sive sampling techniques, identification of key well as giving more reliable and representative
118
informants who possess the particular know- information to researchers. In some cases
ledge sought and also ‘random encounters’ to local people themselves conduct research fol-
cross-check information and/or highlight yet lowing an initial design of specific tools and
more differing perspectives on the problem. training. Some recent NGO innovations pro-
Causality and attribution are directly investi- pose doing this on a large scale (Mayoux
gated through questioning as well as the quali- et al., 2005).
tative analysis of data. Computer programs
may be used to deal systematically with large
amounts of data.5 Qualitative research typically Which method for what?
requires the long-term immersion of a skilled Relevance, reliability and ethics
researcher in the field who engages in a reflex-
ive process of data collection and analsysis. In recent years, however, these ‘different
Participatory methods have their origins worlds’ have begun to merge.7 There is an
in development activism: non-governmental increasing awareness of the extreme complex-
organizations (NGOs) and social movements.6 ity and political contingency of even defini-
Here the main aim is not so much knowledge tions of many development problems, notably
per se, but social change and empowerment – poverty and empowerment but also health/
and this wherever possible as a direct result of well-being and literacy/knowledge, and hence
the research process itself. In particular, it also the inevitable complexity of the informa-
seeks to investigate and give voice to those tion needed to find solutions. Interdisciplinary
groups in society who are most vulnerable courses in development studies have become
and marginalized in development decision- more common. Development agencies com-
making and implementation.The participatory monly require multidisciplinary teams. This
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 119
has led to constructive cross-fertilization of hand, the very openness and flexibility of
tools and more integrated methodologies qualitative methods create problems of a lack
to build on the complementarities between of focus.The participatory solution of relying
different methods. on local priorities raises the question of ‘whose
local priorities?’ as these are often diverse
Traditional disciplinary boundaries are in any and open to manipulation by powerful vested
case far less rigid than the above account interests. Balancing inherent tensions between
would indicate: specificity and flexibility and competing views
from the field will inevitably be a difficult task.
• Quantitative information can be obtained The second challenge is translating these
through participatory and/or qualitative
broad questions into a workable research
methods.8
frame. In all types of research, considerable
• Qualitative information can be obtained
thought must be given to the potential effects
as part of quantitative surveys and/or
of precisely which questions are asked, how,
using participatory methods.
from and by whom.10 Even (or perhaps par-
• Participatory diagram tools can be used
with individual respondents to get both ticularly) in quantitative research, assumptions
qualitative and quantitative information, of ‘objectivity’ must be continuously ques-
and participatory meetings can be used as tioned.11 It should not be assumed that ‘more
a forum for conducting these.9 information is necessarily better information’.
Very long interview schedules may give very
Moreover, new tools and new solutions to inaccurate responses because of both respon-
shortcomings of old tools are continually dent and interviewer fatigue. Very precise
being developed. questions may not give precise answers
119
What is now emerging is a much more because people simply do not know or recall,
nuanced approach. Strengths and weaknesses or they are suspicious of the amount of detail
are not always absolute, but depend very required. Conducting ill-thought-out ques-
much on how particular methods and tools tionnaires, even for large samples, will not
are used and whether they are used ‘well’ or improve the reliability of the information.
‘badly’. Unfortunately also, what may be Any statistical analysis based on these will also
potential strengths in relation to one context be potentially misleading.
or purpose, may be potential weaknesses in In deciding when and how to use quanti-
others. A summary of the main potential tative methods, the critical questions will be:
pros and cons of the different methods is
1 For which questions is quantification
attempted in Table 13.2, although many of
needed, and with what degree of preci-
these are inevitably contentious.What follows
sion? For which questions is qualitative
focuses mainly on comparing the ‘cons’ and
information more useful or sufficient?
the ways in which they might be addressed.
2 For how many people is information
The first key area of debate and contention required to draw reliable practical conclu-
concerns identification of issues to be investi- sions? Which particular people are most
gated and definitions of development prob- important for the analysis or are likely to
lems. On the one hand, the specificity and be able to give reliable information?
clarity of the questions and hypotheses that are
required for rigorous quantitative research In many cases, particularly in small student
lay it open to challenges of ‘measuring the research projects, high levels of precision will
irrelevant’ and/or ‘trying to measure the not be either possible or necessary. More
immeasurable’ (Hulme, 2000). On the other in-depth discussions from a smaller number of
13-Desai-3367.qxd
Table 13.2 Summary pros and cons of different methods
2/14/2006
Quantitative methods Qualitative methods Participatory methods
Relevance Clear focus on External and a priori Holistic May lack focus Based on local May be too
7:22 PM
specific focus may miss relevant understanding of Still filtered by perceptions and context-specific
questions and questions and issues complex issues subjective external priorities May be
hypotheses Not everything can be and processes analysis over-influenced by
measured meaningfully Flexibility and power relations
Page 120
cumulative
understanding
Captures
underlying
meanings, the
unexpected and
sensitive issues
Representation Random In practice the sample Captures different Small-scale, open Can rapidly collect May be
and freedom samples actually interviewed is local perceptions to bias large amounts of over-influenced by
from bias decrease often non-random data power relations
likelihood of Choice of control group Purposive sampling Generalizability of Captures diversity Difficult to control
bias presumes the relevant enables close findings are often who attends
Careful targeting
variables are already focus on cases and difficult to prove and collective Participation in
known issues of interest discussion discussion depends
May under-represent increases the voice on the skill of the
minorities in aggregate of the most facilitator
conclusions vulnerable
(Continued)
13-Desai-3367.qxd
Table 13.2 (Continued)
2/14/2006
Pros Cons Pros Cons Pros Cons
Reliability of Objectivity of Objectivity may be only Captures May be Collective Depends on who
information measurement apparent subjectivity over-influenced by discussion enables participates
Depends on precisely In-depth the biases of the more reasoned Depends on the
7:22 PM
which questions are longitudinal researcher responses and skill and
asked, how and by investigation immediate understanding of
whom decreases the cross-checking of the facilitator
likelihood of different accounts
Page 121
Problems of respondent Difficult to sift all
motivation and falsification Diagram captures the information and
falsification non-linear complexity record it at the time
of the exercise
Credibility of Objectivity of Difficult to ‘prove’ Cumulative Difficult to ‘prove Collective and Difficult to
analysis analysis causality ‘jigsaw’ anything’ beyond immediate analysis aggregate
Depends very much Good at uncovering anecdotes of information May be over-
on the relevance of processes and May be enables immediate influenced by power
the questions and causality over-influenced by cross-checking of relations and
hypotheses and the the biases of the different accounts expectations
reliability of the researcher Diagrams can Diagrams may be
information represent a clear difficult to interpret
analysis of
complex issues It is often the
process rather than
the diagram product
which is key
(Continued)
13-Desai-3367.qxd
2/14/2006
7:22 PM
Table 13.2 (Continued)
Page 122
Pros Cons Pros Cons Pros Cons
Ethics Collection of Direct benefits for Empathy and Aim at Concerns about Empowerment may
‘hard data’ to respondents are understanding non-interference empowerment are be assumed rather
convince policy generally not integral to the than actual
May give people a Even the presence
makers considered process
chance to discuss of an investigator May raise
The very questions and things they have has an impact People learn from unrealistic
ways in which questions never been able to each other and expectations
are asked has an tell anyone before reach new
May make some
impact understandings
people more
vulnerable
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 123
people will get more reliable information. them develop ways forward. Moreover, the
These people may also be able to give a rough very act of organizing group discussions may
estimate of how general their responses are, raise unrealistic expectations. People may be
and these estimates from different people can made even more vulnerable if they express
then be compared to arrive at an approximate their views and problems publicly. Both these
figure. More time- and energy-consuming factors may lead to unreliable information
tasks of questioning a larger number of people being obtained as well as undesirable conse-
can then be reserved for those specific ques- quences for all concerned. In participatory
tions where it is really needed. research, as much as other forms of research,
Qualitative and participatory researchers benefits for participants need to be strategi-
have more often been consciously aware of cally planned rather than assumed (Johnson
these issues, and in some cases treated them as and Mayoux, 1998). In all types of research,
a subject of investigation in themselves. researchers should consider the types of
Nevertheless even these researchers have organization, network and support agency
often been less than open about the subjectiv- which participants might approach in order to
ity of their analysis, and their role in influen- take forward issues raised by the research.
cing the outcomes of research processes. It
is crucial that whatever methods are used,
researchers engage in in-depth reflection and Which method when? An
acknowledgement of their own biases, and integrated research process
seek ways of countering these which take
account of both their privileged ‘overview’ There are therefore no easy answers. All
knowledge and the diversity of local views research, even that with claims to ‘objective
123
and perceptions.12 truth’, is inevitably fraught with complexity
All researchers are at least to some extent and sources of bias and error. Even the sub-
dependent on the enthusiasm and coopera- jective biases in analysing competing subjec-
tion of respondents in conducting their tivities need to be examined. Ethical concerns
research. Ethical issues are therefore practi- must always be an integral and planned ele-
cally as well as morally central to the research ment in research design rather than assumed
process. In quantitative and qualitative outcomes.
research, as well as participatory research, con- Wherever possible, most research will use
siderable thought needs to be given to how an integrated methodology. Using comple-
questionnaires and qualitative investigation mentary methods enables researchers to
can be designed to benefit respondents: par- consolidate strengths, and cross-check and
ticularly how to facilitate a progression of triangulate any information which is central to
thinking through issues important to them, the particular research questions concerned.
sharing information to which the researcher An integrated methodology also helps to dis-
has had access, and the dissemination of the seminate information in different ways for dif-
findings back to those who contributed to the ferent audiences to ensure, as far as possible,
research.13 beneficial outcomes for the participants.
Participatory research has commonly Figure 13.1 gives an outline of a possible
claimed the ‘empowerment high ground’. research process, the questions which each
However, participatory research may merely stage must seek to address and the possible
extract what everybody already knows for the ways in which different methods can be integ-
benefits of the researcher, rather than generat- rated. As can be seen, participatory methods
ing new knowledge for participants or helping play a central role at all stages from conception,
13-Desai-3367.qxd
KEY : METHODS
PARTICIPATORY
2/14/2006
KEY KEY QUALITATIVE
OUTPUTS TASKS
QUANTITATIVE
AT EACH AT EACH
STAGE STAGE
7:22 PM
participant
STAGE 1: observation
SCOPING exploratory interviews
main questions/ literature review with possible key
brainstorming
issues and gathering of informants
exploratory modelling
Page 124
key stakeholders: different information initial contacts with respondents
beneficiaries, and hypotheses institutional links
participants,
informants,
audience exploratory
broad hypotheses participatory
... focus groups and
workshops
focused use of
STAGE 2: contextual information
qualitative methods:
REFINEMENT identification of differing views
purposive observation,
AND PILOTING and perspectives
more interviews
focusing questions who may know or say what? participatory
with key informants
or how do different people respond stakeholder
and purposive sample
hypotheses to different questions and tools? analysis
sampling frame how do initial observed processes participatory
process and tools relate to original models and modelling
to be used hypotheses piloting
participatory tools
revisiting literature
critical examination of
existing statistics
(Continued)
13-Desai-3367.qxd
KEY KEY
OUTPUTS TASKS
AT EACH AT EACH
STAGE STAGE
2/14/2006
cross-checking
carefully designed and
triangulation
targeted participatory
interim reports
STAGE 3: tools to get quantitative
and analysis
’RESEARCH and qualitative
anticipated or quantitative
PROPER’ information
7:22 PM
unexpected findings survey for
main investigation
new questions specific
and analysis
and hypotheses? measurable
indicators
with random or
Page 125
stratified sample
statistical analysis
ongoing qualitative
observation and interviews
STAGE 4:
qualitative follow-up of
DISSEMINATION
interesting case studies,
AND LINKS
outliers and/or typical cases
final write-ups and
dissemination for
different audiences
consolidating links
discussion of findings with key informants/purposive
between respondents
sample
and relevant
gauging initial reactions to findings and presentation of
development
findings
agencies
further participatory questionning of hypotheses
steps for the future
further participatory identification of ways forward
clarification of gaps and limitations
through piloting and refinement to the and for some purposes, quantitative surveys
research proper and then finally dissemination. may also be a very useful introduction and
It is the view of the author that using par- pretext for contacting a range of different
ticipatory methods as the ‘first port of call’ has people. People may be more familiar with the
many advantages in terms of rapidity and reli- idea of questionnaires, providing a structure
ability of collecting many types of qualitative which can then be an opportunity for quali-
as well as quantitative information, manage- tative methods like participant observation
ability in terms of time and resources, and also and unstructured conversation. This ‘random
its potential for contributing to the develop- exposure’ may also then provide a good basis
ment process. In this the author must confess on which to design a more rigorous parti-
to be highly influenced by her early work in cipatory process.
the late 1970s to late 1980s doing long-term, The research process suggested is therefore
in-depth qualitative and quantitative field- more of a ‘rough guide to be adapted or thrown
work in India, based on anthropology but also to circumstance’ than a rigid prescription.
working with economists. Subsequent experi- Particular care must be taken when using par-
ence of using and developing participatory ticipatory methods in conflict or highly polit-
methods have shown many ways in which ically charged environments and/or where
these could have yielded far more reliable the prime intended beneficiaries of the
information far more quickly. Participatory research are very vulnerable. Examples of the
methods are not necessarily a substitute for latter include, for example, research on child
other methods, but enable much more cost- labour, domestic violence and labour relations
effective targeting of everyone’s time and if it is difficult for victims to meet without the
energy on those areas of the research for knowledge of those with power over them.At
126
which participatory methods are rather more the same time participatory methods can be a
problematic.They are also generally an essen- very valuable means of helping resolve misun-
tial component of research dissemination to derstandings and conflict, and identify realistic
those participating in the research, a stage ways forward. Any research on such issues,
which is commonly ignored and omitted. participatory or otherwise, should also as far as
The research process suggested is applica- possible liaise with organizations which can
ble in many, and probably most, research con- follow up on at least serious cases of abuse
texts, both in individual student research and identified.
in multidisciplinary research teams. It will
nevertheless be important to adapt both Notes
process, and particularly specific tools, to the
types of question, the context, the skills, 1. In this chapter the term ‘approach’ is used to
understanding and motivation of respondents refer to underlying philosophies, goals and dis-
and the skills and resources available to the ciplines, the term ‘tools’ to the practical and
researcher. In particular, the relative emphasis specified ‘best practice’ ways in which informa-
on participatory, qualitative and quantitative tion is obtained and/or analysed,‘techniques’ to
methods in the main research stage will ways in which tools may be adapted or applied
to particular circumstances or challenges, and
depend very much on the outcomes of the
the term ‘method’ is used as a generic term
prior two stages. In some contexts also it may
encompassing all three.
not be advisable to do any participatory 2. For overviews of claims and counter-claims see
research before in-depth qualitative research Hulme (2000) and Kanbur (2003).
has overcome sensitivities and/or established 3. For a useful short summary of the issues and
networks which could be used as the basis for pitfalls of statistics in development studies,
bringing people together. In some contexts see Mukherjee and Wuyts (1998). For a more
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 127
detailed discussion and introduction to the 7. See, for example, the debates between
methods, see Mukherjee et al. (1988). For easily eminent development researchers at the 2001
accessible overviews of the strengths and pitfalls ‘Qual-Quant’ workshop held at Cornell
of different statistical techniques, see the website University (Kanbur, 2003).
for Statsoft:http://www.statsoft.com/textbook/ 8. For a discussion of quantification and partici-
stathome.html. For access to many further patory methods, see Mayoux and Chambers
resources, see the quantitative methods, statis- (2005).
tics and quantitative database sections on the 9. For suggestions on how some recent particip-
University of Amsterdam’s SocioSite: http:// atory tools can be used for quantitative and
www2.fmg.uva.nl/sociosite/topics/research. qualitative investigation, see Mayoux (2003a).
html 10. For a critical overview of research on women’s
4. For an overview of qualitative methods, see empowerment in micro-finance, see Kabeer
Denzin and Lincoln (1994) and relevant chap- (2001).For an overview of issues and questions
ters in this volume. For access to many further in relation to different methods, see Mayoux
resources, see the qualitative methods sections (2002).
on the University of Amsterdam’s SocioSite: 11. For a convincing gender critique of inherent
http://www2.fmg.uva.nl/sociosite/topics/rese biases in quantitative research on poverty,
arch.html. social cost–benefit analysis and population
5. For detailed discussion and links to trial ver- policy, see Kabeer (1994).
sions of the different computer-aided qualita- 12. For a very thoughtful self-questioning com-
tive data analysis (CAQDAS) packages, see bination of quantitative and qualitative research
Lewins and Silver (2004). on changing gender relations, see Kabeer
6. For a detailed introduction and overview (2000).
of participatory methods, see Chambers 13. See, for example, Mayoux (2003b) and
(1994). Chapter 3 on ethics in this volume.
127
1. What are the questions and issues which the research is designed to address? Whose
questions and issues are they? What are the underlying assumptions and biases?
2. For which questions is quantification needed, and with what degree of precision? For
which questions is qualitative information more useful or sufficient?
3. For how many people is information required to draw reliable practical conclusions? Which
particular people are most important for the analysis or likely to be able to give reliable
information? Is information likely to be more reliable in private or in a group discussion?
4. What are the implications of the answers to these questions for the conclusions which
can be drawn?
Ethics check
5. Does the research process help build the capacity, skills and learning of those
‘researched’ and increase their understanding?
6. What are likely to be the practical consequences? Who will use the information
generated and how? Are the most disadvantaged and vulnerable participants
adequately represented and protected at all stages?
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 128
Websites
The Social Science Resource website of the University of Amsterdam has lists of resources and
other web links for different types of research method: http://www2.fmg.uva.nl/sociosite/ topics/
research.html
Discussion of quantitative methods, including use of qualitative and participatory methods for quan-
tification, can be found at: www.ssc.reading.ac.uk
The Statsoft website has comprehensive materials on the strengths and weaknesses of different
tools for statistical analysis: http://www.statsoft.com/textbook/stathome.html
The Forum for Qualitative Research website brings together resources and debates in English and
other European languages: http://www.qualitative-research.net/fqs/fqs-eng.htm
Chronic Poverty Research Centre has links to research methodologies and a toolbox: http://www.
devinit.org/CPRC/CPall/www_cprc/www_cprc/default.html
The Monitoring and Evaluation (MandE) website has detailed practical discussions on the use of
different qualitative, quantitative and participatory tools in development agencies: http://www.
mande.co.uk
Chambers, R. (1994) The origins and practice of participatory rural appraisal, World Development,
22(7): 953–969.
128 Chataway, J. and Joffe, A. (1998) Communicating results, in A. Thomas, J. Chataway and M. Wuyts
(eds), Finding Out Fast: Investigative Skills for Policy and Development (pp. 221–236), London:
Sage.
Denzin, N.K. and Lincoln, Y.S. (1994) Handbook of Qualitative Research, Thousand Oaks: Sage.
Helberg, C. (1995) Pitfalls of data analysis (or how to avoid lies and damned lies). Paper presented
to workshop at the Third International Applied Statistics in Industry Conference, Dallas. Available
at: http://my.execpc.com?#helberg/pitfalls/
Hulme, D. (2000) Impact assessment methodologies for microfinance: theory, experience and
better practice. World Development, 28(1): 79–88.
Johnson, H. and Mayoux, L. (1998) Investigation as empowerment: using participatory methods, in
A. Thomas, J. Chataway and M. Wuyts (eds), Finding Out Fast: Investigative Skills for Policy and
Development (pp. 147–172), London: Sage, Open University.
Kabeer, N. (1994) Reversed Realities: Gender Hierarchies in Development Thought, London: Verso.
Kabeer, N. (2000) The Power to Choose: Bangladeshi Women and Labour Market Decisions in
London and Dhaka, London: Verso.
Kabeer, N. (2001) Conflicts over credit: re-evaluating the empowerment potential of loans to women
in rural Bangladesh, World Development, 29(1): 63–84.
Kanbur, R. (ed.) (2003) Q-squared: qualitative and quantitative methods of poverty appraisal,
New Delhi: Permanent Black.
Lewins, A. and Silver, C. (2004) Choosing a CAQDAS (Computer-Aided Qualitative Data Analysis)
Package: a working paper. Available at: http://caqdas.soc.surrey.ac.uk/Choosing%20a%20
CAQDAS%20package%20–%20LewinsandSilver.pdf
13-Desai-3367.qxd 2/14/2006 7:22 PM Page 129
Mayoux, L. (2001) Whom do we talk to? Issues in sampling, EDIAIS. Available at: http://www.
enterprise-impact.org.uk/informationresources/toolbox/sampling.shtml
Mayoux, L. (2002) What do we do with the information? From practical conclusions to influencing
change, EDIAIS. Available at: http://www.enterprise-impact.org.uk/informationresources/toolbox/
whatdowedo.shtml
Mayoux, L. (2003a) Thinking it through: using diagrams in impact assessment: EDIAIS. Available at:
http://www. impact.org.uk/informationresources/toolbox/thinkingitthrough-usingdiagramsinIA.shtml
Mayoux, L. (2003b) Empowering enquiry: a new approach to investigation, EDIAIS. Available at:
http://www.enterpriseimpact.org.uk/informationresources/toolbox/empoweringenquiry.shtml
Mayoux, L., Andharia, J., Hardikar, N., Thacker, S. and Dand, S. (2005) Participatory action learn-
ing in practice: experience of Anandi, India, Journal of International Development, 17: 211–242.
Mayoux, L. and Chambers, R. (2005) Reversing the paradigm: quantification and participatory
methods and pro-poor growth, Journal of International Development, 17: 271–298.
Mukherjee, C. and Wuyts, M. (1998) Thinking with quantitative data, in A. Thomas, J. Chataway and
M. Wuyts (eds), Finding Out Fast: Investigative Skills for Policy and Development (pp. 237–260),
London: Sage.
Mukherjee, C., White, H. and Wuyts, M. (1988) Econometrics and Data Analysis for Developing
Countries, London: Routledge.
Thomas, A. (1998) Challenging cases, in A. Thomas, J. Chataway and M. Wuyts (eds), Finding Out
Fast: Investigative Skills for Policy and Development (pp. 307–332), London: Sage.
Thomas, A., Chataway, J. and Wuyts, M. (1998) Finding Out Fast: Investigative Skills for Policy and
Development, London: Sage and The Open University.
129