---------------------------------------------------------

--------------
What do you understand by policy
evaluation? And to what extent are
policies evaluated in most countries?
___________________________________________

Term paper presented to

Mr. E.C. Kujeke
Calvary University

---------------------------------------------------------
--------------
Submitted in partial fulfillment of the
requirements for the course of
Public Policy
---------------------------------------------------------
--------------

Presented By
Davies Chiriwo
536 Bath Road,
Greenhill,
Bindura.
---------------------------------------------------------
-------------

Due date
25-07-2009

1
---------------------------------------------------------
--------------
Master of
Business Leadership Degree
---------------------------------------------------------
--------------

Contents page

Title page

Contents page

Introduction

Understanding policy evaluation

Extent of policy evaluation in most countries

Conclusion

10

2
References

12

Introduction

According to Dye (1987) public policy is whatever governments choose

to do or not to do. He goes on to state that these public policies

regulate conflict within society, organize society to carry on conflict

with other societies, distributes a variety of symbolic rewards and

material services to members of society and extract money from

society. Since these policies regulate society in all its engagements

with or within its boundaries, it is imperative for those governing these

societies to fully engage in ways that real bring out the best out for

3
their societies they represent by putting up the best monitoring and

evaluation mechanisms that can reveal inadequacies or strengths that

policies they put forward have on the lives of their electorate or

populace.

This allows for the evaluation of the impact of policy formulation

processes have on their ownership and outcomes. Thus this paper will

bring out what we mean by policy evaluation, the importance of policy

evaluation, what it involves, who the main players are in policy

evaluation as well as highlight to what extent policies are evaluated in

this country and the world at large.

Understanding by policy evaluation

4
According to Dye (1987: 351) policy evaluation refers to the’

assessment of the overall effectiveness of a national program in

meeting its objectives, or assessment of the relative effectiveness of

two or more programs in meeting common objectives.’ (Kapfunde

2004:173) views policy evaluation as the ‘systematic measurement of

performance in terms of specific policies, guidelines and procedures.’

Policy evaluation is an essential part of the policy processes which

largely looks at the effectiveness and efficacy of promulgated and

implemented policies in any system of governance or organisation.

Policies are enunciated usually to address certain pertinent issues that

relate to how a nation or organisation interacts with its environment.

Dye (1987) also envisages policy evaluation as the assessment of the

consequences of / or the impact of public policy on the real world

conditions which cover the following five aspects: impact on the target

situation or group; impact on situations or groups other than the

target; impact on future as well as immediate conditions; direct costs

in terms of resources dedicated to the program; and indirect costs,

including loss of opportunities to do other things. Policy evaluation is

conducted for checking the effects of the policies of respective

governance institutions in terms of necessity, efficiency, validity, etc.

to improve the planning and implementation process. The

development of effective institutions and tools for evaluation is

5
essential to adequately view the effectiveness of policies employed by

governments and organisations across the global divide.

Evaluation identifies the progress and gaps in development thus

influencing heavily on future policy formulation and implementation

strategies. It also identifies individuals, agencies and organizations

responsible for various activities and duties thus providing information

on how well institutions of governance across nations are meeting their

mandates or obligations. According to Kapfunde (2004) the objectives

of conducting policy evaluations are the following: to realise efficient

and high quality public administration focusing on the citizens; shift to

performance-based public administration from the viewpoint of the

citizens; and thorough accountability to the citizens. Reliable and valid

data and statistics are critical components of any monitoring and

evaluation system.

Evaluation at the national level, according to the Economic

Commission for Africa (2004) should consist of five fundamental things:

- Benchmark indicators;

- Systematic surveys;

- Impact analysis;

- Dissemination of results; and

- Identification of responsibilities.

6
A thorough approach would consist of developing a comprehensive

statistical structure with benchmark surveys; systems for data

collection through censuses and surveys; provisions for gender-

sensitive and disaggregated data on the basis of regional and

geographical differences, socio-economic groups and ethnic

disparities; timely and reliable means for updating the system

regularly; and systematic dissemination of information. Benchmark

surveys and indicators should be understood, developed and

interpreted with regard to local conditions. Data and indicators are

tools for making policy better; they are not an objective in themselves.

The accessibility, cost and reliability of data are central criteria in

selecting an indicator. Often, finding reliable and accessible data on

problems related to services from citizen is difficult if they are

excluded from the process, and other cost-effective methods to find

data that involve citizens should be considered (e.g., participatory

process).

Even though all elements of this thorough approach have not been

introduced in many countries Zimbabwe included, some countries such

as Lesotho, Malawi and Senegal have been able to evaluate its

capacity development policies in areas of poverty assessment and

monitoring through strengthening national statistical offices (Economic

Commission for Africa 2004). The system of policy evaluation entails

7
the measuring compliance of set targets which are inherent in the each

individual policy statements. These systems should also bear a high

level of congruency to the political, socio-economic situations on the

ground which has direct effect to the general populace within a

nation’s boundaries. The important measures of success or failure and

or effectiveness and efficacy of policy should be part of the policy

itself. Every policy statement should contain in it benchmarks or

targets it is supposed to address or correct stated unambiguously.

The evaluation of policy should be such that the greatest say emanates

from those constituencies that are constantly in touch with various

policies under scrutiny or categorically all policy that are employed as

part of their day-to-day governing machinery. This implies that the

monitoring and evaluation machinery should be largely decentralised

in order to get the correct sentiments and effects of issues or problems

policies purport to address. The evaluation of policy can largely be

viewed from a layman’s perspective if they address problems that the

common man can actually really see.

Extent of policy evaluation in most countries

(Hogwood and Gunn 1984: 220) states that,’ a precondition of

meaningful evaluation-and an essential element of successful

implementation-is that the activities involved in delivering the policy

8
should be specified and the outputs so far as possible identified.’ Dye

(1987) highlighted that policy evaluation has both symbolic and

tangible effects on the populace and that generally the symbolic

aspect which are predominantly people perception is mainly used to

judge government policy rather than the tangible which is the things or

actual impact on the ground. It is the mandate of national

governments to monitor and evaluate its policies in line with its set

targets or benchmarks; after impact analyses; systematic surveys;

dissemination of results; and identification of responsibilities.

Though the task of evaluation is supposedly a domain of national

governments, civil society plays an important role as they promote

democratic governance by providing government policy makers with

information on the needs and demands of citizens and directly

collaborating in the decision-making process for developmental

purposes. According to Dye (1987) policy evaluation can be carried out

through various formats which include among other ways the

following: hearings and discussions; site visits; program measures;

comparison with professional standards; and evaluation of citizen

complaints. Transparency and accountability require that results of

evaluations and impact analyses should be broadly disseminated,

debated and discussed to identify obstacles, learn lessons and create

conditions for future reform. At the same time, it is critical to identify

9
which duty holders have satisfied their obligations and which have not

and to suggest proper measures to rectify the situation. This curtails

the problem of government officials manipulating the evaluation

process to report favourably to their own advantage on the impact of

public policy, as is the case in most developing countries hence the

need to incorporate non-governmental agencies in the process.

Through community and grassroots organizations, civil society can

effectively communicate the needs of vulnerable groups that are often

voiceless and serve as a bridge between them and government

institutions. It is able to articulate, aggregate and represents specific

interests of the population, especially those traditionally excluded from

access to power and representation such as women, minorities and

indigenous peoples and serve as brokers with political parties and

state institutions. Through such activities, civil society can contribute

to making state policies more sensitive and responsive to the needs

and opinions of these groups hence they play a critical role in the

evaluation of policies.

The establishment of a development structure that integrates the

participation of civil society and government at local/village,

intermediate/commune and/or district, regional/province and national

levels within a single coherent framework is of paramount importance

10
in as far as policy evaluation is concerned. This integration can help

countries to capitalize, in policy terms, on experience gained at the

community level and to institutionalize grassroots participation in

policy evaluation. The above being an ideal case but most African

governments do not work hand in glove with Civil society hence policy

evaluation can be deemed as being just lip service as the vital

machinery is not available at the grassroots level were most activities

take place. Most African governance institutions have a tendency of

having the craft competency of promulgating policies that end up not

being evaluated but are immediately replaced by other beautifully

organised policies that do not bear any resemblance to the actual

problems or situations on the ground. This literary translates to the

fact that the methodologies for evaluations being used are out of touch

with reality and thus do not measure their intended variables

effectively and efficiently, as they should.

However, the evaluations done by most governments are desk based

and thus present fictious impressions of the prevailing conditions on

the ground. Whilst policy evaluation is an essential tool for good or

democratic governance, most governments in Africa leave this task to

non-governmental organisations or civic society. They are however

quick to point that these organisations have hidden political agendas

when reports generated by civic society organisation are published

11
highlighting shortcomings of some government policies leading to the

expulsion of these organisations from the so called ‘sovereign nations’

leaving a huge vacuum between governments and their citizens.

Conversely, exclusion of civil society contributes to inefficiencies and

failures and can negatively affect people’s livelihoods. Most

governments are aware that ‘in diverse dynamic and complex areas of

society activity, no single governing agency is able to realize legitimate

and effective governing by itself’, according to Gerry Stoker of the

University of Southampton, UK hence the need for the inclusion of

CSOs in policy formulation and governance issues. Civil society groups

usually champion the causes of vulnerable and marginalized groups

meaning that civil society involvement in policy-making and

subsequently its evaluation requires the creation and maintenance of

their constant links with governments and donors. Although distrust

between the two sectors can limit cooperation, experience indicates

that it is precisely when civil society organizations collaborate with

state institutions that they have more opportunities to strengthen and

improve state policy formulation and evaluation. The absence of these

linkages largely transform these countries into dormant volcanoes that

will later explode as time goes by as policies that will be implemented

by governments do not address the needs or aspirations of its

citizenry. In most developing countries, national monitoring and

12
evaluation is complicated by lack of available and reliable data and

limited expertise on human rights and the justice system; and the

continued twist of facts to tilt the beam of promoting the balance of

power towards certain political players at the expense of people’s

birthrights.

Countries need to pay particular attention to the generation of

indicators and data to measure the progress in people’s enjoyment of

their rights and access to justice, as well as governments’ commitment

and fulfillment of their human rights obligations. The sub-standard

capacity in national statistical offices which are suppose to gather data

which is used to carry out evaluation functions are due to absence of

human, technological and financial resources is common in most

developing countries. There is a dire need for the strengthening of

national statistical units and civil society organisations through better

mandates, improved human resources, technological and adequate

financial resources with more emphasis being placed on

methodologies, statistical techniques and instruments and setting up

of systems. What is critical for governments and all the stakeholders in

public policy is to develop a system that aligns the cost factor to the

benefits of any public policy pronouncements. This includes amongst

other measures according to Dye (1987) comparisons that real can be

attributable to the impact of certain policies pronouncements or vice

13
versa. There are research designs for policy evaluations such as the

before versus after comparison; projected trend line versus post

program comparisons; comparisons between jurisdictions with and

without programs; and comparison between control and experimental

groups before and after policy implementation (ibid: 357-9).

Conclusion

It can be seen that policy evaluation is major part of the policy process

in any country as this sheds the way forward for any policy formulation

process in any country. It is the mandate of national governments to

monitor and evaluate its policies in line with its set targets or

benchmarks; after impact analyses; systematic surveys; dissemination

of results; and identification of responsibilities but the role of other

stakeholders such as the UNDP, NGOs and other CSOs can never be

downplayed as most governments are now taking a back seat in as far

as social issues and social services are concerned.

According to Kapfunde (2004) the objectives of conducting policy

evaluations are the following: to realise efficient and high quality public

administration focusing on the citizens; shift to performance-based

public administration from the viewpoint of the citizens; and thorough

accountability to the citizens. Civil society groups usually champion the

causes of vulnerable and marginalized groups meaning that civil

14
society involvement in policy-making and subsequently its evaluation

requires the creation and maintenance of their constant links with

governments and donors. Although distrust between the two sectors

can limit cooperation, experience indicates that it is precisely when

civil society organizations collaborate with state institutions that they

have more opportunities to strengthen and improve state policy

formulation and evaluation.

In most developing countries, national monitoring and evaluation is

complicated by lack of available and reliable data and limited expertise

on human rights and the justice system; and the continued twist of

facts to tilt the beam of promoting the balance of power towards

certain political players at the expense of people’s birthrights. What is

critical for governments and all the stakeholders in public policy is to

develop a system that aligns the cost factor to the benefits of any

public policy pronouncements and these should be heavily aligned to

meeting the tangible needs of society rather than the symbolic ones,

however the plight of the generality of the populace should be at the

fore front rather than the needs of the elected political office bearers

as is the case in most post colonial African governments were the

liberties of the majority are sacrificed for the sole benefit of a couple of

individuals because of their political inclinations and not for the

15
delivery or compliance with the basic premises to the basic principles

of good governance issues.

References

Dye T.R.1987. Understanding Public Policy 6th Edition, New Jersey:

Prentice-Hall Inc.

Kapfunde C.L. 2004. Policy Analysis, Implementation and Evaluation

(Module DAE504), Harare: Zimbabwe Open University.

Hogwood B. W. and Gunn L.A. 1984. Policy Analysis for the Real World,
Oxford: Oxford University Press.

Economic Commission for Africa. 2004. Public Sector Management

Reforms in Africa: Lessons Learned, Addis Ababa: Development Policy

Management Division (DPMD).

Stoker, G. 1998. Governance as theory: five propositions. International

Social Science Journal 50 (155): 17–28.

16