Table of Contents
Although extension educators, donors and administrators are in favor of evaluating extension programmes, yet the fact remains that most of the people are not enthusiastic to undertake it.
Among the reasons being:
• To mask insecurity, incompetence and inadequacy of our efforts.
• Fear of change that evaluation might precipitate as a result of self protective and defensive.
• The need to avoid embarrassment about potential bad news.
• Evaluation is often an additional workload to an already heavy or almost impossible workload.
Similarly, there have been certain myths that have often discouraged extension managers from engaging themselves in useful evaluation.
• Evaluate only when mandated. Many funded programmes require evaluation as a form of accountability and as a result it becomes a myth that evaluation should take place only if it is mandated. Evaluation is more than just accountability.
• Evaluation is an add-on activity. In reality it should be part and parcel of the programme.
• Evaluation is an activity for experts. Yes when complex methods are used. However, there are many ways of doing evaluation for the purposes of improving delivery which do not need experts.
• Outside evaluators are the best. Not always. Yes, external objective evaluators are often useful in challenging insiders to address certain issues that they tend to overlook because of their involvement in the programme.
• There is one best evaluation approach. Different approaches are better depending on the issue to be evaluated as well as the prevailing situation.
• Quantitative data are the best. Combining qualitative and quantitative methods can lead to better understanding of certain phenomena.
WHAT IS EVALUATION?
• Evaluation basically means passing judgement on something according to a given criterion.
• With respect to education (which extension is part) Ralph Tylor has defined evaluation as being:
The process of determining to what extent the educational objectives are actually being realised by the curriculum (programme) and instruction.
In this case the criteria are the educational objectives which were set in advance.
• A more general and all inclusive definition of evaluation was suggested by Wilbur Harris as:-
The SYSTEMATIC process of judging the worthiness, desirability, effectiveness, or adequacy of something according to definite CRITERIA and PURPOSES
Out of Harris definition three important points emerge:
(1) Evaluation is purposeful and is not done for the sake of it. The way evaluation is conducted must reflect the reason for carrying out that evaluation.
(2) The process of evaluation involves three elements :
- are a standard of what is acceptable.
• Thus, a criterion can be an expectation or a laid down rule which is used as a reference point.
e.g. A farmer is expected to know how to apply sulphate of ammonia by the end of a demonstration.
• Criteria help us to make various decisions or judgements.
• There can be various criteria on the same process.
• Hence, it is important to agree with all categories of people involved in the programme as to which criteria are going to be used in making evaluation.
• This will help to have the same logical judgements, since your views will be based on the same criteria.
• Also in that way you can make relevant improvements in the programme.
Objective (based on certain rules)
Criteria can be or
Subjective (based on ones value judgement)
• In most cases it is better to use objective criteria to avoid one’s biases.
• However, practically speaking, it is very difficult to completely wipe out some elements of subjectivity in your criteria.
• Is all information (data) you have which will be used for comparison to the criteria.
• There must be some kind of measurements in order to get the information (data).
• Measurement – is a process of assigning symbols to the information but which a defining the quantity or quality.
Measurement scales include:
• Sources of evidence
(i) records (e.g. farm records)
(iii) Surveys (e.g. interviews using questionnaires) etc.
• Basically means coming up with some conclusions about the programme based on criteria and evidence.
That is to say:
• if you have the standards (criteria),
• and you have the evidence (observations & measurements),
• then you can compare to see which evidences have met the standards and which have not.
Objective judgement – are judgments made based on empirical evidence.
Subjective judgement – are judgements made based on personal biases.
(3) Evaluation is systematic rather than a haphazard process.
i.e. Depending on the purpose there must be:
o careful selection of criteria
o accurate observations and measurement
o careful comparison (observation Vs criteria),
before making judgements and recommendations.
Evaluation is the process by which the effectiveness of extension is assessed. It is more than simply finding out what happened; it involves passing judgement on what happened. Was the outcome of the programme good enough? Was it better or worse than expected? Could more have been achieved?
Extension programmes are evaluated to
(a) ascertain for the extension organization how well agents perform, so that their suitability for promotion may be assessed;
(b) satisfy the government that public money spent on extension is being used effectively; and
(c) permit the agent to learn from what has happened.
Evaluation is a waste of time unless the results have an influence on future extension decisions.
Agents should, therefore, ask questions about the following aspects of the programme:
Results: What happened as a result of the- extension programme? Were they the results that were expected, and were there any unexpected results?
Inputs: Were all the planned inputs available and, if not, why?
Levels of evaluation
• There are several levels of evaluation of extension programmes. At the most general level, the effect of extension on agricultural production, family incomes and standards of living can be evaluated
• However, extension is not the only factor that leads to higher production and living standards; changes in prices and in the availability of inputs are two of the many additional factors that affect the level of crop and of livestock production
• He/she should also observe who is benefiting from extension. Is a broad cross-section of the farming population sharing the benefits, for example, or do one or two particular groups benefit most?
• Did the extension activities take place in the planned sequence and at the right time?
• Did these activities lead to the expected results?
• Finally, the agent can evaluate at the level of each extension activity. All extension activities, such as demonstrations, talks or meetings with a farmers' group, have a purpose. The agent should try to check, wherever possible, not only how well the activity itself was conducted but whether the purpose was achieved.
• At all levels of evaluation, the agent needs to collect information to compare the situation after the activity with the situation existing before.
• Some effects can be assessed much sooner than others. Immediately after a public meeting, for example, the agent can talk to a few members of the audience and check how clearly they understood what he was saying. Changes in behaviour, on the other hand, will not happen at once and the agent must wait before checking these.
There are several ways of collecting information for evaluation at the village level.
Agent's reports: Whether or not a formal report of each extension activity is required from agents by their extension officers, the agent should make some notes on each activity for his own use, concentrating on his conduct of the activity and on points to note for future occasions.
Supervisors: It is not easy for an agent to assess how well he/she conducts an extension activity; in particular, he cannot see himself/herself through the eyes of the farmers who attend. It is useful, therefore, to have constructive comments from a supervisor or colleague.
Discussions: Informal discussion with farmers after the extension activity will reveal their immediate reactions. It is often useful to record such discussions using a tape recorder for later transcription and fuller analysis.
Questionnaires: Simple check-lists and questionnaires can be used when the agent has the time and opportunity to carry out a more formal evaluation of extension activities
Observation: Where changes in farming practice are concerned, observation is an accurate source of information. The agent can see whether or not his advice is being adopted on farms in the area.
• Many extension organizations have their own formal procedures for evaluation
• Whatever the formal procedures in a particular organization, however, the agent should think of evaluation as an attitude of mind. He/she should develop a readiness to ask what happened, why it happened and how it could be done better in the future. In this way, he/she will continue to learn and improve his/her extension work.
THE PROCESS OF EVALUATION
Involves certain sequential steps:
(1) You have to decide on what to evaluate
(2) Then set the criteria or standards
(3) Make observations and measurements
(4) Compare observations to criteria
(5) Make judgements and recommendations
However, the evaluation process is a cyclical one, or in other words is a continuous process since you have to make an evaluation of what you decided to implement as improvements following the previous evaluation.
In other-words we are asking ourselves why evaluation should be part and parcel of the programme development.
(1) To improve existing programmes and to form a basis for planning future Programmes.
(2) To determine programme effectiveness and efficiency.
• Since resources are limited, programmes have to be implemented as efficiently as possible. The rate of efficiency of a programme offers a basis for either modifying or completely abandoning a program.
• It also helps to tell whether the set targets have been reached or not.
When resources are very limited it becomes necessary to justify the expenditure on a particular programme. It is only through evaluation that you can justify this.
(4) To assess external factors that can affect the implementation, especially factors that cannot be identified in advance.
• Without evaluation you cannot tell which extra factors affected the programme and which ones for example forced the modification of the programme implementation.
(5) Evaluation is important as a learning process.
• While evaluating you can learn new principles or techniques.
• You can test new programming models to see how they work.
• In doing so you generate knowledge that can be used in future by yourself or others who will have shared your experiences.
(6) Generally we can say evaluation sets a basis for our future decisions in all facets of the programme.
There are at least five major elements in most evaluations:
1. Focus questions
2. Objects or events to be evaluated
3. Data or evidence
4. Analysis and interpretation using judgement perspective.
5. Judgements, conclusions or findings.
NOTE: Purpose and approaches/models will differ or may vary, but these elements will be present in one form or the other.