|
![]() |
|
![]() | |||||||||
![]() | |||||||||
![]() | |||||||||
![]() |
Activities
|
Starting the discussion: steps to
making sex safer 52
Page 53 54
Part 4: Assessing change |
All of us involved in HIV education or care need to find out how useful our work is, and how to make it better. However, many training and education projects stop short of evaluating and assessing change. This can happen because evaluation is felt to be too difficult or it is left until too late, when the project is nearly finished. Evaluation is often avoided because changes in attitudes or behaviour can take a long time and are difficult to measure.
Evaluation can also feel threatening when it is carried out by 'experts' outside the project who may expose problems in the work.
Evaluation involves collecting information, keeping records and making careful observations about the project's activities. It assists in finding out whether education activities are being carried out as planned, and whether the project is enabling people to make changes or solve problems.
It is very important to involve the community in planning and evaluation activities. People can contribute ideas at all stages and be part of the process.
Starting the discussion: steps to
making sex safer 53 Page 54
55
Part 4: Assessing change |
Why evaluation is important
In order to evaluate projects effectively, we need to ask questions and seek answers at different stages of the project:
Before the project starts
What is the situation like? What are people's priorities and what needs to be done? This is called a needs assessment or base-line survey.
During the project
Are the activities being carried out as planned? This is called project monitoring or process evaluation.
After the project has ended
Did it make a real difference in people's lives and what changes have occurred? This is called an end-point or impact evaluation.
Results from this evaluation are used to plan the next phase of the project or any new activities.
Starting the discussion: steps to
making sex safer
54 Page 55 56
Part 4: Assessing change |
Stages in evaluation
Evaluation starts in the planning phase of the project and continues throughout the programme.
Assessing the situation before the project starts e. g. recording numbers of clinic visits and sexually transmitted infections, and running focus group discussions, listening surveys, etc.
Setting a broad project goal or aim e.g.
to increase the use of STD services among workers in a particular area and raise awareness of the need for safer sex
and clear and realistic objectives (targets) which can be measured e.g.
to provide at least 10 condoms per month per adult and improve treatment for sexually transmitted infections
to carry out educational and training activities.
Planning detailed activities to achieve the goal and the objectives
Selecting simple monitoring methods (collecting information) to measure changes that have occurred as a result of your activities and check that targets are being achieved e. g. reviewing clinic records, setting up focus group discussions and interviews.
Doing an evaluation at the end of the project to explore if objectives have been achieved and carry out more in-depth activities to assess any overall changes.
Finding ways to involve people and feed the information back to the community.
Starting the discussion: steps to
making sex safer
55 Page 56 57
Part 4: Assessing change |
Collecting information
Evaluation involves collecting and analysing information about the project. Most organisations are already collecting information about their project activities -and often collecting more information than can be used! Whatever methods you decide to use, think about how to:
collect information systematically - record sheets and checklists are only useful if they are filled in regularly
be selective - only collect information that refers specifically to what you want to measure
ensure that the recording system is easy to use and that the person using it understands what it is for.
Information can be collected in a number of ways:
listening surveys and focus group discussions (see Part 1)
community mapping
through personal interviews
by questionnaire
through group discussions
by observation
through looking at records at clinics or pharmacies
diaries and checklists kept by educators or health workers.
Information that can be measured in numbers is called quantitative. Information that concerns people's feelings or attitudes and cannot be measured in numbers is often called qualitative.
This information is sometimes more difficult to analyse but it provides a very useful guide to what people are really saying, thinking, feeling and doing.
The information can be collected by people working in the project as health workers or trainers, or by external researchers.
Activities in this section describe how to collect:
baseline information using community mapping (See also the listening survey in Part 1)
information about the process of the project using a structured group discussion and a checklist
end-point information using a questionnaire.
Starting the discussion: steps to
making sex safer
56 Page 57 58
Part 4: Assessing change |
Activity: Community Mapping
Aim: to gather information about what is happening in the community MATERIALS large sheets of paper and
marker pens
Time: 1 day
Community maps are simple diagrams which show information about an area in a way that is easy to understand e. g the location of the clinic, the schools, the number of houses in an area, the beerhall, the bus terminus etc. They are prepared by community members and can be used to analyse the situation and the problem.
Example
Programme staff wanted to explore where people felt at risk of unsafe sex, pressure or violence. Small groups (young men, older men, young women, older women) were invited to draw maps of their community.
The drawings showed places where people lived and worked as well as places such as bus stops, bars, hotels and other meeting places where risky or unsafe sexual behaviour can occur. For example, men listed places where they met in their leisure time, such as beer shops or bars, or special houses where they were able to meet women. Information was also gathered about places where condoms were available.
The group then discussed what steps could be taken to ensure that condoms are made available at the local stalls, family planning clinic, bars and beer shops. They also realised that neither the women nor the men had easy access to treatment for sexually transmitted infections, and that school girls were being approached by men. The programme staff -decided to look at addressing these issues in more detail.
At the end of the year the group looked again at the maps they had drawn earlier. Participants drew a new map showing any changes such as where condom outlets had been set up.
Starting the discussion: steps to
making sex safer
57 Page 58 59
Part 4: Assessing change |
Activity: Structured group discussion
Aim: to assess whether the project activities are being carried out as planned (project monitoring or process evaluation)
Materials: large sheets of paper and marker pens
Time: about half a day
Prepare a few questions for the discussion:
![]() |
What are the main successes and problems that have been experienced during the project? |
![]() |
What changes in attitudes or behaviour have been observed in the community since the project started? |
![]() |
What are the possible barriers to change? |
![]() |
How should the project develop? |
Ask the participants to form groups of six to eight people.
Give each group a pen and a large sheet of paper.
Ask each group to discuss the questions and write their answers in order of importance.
Re-group with the other workshop participants, and display all the response sheets.
Discuss the similarities and differences in the large group.
Reach agreement about the project priorities in either small groups or as the large group.
Make a final list which will be displayed in the project offices as a
reminder of the discussion.
Starting the discussion: steps to
making sex safer
58 Page 59 60
Part 4: Assessing change |
Activity: Using checklists
Aim: to keep simple records about project activities
Materials: copies of checklists (see example)
These forms are completed regularly by people working with the project. They can provide information about a wide range of things, for example:
reach - who is attending activities
methods used - group activities, drama sessions, picture code sessions etc
numbers of condoms distributed, where and by whom
new activities within the community
other interesting observations about the project.
Starting the discussion: steps to
making sex safer
59 Page 60 61
Part 4: Assessing change |
Activity: Using Questionnaires for a survey
Aim: to collect information for an end point evaluation
Questionnaires are often used in surveys to collect information from a large number of people. If people are literate, then they can complete the questionnaire themselves. In non-literate communities, trained researchers will use the questionnaire to collect information through a personal interview. Which ever method is used, everyone must be asked the same questions in the same way so that the results can be analysed and compared.
Designing a questionnaire
Decide:
![]() |
what you want to find out |
![]() |
who will collect the information |
![]() |
from whom and how many people |
![]() |
when you need to collect the information |
![]() |
how the information will be collected (personal interviews or written questionnaires) |
![]() |
how the information will be analysed |
![]() |
what will be done with the information you have collected. |
Then, think carefully about what information is needed, in relation to
the aims and objectives of the project.
![]() |
Keep questions brief and use simple language. A question is easiest to understand when it addresses one idea at a time. |
![]() |
Use exact words which cannot be misunderstood to obtain precise answers and accurate information - ask How many times have you had diarrhoea in the last week? rather than How many times recently? |
![]() |
Use words which are non-judgemental, such as person with HIV rather than AIDS victim, or more than one sexual partner rather than promiscuous. |
![]() |
Use closed questions if a 'yes' or 'no' answer or a number is needed. For example Have you received any AIDS education at school? |
Starting the discussion: steps to
making sex safer
60 Page 61 62
Part 4: Assessing change |
![]() |
Open-ended questions require the respondent to give a longer answer in their own words. For example: What have you learnt at school about HIV and AIDS? Open-ended questions help to gather opinions that you may not have expected. |
![]() |
Ask straight-forward questions rather than ones which threaten or make a person feel uncomfortable, such as: In your opinion, how do you feel about the treatment which you receive from health workers at this clinic rather than Do you think that the health worker at this clinic is polite and caring? |
![]() |
Most importantly, keep the questionnaire short by avoiding unnecessary questions. |
Before finalising the questions
Always go through the questions (pre-test) with a small group of
people who are similar to the people who will be interviewed. This
ensures that the questions are easily understood and that people are
willing to answer them - and that the interviewers know how to
complete the questionnaires. The pre-test will also show whether the
information collected is useful and can be easily analysed.
Carrying out the survey
![]() |
Choose interviewers who are trusted and with whom people can easily talk. |
![]() |
Go through the questionnaire in a place where people feel comfortable and have privacy. |
![]() |
Interviewers must be sure to explain why the survey is being conducted and what the findings will be used for. |
![]() |
Ensure that respondents have time to answer questions |
![]() |
Invite further comments at the end of the interview. |