Study Design for the AME-SADA Cholera Prevention Program
Gwenn Fairall, Dalit Gulak, Eileen Searle
Purpose
AME-SADA’s Cholera Prevention Program is being implemented to help fight the on-going cholera epidemic in Arcahaie and Cabaret, Haiti, two towns without a presence of other non-profit organizations. The program seeks to significantly decrease the incidence of new cholera cases through the training of local healthcare practitioners; obtaining necessary supplies for the treatment of cholera and water sanitation; and through educational training for the public. This evaluation is focused on the impact of the education efforts targeting the public. We will evaluate the effectiveness of the prevention arm of the program (education and community mobilization/empowerment) to ultimately decrease the incidence of cholera in these two towns. This evaluation will be useful for future prevention projects that organizations might implement in the area. It will also be an effective tool in providing concrete data on the success of this program for grant proposals and funding of similar projects by AME-SADA.
Objectives
General Objectives
- To evaluate the overall objective of the AME-SADA Cholera Prevention Program aimed at reducing the number of cholera deaths in Arcahaie and Cabaret, Haiti by 75% in 18 months.
Specific Objectives
- To quantify change in cholera levels due to the prevention arm of the AME-SADA program.
- To determine if the prevention project implemented by AME-SADA had an effect on the community’s sanitation practices.
- Use qualitative measures to understand potential weak areas of the program that prevent the community from reducing the incidence of cholera.
- To make recommendations to AME-SADA and key stakeholders in order to continue improving their efforts to reduce morbidity/mortality caused by cholera.
Design
This study will be a longitudinal, randomized control trial that will occur in Arcahaie, Haiti. Because the efficacy of treatment for cholera is well established, all persons living in the target area of Arcahaie and Cabaret will receive any necessary medical care and supplies as it would be unethical and inappropriate to withhold them. The new part of the intervention program is the prevention arm. Due to budgetary constrictions we chose to focus our intervention on the town of Arcahaie. Neighborhoods within Arcahaie will be mapped and then randomized to receive the educational intervention or serve as controls.
Our study will also be a simple factorial design. If the arm of the monitoring and evaluation study receiving the intervention shows that the program is successful in reducing the incidence of cholera, it will be implemented in the control group; and in this case extended to include all of Arcahaie and Cabaret. Once the scale of the program is increased, the monitoring and evaluation will continue to determine if the program implementation remains successful in its larger form. The program is designed to be implemented as a response to an outbreak to reduce incidence of cholera. However, the activities in the program also serve as prevention by increasing knowledge and improving sanitation practices. There is potential that this model could be used elsewhere when there is an outbreak of a disease that requires an increase in knowledge and change of behavior in order to be successful.
|
Longitudinal Study of Evaluation |
|||||||
|
Time |
0 Month |
1 Month |
3 Months |
6 Months |
9 Months |
12 Months |
18 Months |
|
R Intervention |
O1 |
X1O2 |
O2 |
O3 |
O2 |
O3 |
O2 |
|
R Control |
O1 |
O2 |
O2 |
X2O3 |
O2 |
O3 |
O2 |
R: randomization of communities within Arcahaie
X1: Implementation of program in neighborhoods randomized to be the intervention group
X2: Program implementation expanded to all other neighborhoods if it is shown to be effective
O1: Mixed-methods data collection as outlined below
O2: Quantitative data collection only
O3: Mixed-methods data collection
O1 (One month before the first intervention is implemented): Baseline data collection will include both the control and intervention groups (it will be used to evaluate the efficacy of the program and also to demonstrate successful randomization). Data collected will include: demographics, incidence of cholera in the previous month, number and type of water supply available and used in each community, and knowledge about the causes of cholera. This will be done by reviewing health-care records to see if they provide adequate information to determine the number of cholera cases and which neighborhood the case resides in. If record keeping is not adequate, a new system will be introduced for the duration of the study. The program being evaluated provides the only health-care in the area so all cases requiring treatment should present to the hospital.
Additionally, 47 HAs and 192 TBAs will be selected in the community and taught to educate community members and mothers pre and post partum. Within each neighborhood a survey on household demographics, water source/supply, water sanitation practices, knowledge about cholera, and any community measures being undertaken to improve water sanitation and/or decrease cholera will be conducted.
X1 (Month 1) This point marks the first evaluation intervention that will be implemented in the households randomized into the intervention group.
O2 (Months 1, 3, 9, and 18) At these observation points, the incidence of cholera (actual cases requiring treatment and probable cases identified by HAs and TBAs) will be collected and analyzed in aggregate, comparing the intervention group to the control group.
X2 (Month 6) If the program is found to be successful in the first intervention group, program implementation will be expanded to all neighborhoods in the control group.
O3 (Months 6 and 12) At these observation points, the incidence of cholera will again be analyzed. Additionally, the initial survey will be re-administered to assess for changes in knowledge and practice regarding water sanitation and cholera prevention.
Furthermore, HAs, TBAs, and neighborhood leaders will be interviewed about whether or not community meetings have taken place regarding cholera/water sanitation (place, topic, and # of attendees) and what the outcomes of those meetings have been.
The study will employ mixed-methods for data collection including household surveys, interviews with community leaders, assessment of water sources, and identification of cholera cases to calculate incidence. The triangulation of multiple design modalities will enhance the validity of any results and with an adequate sample size will also make the results generalizable.
Impact to Measure
This study is designed to evaluate and measure the impact of the prevention arm of the AME-SADA program. Indicators we will examine include percentage of households surveyed that show an increase in knowledge of cholera prevention techniques (including water sanitation and waste disposal practices). In addition to knowledge, we will look to evaluate the impact the program has on change in practice by households in the intervention groups. Ultimately, we hope to measure the long term impact this program is having on reducing the incidence of and mortality from cholera and other water-borne illnesses.
Inclusion criteria
In the initial portion of the evaluation all persons residing in Arcahaie, Haiti will have an equal opportunity of being included in the study. The program implementation will focus on neighborhoods and not specifically target individuals. For the purposes of the evaluation, all individuals have the potential to be part of the evaluation. For the survey portion of the evaluation individuals within the identified intervention and control neighborhoods will be targeted on the basis of household status.
Exclusion criteria
Individuals and households outside the identified intervention and control neighborhoods in Arcahaie will be excluded from the initial survey evaluation. Additionally, individuals residing outside Arcahaie and Cabaret will be excluded from all portions of the evaluation.
Assumptions and Threats to Validity
Randomized control trials are the strongest type of study; however threats to validity still exist and assumptions have been made for the success of this study:
Assumptions:
The following assumptions were made at the time of program design:
- Donor participation will remain active throughout the evaluation
- Community members will perceive cholera as a problem they wish to engage in a solution for
- Health Attendants and Traditional Birth Attendants working with the program at the start will remain committed and in Arcahaie throughout the entire program
- Local government will support community efforts for cholera prevention education
Internal Validity Threats:
- History – Post 2010 earthquake and a recent presidential election, the current political climate in Haiti is unstable and the health care system is in transition. While strong efforts will be made to maintain transparency and substantial involvement with key stakeholders in the program, some external political factors will be out of the control of this program.
- Mortality – Mortality from cholera is high if untreated and if patients die that are part of the study this could effect the validity and results of the evaluation.
Construct Validity threats:
- Compensatory Rivalry – It could be possible that the health providers and community workers implementing the treatment part of the cholera program but not the prevention part (so they are in the control group) will compensate for this by giving a better version of their program. This could include prevention education along with treatment, as they might feel that is not fair their patients only receive one portion of the entire program.
- Resentful Demoralization – The control group could find out about the intervention group and act negatively, believing it is unfair they are only receiving treatment for active cholera cases and not any efforts to prevent new cases from occurring.
We have taken steps to ameliorate these issues in our study design by doing the following:
- Gathering baseline data in both the intervention and control groups, to ensure the right information is known and gathered in the proper way in order to serve as a comparison for future measurements of success.
- Utilizing a control group is imperative to allow comparison of data between a group with the program intervention and a group without.
- Triangulation (mixed methods of both qualitative and quantitative data collection) in order to obtain as much varied data as possible.
- Multiple follow-up which controls for the John Henry effect or people working hard while they are being observed yet not working as hard when not being directly observed
- Adequate sample size will increase the validity and generalizability of the results.
Procedure
Data will be collected through several methods detailed below:
Hospital and health facility record review- this quantitative data will be collected to determine incidence of cholera and identify changes over time. Prior to the start of the program, current health facility record keeping will be evaluated to determine if it is adequate to meet the needs of the study. As AME-SADA provides all the western-based medical care in the area, we have access to all the records. Records must identify cholera cases and the neighborhood in which the case resides. If current record keeping system is not adequate, a new system will be introduced for the duration of the study. Cases and their neighborhood of residence will be entered in to a spreadsheet/database and the incidence of cholera in intervention and control areas will be tabulated and reported at O1, O2, and O3 observation points in the study timeline.
Household survey on knowledge, attitudes, and practice regarding water sanitation- this quantitative data will be gathered to identify changes over time in the knowledge, attitudes, and practices regarding water sanitation of the intervention group compared the control group. This will provide the greatest amount of information on the efficacy of the prevention education aspect of the program. The survey will be administered at the O1 and O3 observation points on the study timeline. The data will be collected by local data collectors imported into Cardiff TeleForm and analyzed using CsPro.
Sample Selection- We will access and use the program implementer’s maps that randomized the control and intervention neighborhoods. We will then create a sampling frame within these neighborhoods with the assistance of the Health Assistants (HAs) and Traditional Birth Attendants (TBAs) that work in them as well as identified neighborhood leaders.
Sample size- 870 households will be targeted.
Arcahaie has a population of 102,639 individuals. When administering our survey we are looking for an alpha of 0.05 and a power of 0.8 to detect a statistically significant change over time between the comparison and intervention groups. The rate of change in knowledge, attitude, and practice overtime is unclear so we will use the most conservative estimate of 50%.
Because we are looking at household data, we estimated the number of households in Arcahaie using the Haitian national average of 4.5 individuals per household; which yields about 22,809 households in the city. Using Decision and Support System’s free online power and sample size calculator (www.dssresearch.com) and inputting the above parameters, we will need to survey 378 households in the intervention group and 378 households in the control group. Accounting for an estimated 15% non-participation and attrition of households (due to the longitudinal nature of the study) we will target 435 in each group. The households targeted will be spread evenly over the intervention and control groups and geographically over the city.
One on one interview- these interviews will be conducted with the HAs, TBAs, and identified neighborhood leaders. The interviews will address whether or not community meetings have taken place regarding cholera/water sanitation (place, topic, # of attendees) and what the outcomes of those meetings have been. Also, these interviews will address challenges of implementing the program, community responsiveness to intervention, and any steps needed to improve the program.
In summation, our evaluation of the impact of AME-SADA’s educational intervention on cholera incidence will employ multiple research modalities and a randomized control trial. We hope that our results will demonstrate the efficacy and generalizability of the program in reducing cholera incidence in resource-poor settings.
PDF of the Study Design for the AME-SADA Cholera Prevention Program can be found in the below link: