Client Context

The floodplains of western Kenya flood on a schedule that everyone living there understands and nobody can plan around precisely. Seasonal inundation displaces households, destroys crops close to harvest, contaminates water sources, cuts road access, and drives the cycle in which families rebuild the same asset repeatedly rather than accumulating anything. The losses are recurrent rather than catastrophic, which is precisely why they are so corrosive: each event is survivable, and the cumulative effect is a population unable to move beyond subsistence.

The Western Kenya Community-Driven Development and Flood Mitigation Project addressed this with a dual design. Flood mitigation infrastructure reduced exposure, while community-driven development placed investment decisions with the communities themselves, on the reasoning that people living with the risk understand their own priorities better than a central planner does. The programme was implemented by the State Department of Special Programmes under the Ministry of Devolution and Planning, and ran for eight years.

At that duration and design, the programme accumulated a substantial claim about what it had achieved. Verifying that claim required going to the households.

The Challenge

Measuring the impact of flood mitigation is methodologically awkward for a reason that sounds trivial and is not: success looks like an absence. A household that was not displaced, a crop that was not lost, and an asset that did not need rebuilding are outcomes that leave no trace in any administrative record. Establishing them requires asking people directly, at sufficient scale for the answers to mean something.

The assignment compounded that with breadth. It had to assess the programme against relevance, efficiency, effectiveness, sustainability, and impact, five criteria testing different things, from whether the design matched community needs to whether the infrastructure would still function once the programme closed. It had to cover a community-driven development component whose whole design point is that investments differ from community to community, which defeats standardised outcome measurement. And it had to assess institutional capacity, training, and awareness alongside household outcomes, since a community-driven programme’s sustainability depends as much on the structures it leaves behind as on the assets.

Then there was the field reality. Household data collection across dispersed rural populations in flood-affected areas, at the scale required for credible findings, within a four-month window, using paper instruments would have produced a data quality problem larger than the analysis could absorb.

Our Approach

ACAL treated the data collection method as a design decision rather than an implementation detail, developing a mobile data collection application for smartphones specifically for the assignment. Digital capture at source removed the transcription errors, illegibility, and enumerator inconsistency that paper-based household surveys of this scale routinely generate, and allowed quality assurance while teams were still in the field rather than after demobilisation. Around that, the team built an evaluation design and sampling framework covering the programme’s geography, recruited and trained enumerators to a common standard, and structured the assessment around the five evaluation criteria alongside institutional capacity and beneficiary impact. Findings were tested with stakeholders in workshop and again at validation before the final report.