Rural water systems in Africa have room to improve water quality monitoring. However, the most cost-effective approach for microbial water testing remains uncertain. This study compared the cost per E. coli test (membrane filtration) of four approaches representing different levels of centralization: (i) one centralized laboratory serving all water systems, (ii) a mobile laboratory serving all systems, (iii) multiple semi-centralized laboratories serving clusters of systems, and (iv) decentralized analysis at each system. We employed Monte Carlo analyses to model the costs of these approaches in three real-world contexts in Ghana and Uganda and in hypothetical simulations capturing various conditions across rural Africa. Centralized testing was the lowest cost in two real-world settings and the widest variety of simulations, especially those with water systems close to a central laboratory (<36 km). Semi-centralized testing was the lowest cost in one real-world setting and in simulations with clustered water systems and intermediate sampling frequencies (1−2 monthly samples per system). The mobile lab was the lowest cost in the fewest simulations, requiring few systems and infrequent sampling. Decentralized testing was cost-effective for remote systems and frequent sampling, but only if sampling did not require a dedicated vehicle. Alternative low-cost testing methods could make decentralized testing more competitive.