The representation of convection remains one of the most important sources of bias in global models, and evaluation methods are needed that show that models provide the correct mean state and variability, both for the correct reasons. Here we develop a novel approach for evaluating rainfall variability due to convectively coupled Kelvin waves (CCKWs) in this region. A phase cycle was defined for the CCKW cycle in OLR and used to composite rainfall anomalies. We characterize the observed (TRMM) rainfall response to CCKWs over tropical Africa in April and evaluate the performance of regional climate model (RCM) simulations: a parameterized convection simulation (P25) and the first pan-Africa convection-permitting simulation (CP4). TRMM mean rainfall is enhanced and suppressed by CCKW activity, and the occurrence of extreme rainfall and dry days is coupled with CCKW activity. Focusing on regional differences, we show for the first time that there is a dipole between West Africa and the Gulf of Guinea involving onshore/offshore shifts in rainfall, and the transition to enhanced rainfall over west equatorial Africa occurs one phase before the transition over east equatorial Africa. The global model used to drive the RCMs simulated CCKWs with mean amplitudes of 75%–82% of observations. The RCMs simulated coherent responses to the CCKWs and captured the large-scale spatial patterns and phase relationships in rainfall although the simulated rainfall response is weaker than observations and there are regional biases that are bigger away from the equator. P25 produced a closer match to TRMM mean rainfall anomalies than CP4 although the response in dry days was more closely simulated by CP4.