Conservation agriculture (CA) is widely promoted in sub-Saharan Africa both in open fields and in agroforestry where the practice is known as ‘conservation agriculture with trees’ (CAWT). Although advantages and disadvantages of CA are well studied under sole cropping, less is known about its impact in agroforestry systems. The performance of open pollinated maize varieties under CA, CAWT, sole maize under conventional tillage (CT) and conventional tillage with trees (CTWT) was compared on-farm in equatorial savannah areas over four consecutive seasons in Rwanda and two seasons in Ethiopia. The tree species considered in the study were mature Grevillea robusta (A. Cunn.) and Senna spectabilis (DC.) in Rwanda and mature Acacia tortilis (Forssk.) in Ethiopia. Both CA and the presence of trees consistently reduced maize emergence, leaf area (LA), plant height, and maize yields. Crop emergence was significantly reduced under CAWT compared with CTWT. Maize emergence rates in CAWT and CTWT were respectively 46.9% and 70.1%, compared with 74.7% and 79.8% in sole maize under CA and CT. Grain yield in CAWT and CTWT were respectively 0.37 t dry matter (DM) ha−1 and 1.18 t DM ha−1 as compared with 1.65 t DM ha−1 and 1.95 t DM ha−1 in CA and CT. We conclude that CAWT strongly reduces crop yield in the equatorial savannah of East Africa. CA is incompatible with agroforestry under the conditions of our study. There is an urgent need for rigorous research to revisit if, when and where CAWT can generate benefits for smallholder farmers.