Soil quality degradation associated with resources scarcity is the major concern for the sustainability of conventional rice-wheat system in South Asia. Replacement of conventional management practices with conservation agriculture (CA) is required to improve soil quality. A field experiment was conducted to assess the effect of CA on soil physical (bulk density, penetration resistance, infiltration) and chemical (N, P, K, S, micronutrients) properties after 4 years in North-West India. There were four scenarios (Sc) namely conventional rice-wheat cropping system (Sc1); partial CA-based rice-wheat-mungbean system (RWMS) (Sc2); CA-based RWMS (Sc3); and CA-based maize-wheat-mungbean (Sc4) system. Sc2 (1.52 Mg m−3) showed significantly lower soil bulk density (BD). In Sc3 and Sc4, soil penetration resistance (SPR) was reduced and infiltration was improved compared to Sc1. Soil organic C was significantly higher in Sc4 than Sc1. Available N was 33% and 68% higher at 0–15 cm depth in Sc3 and Sc4, respectively, than Sc1. DTPA extractable Zn and Mn were significantly higher under Sc3 and Sc4 compared to Sc1. Omission study showed 30% saving in N and 50% in K in wheat after four years. Therefore, CA improved soil properties and nutrient availability and have potential to reduce external fertilizer inputs in long run.