Based on their structure, non-crystalline phases can fail in a brittle or ductile fashion. However, the nature of the link between structure and propensity for ductility in disordered materials has remained elusive. Here, based on MD simulations, we investigate how the degree of structural disorder affects the fracture of disordered materials. By applying the activation-relaxation technique (an open-ended saddle point search algorithm), we demonstrate that the propensity for ductility is controlled by the topography of the energy landscape. We observe a power-law relationship between the particle non-affine displacement upon fracture and the average local energy barrier. This reveals that the dynamics of the particles upon fracture is encoded in the static energy landscape, i.e., before any load is applied. This relationship is shown to apply to several classes of non-crystalline materials (oxide and metallic glasses, amorphous solid, and colloidal gels).