Many virtues of High Entropy Alloys (HEAs), including their strength, ductility, and fatigue resistance, are highly sensitive to vacancy diffusivity. Similarly, solute interdiffusion is governed by vacancy diffusion – it is often unclear whether HEAs are truly stable, or effectively stabilized by slow interdiffusion. Efforts to tune the kinetic stability and the properties of HEAs depend on both the knowledge of the vacancy transition barriers within a given alloy and an understanding of how these barriers influence vacancy diffusivity. We present a generalized theory of vacancy diffusion in rugged energy landscapes, paired with Kinetic Monte Carlo simulations of HEA vacancy diffusion, driven by energy barrier statistics measured by nudged elastic band calculations of equiatomic CoNiCrFeMn. Theory and simulations show that vacancy diffusion in HEAs is not necessarily sluggish, but can potentially be tuned, and that trap models are an insufficient explanation for sluggish diffusion in the CoNiCrFeMn HEA.