Engineering a microstructure morphology has been a long-lasting challenge in materials science. Historically, forward-based models, including experimental and high-fidelity models, based on trial and error, have been used to engineer a specific microstructure. In this work, we have developed a novel fused-data deep learning algorithm that is able to predict the required chemistry and processing to reach specific microstructure morphologies. FeCrCo permanent magnets are the model alloy in this work. The model input is the Fe distribution morphology and it predicts the Cr and Co concentrations and processing time and temperature for that particular morphology. The model analysis shows that shallow networks can predict chemistry well. However, deep networks are required to predict the processing time and temperature. We validated the model against a TEM micrograph and while the model is trained with synthetic data it performs reasonably well in chemistry and processing prediction for a TEM micrograph.