Next generation materials discovery is heavily dependent on the use of Machine Learning (ML). Key ingredients in ML models are well curated data, descriptors and appropriate algorithms. In this talk, we discuss the use of ML modeling in materials discovery for crystalline materials. Using density functional theory-generated crystalline materials data as training, we will compare descriptors and ML algorithms effectiveness across physical properties. In addition to predicting property, it is essential to assess the reliability of the ML model through uncertainty evaluation. While uncertainty on an estimated population variable are commonly reported for ML models, through quantities like mean absolute error or root mean square error, the uncertainty of each prediction is rarely evaluated. In this talk we address this issue by comparing different ways of determining the error on single predictions for a variety of material properties.