Ptychography permits imaging macroscopic specimens at nanometer wavelength resolutions while retrieving chemical, magnetic or atomic information about the sample. It is a remarkably robust technique for the characterization of nano materials, being currently used in a variety of scientific fields. The main challenge of ptychography resides in solving a phase retrieval problem in order to retrieve a reconstruction of the imaged specimen. However, the end-to-end reconstruction normally also involves post-phase retrieval operations, e.g. segmentation, denoising, or super resolution. Currently, machine learning techniques are scarcely used on ptychographic reconstruction pipelines on DOE synchrotron facilities. This talk will present the challenges, state of the art, and preliminary results on the use of machine learning techniques for X-ray ptychography reconstruction. Exploiting such techniques has the potential to enable smart-, sparse-scanning and analysis, which would reduce the acquired data by orders of magnitude while heavily reducing the computational cost of an end-to-end ptychographic experiment.