Artificial intelligence and machine learning are enabling automation of decision-making in various scientific domains, but still face a number of fundamental obstacles in materials science. We provide an overview of one such platform, Computational Autonomy for Materials Discovery (CAMD), designed to help materials scientists simulate and design their discovery processes using machine learning tools. CAMD has specifically been engineered to maximize the likelihood that sequential iterations of an experimental or simulation-based workflow will produce materials data with target properties. To date, CAMD's primary application is in the prediction of new, phase-stable crystal structures from structural prototypes in various chemical spaces. In addition, we have begun designing multi-fidelity sequential learning agents using data streams from experiment and theory. We review these capabilities with a view towards the future of AI-assisted tools for materials discovery.