The past century has seen a steady increase in the use of statistical science in the design of materials. Real-world materials design problems typically involve large-dimensional optimizations on complex models with nonlinear constraints, incomplete information, and potentially very limited data. Most statistical tools, however, are built for linear models, with few constraints, and require a critical amount of data. Worse yet, problems formulated in the design of materials under uncertainty generally have solutions that are rare-events in terms of the statistical distribution of potential materials states, and thus many of the standard methods cannot be directly applied to problems such as structure determination. Thus, although computers have enabled numerical evaluation of increasingly sophisticated materials models, these models are still designed by hand by multi-disciplinary teams of physicists, computer scientists, and statisticians. Can we do better? More importantly, what tools are needed to enable rigorous, automated, statistical design of materials?