---

First World Congress on Artificial Intelligence in Materials and Manufacturing (AIM 2022): Computer Vision for Materials and Manufacturing R&D II
Program Organizers: Taylor Sparks, University of Utah; Michael Dawson-Haggerty, Kerfed, Inc.; Elizabeth Holm, University of Michigan; Jin Kocsis, Purdue University; Adam Kopper, Mercury Marine; Benji Maruyama; James Warren, National Institute of Standards and Technology

Monday 1:30 PM
April 4, 2022
Room: 3' Rivers
Location: Omni William Penn Hotel

---

1:30 PM  
Use of Computer Vision to Characterize Non-Metallic Inclusions in Steel: Nan Gao¹; Mohammad Abdulsalam¹; Elizabeth Holm¹; Bryan Webler¹; ¹Carnegie Mellon University
    Non-metallic inclusions are small oxide, sulfide, or nitride particles that have numerous effects on the processing and properties of steel. Inclusions arise during liquid steel processing and their control is an important objective of steel refining. Characterization of inclusions is typically performed with automated scanning electron microscopy plus energy dispersive x-ray spectroscopy (SEM/EDS). In this work we used computer vision and machine learning methods to classify SEM images of inclusions by composition. Use of images alone could reduce the need for EDS measurements of inclusion composition. Results from a random forest algorithm and a convolutional neural network (CNN) were compared. Both methods were able to distinguish images of inclusions from non-inclusions. Prediction accuracy decreased as the number of composition classes were increased. The CNN method exhibited better performance than the random forest method.

1:50 PM  
Classification of Material Defects in Ni-Base Superalloys Using Deep Learning: Yann Schöbel¹; Simon Pfingstl²; Markus Kolb¹; Marco Hüller¹; ¹MTU Aero Engines AG; ²Technical University of Munich
    Rotor discs in aircraft engines are exposed to high stresses and temperatures. The burst of rotor discs is one of the catastrophic failure modes of gas turbine engines. Therefore, these critical components must be carefully inspected to identify material defects before commissioning. One of these inspection methods is the macro etch process, which visualizes anomalies on the surface of the parts. These findings are difficult to distinguish and experience is needed to reliably assign them to a specific defect type. In this approach, optical microscopic images of the detected material defects are classified automatically by a Deep Learning Image Classification model which reaches a test set accuracy of 81% while misclassifying only 2.3% of the most critical findings into the class of harmless defects. Compared to human experts, a similar error rate is achieved while the objectivity of the process is increased and more consistent results are obtained.

2:10 PM  
Comparing Transfer Learning to Feature Optimization in Microstructure Classification: Debanshu Banerjee¹; Taylor Sparks²; ¹Jadavpur University; ²University of Utah
    Human analysis of research data is slow and inefficient. In recent years machine learning tools have advanced our capability to perform tasks normally carried out by humans, such as image segmentation and classification. In this work, we seek to further improve binary classification models for high throughput identification of different microstructural morphologies. We utilize a dataset with limited observations (133 dendritic structures, 444 non-dendritic) and employ data augmentation via rotation and translation to enhance the dataset six-fold. Then, transfer learning is carried out using pre-trained networks VGG16, InceptionV3, and Xception achieving only moderate F1 scores (0.801 to 0.822). We hypothesize that feature engineering could yield better results than transfer learning alone. To test this, we employ a new nature-inspired feature optimization algorithm, the Binary Red Deer Algorithm (BRDA), to carry out binary classification and observe F1 scores in the range of 0.96.

2:30 PM Break

3:00 PM  
Efficient Microstructure Image Segmentation Using Deep Learning with Low-Cost Data Annotations: Bo Lei¹; Elizabeth Holm¹; ¹Carnegie Mellon University
    Image segmentation plays a central role in quantitative analysis of material microstructures. Developing efficient and effective methods for automating the segmentation process is highly valued for materials research and manufacturing. Deep neural network methods have recently demonstrated great performance in identifying different constituents in complicated microstructure datasets. However, a considerable amount of data annotations is necessary to get effective solutions while collecting high quality annotations for materials images is difficult and time-consuming. Here, we explored two strategies that can significantly reduce the amount of annotations: (1) use few image-level annotations, (2) use pixel-level scribbles. To maintain a good segmentation performance, transfer learning, data augmentation and continuity loss function were applied. The selection of images or regions to annotate was crucial in this annotation frugal segmentation pipeline.

3:20 PM  
Microscopy Segmentation Models with Transfer Learning from a Large Microscopy Dataset: Joshua Stuckner; ¹
    A transfer learning approach for the automatic segmentation of microscopy data is presented. Many encoder architectures, including VGG, Inception, ResNet, and others, were trained on ~100,000 microscopy images from 54 material classes to create pre-trained models with learned representations that are more relevant to downstream microscopy analysis tasks than models pre-trained on natural images. The pre-trained encoders were embedded into segmentation architectures including U-Net and DeepLabV3+ to evaluate the performance of models pre-trained on a large microscopy dataset. Each encoder/decoder pair was evaluated on several benchmark datasets. Our testing shows that models pre-trained on a large microscopy dataset generalize better to out-of-distribution data (micrographs taken under different imaging or sample conditions) and are more accurate when training data is limited than models pre-trained on ImageNet. Finally, the value of this method is demonstrated by developing two software tools to automatically quantify microstructure features of Ni-superalloys and environmental barrier coatings.

3:40 PM  
Automated Defect Identification in Electroluminescence Images of Solar Modules: Xin Chen¹; Todd Karin¹; Anubhav Jain¹; ¹Lawrence Berkeley National Laboratory
    Solar modules used in field may suffer from degradation caused by defects, which can be detected with electroluminescence (EL) imaging. However, it is not feasible to analyze millions of EL images manually. Therefore, we develop an automatic pipeline to analyze EL images and detect various defect types including cracks, intra-cell defects, oxygen-induced defects and solder disconnection. We train neural networks including ResNet18, ResNet50, ResNet152 and YOLO models with 896 EL images of solar modules, and determine that ResNet18 and YOLO are the best-performing models with macro f1 scores of 0.83 for ResNet18 and 0.78 for YOLO on the testing set(129 images of modules). We provide a detailed analysis of the selection of the models based on users’ demands. Using a field EL survey of a PV power plant damaged in a vegetation fire, we analyze over 18,000 EL images (2.4 million cells) and inspect the spatial distribution of defects on the solar modules.

4:00 PM  
AMPIS: Automated Materials Particle Instance Segmentation: Ryan Cohn¹; Elizabeth Holm¹; ¹Carnegie Mellon University
    We present AMPIS, an open-source framework designed to make instance segmentation more accessible to materials scientists. Instance segmentation generates individual segmentation masks for every recognized object in an image. This facilitates the characterization of materials with multiple components from images, automating image processing and enabling new characterization techniques. AMPIS was originally developed to measure satellites, which influence powder rheology but cannot be experimentally characterized, on additive manufacturing feedstock powders. Leveraging transfer learning with Mask R-CNN enabled instance segmentation of individual powder particles and satellites from images, despite only having a small labeled dataset to train the model with. The results demonstrated the ability of this approach to generate the first quantitative, repeatable measurements of satellites in metal powders. However, the benefits of instance segmentation and AMPIS are not limited to powder characterization. AMPIS is a flexible tool and may be applied to a variety of material systems and applications.

---