Weston Fulton Professor Sergei Kalinin and his team watched with pride as their newest lab member identified an area of interest within the sample, recalibrated the probe to the optimal imaging parameters, and took another measurement.
It was the first time that ISAACS, a University of Tennessee Knoxville supercomputer, had autonomously conducted a microscopy experiment.
“In terms of simply putting human and artificial intelligence (AI) resources together, companies like Google are light years ahead of academia,” said Kalinin, who works in the UT Department of Materials Science and Engineering (MSE). “But applying machine learning (ML) to real-world problems like scanning probe microscopy (SPM) is a different situation.”
Kalinin uses atomic force microscopy (AFM), a type of SPM, to study novel materials at the nanoscale.
“There is no such thing as a perfectly uniform material,” Kalinin said. “They have grain boundaries, internal defects, and other unpredictable microstructural elements that we need to understand before we apply the materials outside the lab.”
Once scientists identify a performance-impacting defect, they can recommend changes to the material’s fabrication protocol, improving its future efficiency and performance.
Unfortunately, those vital, nanometer-sized features can make up as little as one percent of a material’s surface. Finding all of them usually involves taking detailed measurements on every part of the sample, which is not only exhausting to human operators but slows down the discovery of useful new materials.
“At UT, our colleagues use lab robotics and combinatorial synthesis to make hundreds of materials in the time that it used to take to make one,” Kalinin said, “but that is not enough to accelerate discovery. Development of new materials requires multiple scientists to come up with theories, propose experiments, execute experiments, and analyze results that feed back into our theories. For decades, material analysis has been the bottleneck in this process.”
Finding a way to perform measurements only on regions of interest would significantly accelerate material characterization, stimulating material discovery at an unprecedented rate.
So Kalinin and his colleagues found a way.
A Self-Driving Microscope
A static microscope may seem like an easier piece of equipment to automate than a moving car. However, while all cars are governed by similar safety and traffic laws, each operator on a microscope can have a different experimental priority.
“The way a microscopy experiment progresses depends on the experience, goals, and prior training of the operator,” explained Richard Liu, a postdoctoral researcher in Kalinin’s lab who studied quantum materials during his doctoral research at Harvard. “A microscopist studying high-key dielectrics for semiconductors will have very different goals than someone looking at lithium-ion batteries.”
Kalinin’s PhD student Utkarsh Pratiush brought two years of industrial AI experience to the project.
“The struggle here was making sure that the ML agent makes as good or better decisions than a human agent in a way that allows different teams to accomplish their specific goals,” Pratiush said.
The new microscopes and control schemes needed for such a project are typically not feasible for labs outside the private sector. However, with support from the US Department of Energy’s Energy Frontier Research Center for 3D Ferroelectric Microelectronics, Kalinin’s team was able to launch a new workflow integrating cutting edge technology, industry, and academia.
Working with graduate student Utkarsh Pratiush, Liu began integrating UT’s AFM, Jupiter, with the university’s supercomputer, the Infrastructure for Scientific Applications and Advanced Computing (ISAAC). Liu and Pratiush collaborated with members of UT’s Higher Performance and Scientific Computing office and Jupiter’s manufacturer, Oxford Instruments Asylum Research Inc., over several months to achieve the integration.
On February 2, 2024, the new ML agent correctly identified its first area of interest on a material sample. Since then, Liu and Pratiush have employed the agent on a broad range of workflows ranging from simple feature discovery to image condition optimization and deep kernel learning, gathering useful data up to 100 times faster than can be achieved with the conventional total coverage method.
“This is the strength of a university like UT: being able to bring together the people who do the instrumentation, modify the instrumentation, and make an effective workflow as a team,” said Kalinin.
Kalinin’s team is committed to making the workflow publicly accessible and has already started broadening its application. With support from UT’s AI Tennessee Initiative, MSE Professor Gerd Duscher and his graduate student Austin Houston led the team in implementing the workflow on Duscher’s scanning transmission electron microscope.
“Our success was dependent on other people’s publicly available work, and now we want to turn around and make this automated SPM agent widely available to close the material characterization bottleneck,” Kalinin said. “The key in any field is doing work that is not just useful as abstract knowledge but allows someone else to build upon it.”
Contact
Izzie Gall (865-974-7203, egall4@utk.edu)