Skip to content Skip to main navigation Report an accessibility issue

Dr. Sergei Kalinin, October 28

Discovery of Hidden Physics in (Automated) Scanning Probe and Electron Microscopy


Machine learning and artificial intelligence (ML/AI) are rapidly becoming an indispensable part of research in physics and materials science, with applications ranging from theory and materials prediction to high-throughput data analysis. In parallel, over the last decade advances in physical imaging tools including scanning probe and electron microscopies and chemical and optical imaging have opened the floodgates of high-veracity data on materials structure and functionality. Yet physical insights derived from these data are still very limited, with the analysis often limited to purely human-based interpretations and hypothesis testing.

I will discuss recent progress and opportunities in physics discovery from the microscopic observations. Until now, the primary limiting factor has been the correlative nature of classical deep learning (DL) methods, limiting their applicability and universality. I will illustrate the potential of DL for discovering structures and structure-property relationships with imposed inferential biases encoding prior knowledge, including Bayesian priors, and transition from correlative to generative physical models. I pose that the ultimate goal of observational studies is the discovery of causal physical mechanisms, describing the emergence of non-equilibrium and non-ergodic systems via reconstruction of the corresponding generative mechanisms.

I will delineate the vision for transition from post-experiment data analysis to active learning in imaging. Here, the recent successes in applying ML/AI methods for autonomous systems from robotics through self-driving cars to organic and inorganic synthesis are generating enthusiasm for the potential of these techniques to enable automated and autonomous experiment (AE). Here, the strategies based on simple Gaussian Processes (GP) often tend to produce sub-optimal results due to the lack of prior knowledge and very simplified (via learned kernel function) representation of spatial complexity of the system. Comparatively, deep kernel learning (DKL) and structured GP methods allow to realize both the exploration of complex systems towards the discovery of structure-property relationship, and enable automated experiment targeting physics (rather than simple spatial feature) discovery. Notably, many of these strategies are universal and can be applied in fields such materials synthesis or ML-assisted modelling and computation.


Sergei Kalinin is a corporate fellow and a group leader at the Center for Nanophase Materials Sciences at Oak Ridge National Laboratory. He received his MS degree from Moscow State University in 1998 and Ph.D. from the University of Pennsylvania (with Dawn Bonnell) in 2002. His research presently focuses on the applications of big data and artificial intelligence methods in atomically resolved imaging by scanning transmission electron microscopy and scanning probes for applications including physics discovery and atomic fabrication, as well as mesoscopic studies of electrochemical, ferroelectric, and transport phenomena via scanning probe microscopy.