ABSTRACT Scanning transmission electron microscopy (STEM) has emerged as a uniquely powerful tool for structural and functional imaging of materials on the atomic level. Driven by advances

in aberration correction, STEM now allows the routine imaging of structures with single-digit picometre-level precision for localization of atomic units. This Primer focuses on the

opportunities emerging at the interface between STEM and machine learning (ML) methods. We review the primary STEM imaging methods, including structural imaging, electron energy loss

spectroscopy and its momentum-resolved modalities and 4D-STEM. We discuss the quantification of STEM structural data as a necessary step towards meaningful ML applications and its analysis

of ML methods in the STEM community, including the storage of data and metadata to allow the reproduction of experiments. Finally, we discuss the application of ML to automating experiments

and novel scanning modes. Access through your institution Buy or subscribe This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your

institution Access Nature and 54 other Nature Portfolio journals Get Nature+, our best-value online-access subscription $29.99 / 30 days cancel any time Learn more Subscribe to this journal

Receive 1 digital issues and online access to articles $119.00 per year only $119.00 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy

now Prices may be subject to local taxes which are calculated during checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact

customer support SIMILAR CONTENT BEING VIEWED BY OTHERS UNSUPERVISED DEEP DENOISING FOR FOUR-DIMENSIONAL SCANNING TRANSMISSION ELECTRON MICROSCOPY Article Open access 13 October 2024 ATOMAI

FRAMEWORK FOR DEEP LEARNING ANALYSIS OF IMAGE AND SPECTROSCOPY DATA IN ELECTRON AND SCANNING PROBE MICROSCOPY Article 08 December 2022 UNCOVERING MATERIAL DEFORMATIONS VIA MACHINE LEARNING

COMBINED WITH FOUR-DIMENSIONAL SCANNING TRANSMISSION ELECTRON MICROSCOPY Article Open access 18 May 2022 REFERENCES * Keyse, R. J., Garratt-Reed, A. J., Goodhew, P. J. & Lorimer, G. W.

_Introduction to Scanning Transmission Electron Microscopy_ (Routledge, 2018). * Shibata, N. et al. Electric field imaging of single atoms. _Nat. Commun._ 8, 15631 (2017). Article  ADS 

Google Scholar  * Campanini, M., Nasi, L., Albertini, F. & Erni, R. Disentangling nanoscale electric and magnetic fields by time-reversal operation in differential phase-contrast STEM.

_Appl. Phys. Lett._ 117, 154102 (2020). Article  ADS  Google Scholar  * Müller, K. et al. Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction. _Nat.

Commun._ 5, 5653 (2014). Article  ADS  Google Scholar  * Nellist, P. D., McCallum, B. C. & Rodenburg, J. M. Resolution beyond the information limit in transmission electron-microscopy.

_Nature_ 374, 630–632 (1995). Article  ADS  Google Scholar  * Hwang, J., Zhang, J. Y., D’Alfonso, A. J., Allen, L. J. & Stemmer, S. Three-dimensional imaging of individual dopant atoms

in SrTiO3. _Phys. Rev. Lett._ 111, 266101 (2013). Article  ADS  Google Scholar  * Xin, H. L. & Muller, D. A. Aberration-corrected ADF-STEM depth sectioning and prospects for reliable 3D

imaging in S/TEM. _J. Electron. Microsc._ 58, 157–165 (2009). Article  Google Scholar  * Benthem, K. V. et al. Three-dimensional imaging of individual hafnium atoms inside a semiconductor

device. _Appl. Phys. Lett._ 87, 034104 (2005). Article  ADS  Google Scholar  * Borisevich, A. Y., Lupini, A. R. & Pennycook, S. J. Depth sectioning with the aberration-corrected scanning

transmission electron microscope. _Proc. Natl Acad. Sci. USA_ 103, 3044–3048 (2006). Article  ADS  Google Scholar  * Nellist, P. D. & Wang, P. Optical sectioning and confocal imaging

and analysis in the transmission electron microscope. _Annu. Rev. Mater. Res._ 42, 125–143 (2012). Article  ADS  Google Scholar  * Wang, P. et al. Bright-field scanning confocal electron

microscopy using a double aberration-corrected transmission electron microscope. _Ultramicroscopy_ 111, 877–886 (2011). Article  Google Scholar  * Ishikawa, R. et al. Single atom visibility

in STEM optical depth sectioning. _Appl. Phys. Lett._ 109, 163102 (2016). Article  ADS  Google Scholar  * Levin, B. D. A. et al. Nanomaterial datasets to advance tomography in scanning

transmission electron microscopy. _Sci. Data_ 3, 160041 (2016). Article  Google Scholar  * Alania, M. et al. Depth sectioning combined with atom-counting in HAADF STEM to retrieve the 3D

atomic structure. _Ultramicroscopy_ 177, 36–42 (2017). Article  Google Scholar  * Dahmen, T. et al. Combined scanning transmission electron microscopy tilt- and focal series. _Microsc.

Microanal._ 20, 548–560 (2014). Article  ADS  Google Scholar  * Hovden, R. et al. Breaking the Crowther limit: combining depth-sectioning and tilt tomography for high-resolution, wide-field

3D reconstructions. _Ultramicroscopy_ 140, 26–31 (2014). Article  Google Scholar  * Krivanek, O. L. et al. Vibrational spectroscopy in the electron microscope. _Nature_ 514, 209–212 (2014).

Article  ADS  Google Scholar  * Idrobo, J. C. et al. Temperature measurement by a nanoscale electron probe using energy gain and loss spectroscopy. _Phys. Rev. Lett._ 120, 095901 (2018).

Article  ADS  Google Scholar  * Bonnet, N. Multivariate statistical methods for the analysis of microscope image series: applications in materials science. _J. Microscopy_ 190, 2–18 (1998).

Article  Google Scholar  * Trebbia, P. & Bonnet, N. EELS elemental mapping with unconventional methods I. Theoretical basis: image analysis with multivariate statistics and entropy

concepts. _Ultramicroscopy_ 34, 165–178 (1990). Article  Google Scholar  * Trebbia, P. & Mory, C. EELS elemental mapping with unconventional methods II. Applications to biological

specimens. _Ultramicroscopy_ 34, 179–203 (1990). Article  Google Scholar  * Dan, J., Zhao, X. & Pennycook, S. J. A machine perspective of atomic defects in scanning transmission electron

microscopy. _InfoMat_ 1, 359–375 (2019). Article  Google Scholar  * Maksov, A. et al. Deep learning analysis of defect and phase evolution during electron beam-induced transformations in

WS2. _npj Comput. Mater._ 5, 12 (2019). Article  ADS  Google Scholar  * Roccapriore, K. M., Kalinin, S. V. & Ziatdinov, M. Physics discovery in nanoplasmonic systems via autonomous

experiments in scanning transmission electron microscopy. Preprint at https://arxiv.org/abs/2108.03290 (2021). * Susi, T. et al. Towards atomically precise manipulation of 2D nanostructures

in the electron microscope. _2D Mater._ 4, 042004 (2017). Article  Google Scholar  * Dyck, O., Kim, S., Kalinin, S. V. & Jesse, S. Placing single atoms in graphene with a scanning

transmission electron microscope. _Appl. Phys. Lett._ 111, 113104 (2017). Article  ADS  Google Scholar  * Dyck, O. et al. Electron-beam introduction of heteroatomic Pt–Si structures in

graphene. _Carbon_ 161, 750–757 (2020). Article  Google Scholar  * Oxley, M. P., Lupini, A. R. & Pennycook, S. J. Ultra-high resolution electron microscopy. _Rep. Prog. Phys._ 80, 026101

(2016). Article  ADS  Google Scholar  * Loth, S., Etzkorn, M., Lutz, C. P., Eigler, D. M. & Heinrich, A. J. Measurement of fast electron spin relaxation times with atomic resolution.

_Science_ 329, 1628–1630 (2010). Article  ADS  Google Scholar  * Giessibl, F. J. AFM’s path to atomic resolution. _Mater. Today_ 8, 32–41 (2005). Article  Google Scholar  * Park, J. Y.,

Maier, S., Hendriksen, B. & Salmeron, M. Sensing current and forces with SPM. _Mater. Today_ 13, 38–45 (2010). Article  Google Scholar  * Abbe, E. Beiträge zur Theorie des Mikroskops und

der mikroskopischen Wahrnehmung. _Arch. Mikrosk. Anat._ 9, 413–468 (1873). Article  Google Scholar  * Scherzer, O. Uber einige Fehler von Elektonenlinsen. _Zeit. Phys._ 101, 593–603 (1936).

Article  ADS  MATH  Google Scholar  * Hawkes, P. W. in _Scienc_e _of Microscopy_ (eds Hawkes, P. W. & Spence, J. C.) 696–747 (Springer, 2007). * Dellby, N., Krivanek, O. L., Nellist, P.

D., Batson, P. E. & Lupini, A. R. Progress in aberration-corrected scanning transmission electron microscopy. _J. Electron. Microsc._ 50, 177–185 (2001). Google Scholar  * Krivanek, O.

L., Dellby, N. & Lupini, A. R. Towards sub-ångstrom electron beams. _Ultramicroscopy_ 78, 1–11 (1999). Article  Google Scholar  * Haider, M. et al. A spherical-aberration-corrected 200 

kV transmission electron microscope. _Ultramicroscopy_ 75, 53–60 (1998). Article  Google Scholar  * Haider, M. et al. Electron microscopy image enhanced. _Nature_ 392, 768–769 (1998).

Article  ADS  Google Scholar  * Rose, H. Elektronenoptische Aplanate. _Optik_ 34, 285–311 (1971). Google Scholar  * Zach, J. & Haider, M. Aberration correction in a low voltage SEM by a

multipole corrector. _Nucl. Instrum. Methods Phys. Res. A_ 363, 316–325 (1995). Article  ADS  Google Scholar  * Batson, P. E. Aberration correction results in the IBM STEM instrument.

_Ultramicroscopy_ 96, 239–249 (2003). Article  Google Scholar  * Nellist, P. D. et al. Direct sub-ångstrom imaging of a crystal lattice. _Science_ 305, 1741–1741 (2004). Article  Google

Scholar  * Kisielowski, C. et al. Detection of single atoms and buried defects in three dimensions by aberration-corrected electron microscope with 0.5-ångstrom information limit. _Microsc.

Microanal._ 14, 469–477 (2008). Article  ADS  Google Scholar  * Shibata, N. et al. Differential phase-contrast microscopy at atomic resolution. _Nat. Phys._ 8, 611–615 (2012). Article 

Google Scholar  * Shibata, N. et al. New area detector for atomic-resolution scanning transmission electron microscopy. _J. Electron. Microscopy_ 59, 473–479 (2010). Article  Google Scholar

  * Ophus, C. Four-dimensional scanning transmission electron microscopy (4D-STEM): from scanning nanodiffraction to ptychography and beyond. _Microsc. Microanal._ 25, 563–582 (2019).

Article  ADS  Google Scholar  * Egerton, R. F. _Electron Energy-Loss Spectroscopy in the Electron Microscope_ (Springer, 2011). * Hillier, J. & Baker, R. F. Microanalysis by means of

electrons. _J. Appl. Phys._ 15, 663 (1944). Article  ADS  Google Scholar  * Gubbens, A. J., Kraus, B., Krivanek, O. L. & Mooney, P. An imaging filter for high voltage electron

microscopy. _Ultramicroscopy_ 59, 255–265 (1995). Article  Google Scholar  * Silcox, J. Core-loss EELS. _Curr. Opin. Solid. State Mater. Sci._ 3, 336–342 (1998). Article  ADS  Google Scholar

  * Nelayah, J. et al. Mapping surface plasmons on a single metallic nanoparticle. _Nat. Phys._ 3, 348–353 (2007). Article  Google Scholar  * Zhou, W. et al. Atomically localized plasmon

enhancement in monolayer graphene. _Nat. Nanotechnol._ 7, 161–165 (2012). Article  ADS  Google Scholar  * Suenaga, K. & Koshino, M. Atom-by-atom spectroscopy at graphene edge. _Nature_

468, 1088–1090 (2010). Article  ADS  Google Scholar  * Colliex, C., Kociak, M. & Stéphan, O. Electron energy loss spectroscopy imaging of surface plasmons at the nanometer scale.

_Ultramicroscopy_ 162, A1–A24 (2016). Article  Google Scholar  * El-Sherif, H., Jovanovic, S., Preston, J. & Basim, N. Electron microscopy and interface plasmons characterization of

cadmium telluride thin film grown incommensurately with weak bonding on sapphire. _Microsc. Microanal._ 26, 3116–3118 (2020). Article  ADS  Google Scholar  * Yang, H., Garfunkel, E. L. &

Batson, P. E. Probing free carrier plasmons in doped semiconductors using spatially resolved electron energy loss spectroscopy. _Phys. Rev. B_ 102, 205427 (2020). Article  ADS  Google

Scholar  * Bonnet, N., Brun, N. & Colliex, C. Extracting information from sequences of spatially resolved EELS spectra using multivariate statistical analysis. _Ultramicroscopy_ 77,

97–112 (1999). Article  Google Scholar  * Jeanguillaume, C. & Colliex, C. Spectrum-image: the next step in EELS digital acquisition and processing. _Ultramicroscopy_ 28, 252–257 (1989).

Article  Google Scholar  * Polman, A., Kociak, M. & García de Abajo, F. J. Electron-beam spectroscopy for nanophotonics. _Nat. Mater._ 18, 1158–1171 (2019). Article  ADS  Google Scholar

  * Hachtel, J., Lupini, A. & Idrobo, J. Exploring the capabilities of monochromated electron energy loss spectroscopy in the infrared regime. _Sci. Rep._ 8, 5637 (2018). Article  ADS 

Google Scholar  * Krivanek, O. et al. Towards sub-10 meV energy resolution STEM-EELS. _J. Phys.Conf. Ser._ 522, 012023 (2014). Article  Google Scholar  * Hage, F. S. et al. Nanoscale

momentum-resolved vibrational spectroscopy. _Sci. Adv._ 4, eaar7495 (2018). Article  ADS  Google Scholar  * Hage, F. S., Radtke, G., Kepaptsoglou, D. M., Lazzeri, M. & Ramasse, Q. M.

Single-atom vibrational spectroscopy in the scanning transmission electron microscope. _Science_ 367, 1124 (2020). Article  ADS  Google Scholar  * Lagos, M. J., Trügler, A., Hohenester, U.

& Batson, P. E. Mapping vibrational surface and bulk modes in a single nanocube. _Nature_ 543, 529–532 (2017). Article  ADS  Google Scholar  * Boersch, H., Geiger, J. & Stickel, W.

Interaction of 25-keV electrons with lattice vibrations in LiF. Experimental evidence for surface modes of lattice vibration. _Phys. Rev. Lett._ 17, 379–381 (1966). Article  ADS  Google

Scholar  * Egerton, R. F. Electron energy-loss spectroscopy in the TEM. _Rep. Prog. Phys._ 72, 25 (2009). Article  Google Scholar  * Leapman, R. D., Fejes, P. L. & Silcox, J. Orientation

dependence of core edges from anisotropic materials determined by inelastic-scattering of fast electrons. _Phys. Rev. B_ 28, 2361–2373 (1983). Article  ADS  Google Scholar  * Botton, G. A.

A new approach to study bonding anisotropy with EELS. _J. Electron. Spectrosc. Relat. Phenom._ 143, 129–137 (2005). Article  Google Scholar  * Hage, F. S. et al. Topologically induced

confinement of collective modes in multilayer graphene nanocones measured by momentum-resolved STEM-VEELS. _Phys. Rev. B_ 88, 12 (2013). Article  Google Scholar  * Schattschneider, P.

Exchange of angular momentum in EMCD experiments. _Ultramicroscopy_ 109, 91–95 (2008). Article  Google Scholar  * Wang, Y. Y. et al. Evolution of the low-energy excitations and dielectric

function of Ba1-_x_K_x_BiO3 (0 <= _x_ <= 0.50). _Phys. Rev. B_ 47, 14503–14509 (1993). Article  ADS  Google Scholar  * Liou, S. C. et al. Plasmons dispersion and nonvertical interband

transitions in single crystal Bi2Se3 investigated by electron energy-loss spectroscopy. _Phys. Rev. B_ 87, 6 (2013). Article  Google Scholar  * Schuster, R., Kraus, R., Knupfer, M., Berger,

H. & Buchner, B. Negative plasmon dispersion in the transition-metal dichalcogenide 2H-TaSe2. _Phys. Rev. B_ 79, 5 (2009). Article  Google Scholar  * Gloter, A., Chu, M. W., Kociak, M.,

Chen, C. H. & Colliex, C. Probing non-dipole allowed excitations in highly correlated materials with nanoscale resolution. _Ultramicroscopy_ 109, 1333–1337 (2009). Article  Google

Scholar  * Senga, R. et al. Position and momentum mapping of vibrations in graphene nanostructures. _Nature_ 573, 247 (2019). Article  ADS  Google Scholar  * Egerton, R. F. Limits to the

spatial, energy and momentum resolution of electron energy-loss spectroscopy. _Ultramicroscopy_ 107, 575–586 (2007). Article  Google Scholar  * Wang, Y. Y., Cheng, S. C., Dravid, V. P. &

Zhang, F. C. Momentum-transfer resolved electron-energy-loss spectroscopy of solids — problems, solutions and applications. _Ultramicroscopy_ 59, 109–119 (1995). Article  Google Scholar  *

Plotkin-Swing, B. et al. Hybrid pixel direct detector for electron energy loss spectroscopy. _Ultramicroscopy_ 217, 10 (2020). Article  Google Scholar  * Pogany, A. P. & Turner, P. S.

Reciprocity in electron diffraction and microscopy. _Acta Crystallogr. A_ 24, 103–109 (1968). Article  ADS  Google Scholar  * Frank, J. Envelope of electron-microscopic transfer-functions

for partially coherent illumination. _Optik_ 38, 519–536 (1973). Google Scholar  * Cowley, J. M. Electron nanodiffraction. _Microsc. Res. Tech._ 46, 75–97 (1999). Article  Google Scholar  *

Cowley, J. M. Adjustment of a STEM instrument by use of shadow images. _Ultramicroscopy_ 4, 413–418 (1979). Article  Google Scholar  * Cowley, J. M. High-resolution electron-microscopy and

microdiffraction. _Ultramicroscopy_ 18, 11–17 (1985). Article  Google Scholar  * Cowley, J. M. in _Advances in Electronics and Electron Physics_ Vol. 46 1–53 (Elsevier, 1978). * Ronchi, V.

Forty years of history of a grating interferometer. _Appl. Opt._ 3, 437–451 (1964). Article  ADS  Google Scholar  * Lupini, A. R. & Pennycook, S. J. Rapid autotuning for crystalline

specimens from an inline hologram. _J. Electron. Microsc._ 57, 195–201 (2008). Google Scholar  * Lupini, A. R., Wang, P., Nellist, P. D., Kirkland, A. I. & Pennycook, S. J. Aberration

measurement using the Ronchigram contrast transfer function. _Ultramicroscopy_ 110, 891–898 (2010). Article  Google Scholar  * Sawada, H., Allen, C. S., Wang, S., Warner, J. H. &

Kirkland, A. I. Aberration measurement of the probe-forming system of an electron microscope using two-dimensional materials. _Ultramicroscopy_ 182, 195–204 (2017). Article  Google Scholar 

* Ramasse, Q. M. Twenty years after: how “aberration correction in the STEM” truly placed a “a synchrotron in a microscope”. _Ultramicroscopy_ 180, 41–51 (2017). Article  Google Scholar  *

Ramasse, Q. M. & Bleloch, A. L. Diagnosis of aberrations from crystalline samples in scanning transmission electron microscopy. _Ultramicroscopy_ 106, 37–56 (2005). Article  Google

Scholar  * Sawada, H. et al. Measurement method of aberration from Ronchigram by autocorrelation function. _Ultramicroscopy_ 108, 1467–1475 (2008). Article  Google Scholar  * Lin, J. A.

& Cowley, J. M. Calibration of the operating parameters for an HB5 stem instrument. _Ultramicroscopy_ 19, 31–42 (1986). Article  Google Scholar  * Dwyer, C., Erni, R. & Etheridge, J.

Method to measure spatial coherence of subångstrom electron beams. _Appl. Phys. Lett_. 93, 021115 (2008). * Dwyer, C., Erni, R. & Etheridge, J. Measurement of effective source

distribution and its importance for quantitative interpretation of STEM images. _Ultramicroscopy_ 110, 952–957 (2010). Article  Google Scholar  * Lupini, A. R., Chi, M. & Jesse, S. Rapid

aberration measurement with pixelated detectors. _J. Microsc._ 263, 43–50 (2016). Article  Google Scholar  * Lupini, A. R. et al. Fast aberration measurement in multi-dimensional STEM.

_Microsc. Microanal._ 22, 252–253 (2016). Article  Google Scholar  * Pennycook, T. J. et al. Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: experimental

demonstration at atomic resolution. _Ultramicroscopy_ 151, 160–167 (2015). Article  Google Scholar  * Krajnak, M. & Etheridge, J. A symmetry-derived mechanism for atomic resolution

imaging. _Proc. Natl Acad. Sci. USA_ 117, 27805–27810 (2020). Article  ADS  Google Scholar  * Dekkers, N. H. & de Lqng, H. Differential phase contrast in a STEM. _Optik_ 41, 452–456

(1974). Google Scholar  * Lazić, I., Bosch, E. G. T. & Lazar, S. Phase contrast STEM for thin samples: integrated differential phase contrast. _Ultramicroscopy_ 160, 265–280 (2016).

Article  Google Scholar  * Tsuda, K. & Tanaka, M. Direct observation of the symmetry breaking of the nanometer-scale local structure in the paraelectric cubic phase of BaTiO3 using

convergent-beam electron diffraction. _Appl. Phys. Express_ 9, 071501 (2016). * Jesse, S. et al. Big data analytics for scanning transmission electron microscopy ptychography. _Sci. Rep._ 6,

26348 (2016). Article  ADS  Google Scholar  * Nord, M. et al. Strain anisotropy and magnetic domains in embedded nanomagnets. _Small_ 15, 6 (2019). Google Scholar  * Pekin, T. C., Gammer,

C., Ciston, J., Minor, A. M. & Ophus, C. Optimizing disk registration algorithms for nanobeam electron diffraction strain mapping. _Ultramicroscopy_ 176, 170–176 (2017). Article  Google

Scholar  * Grieb, T. et al. Strain analysis from nano-beam electron diffraction: influence of specimen tilt and beam convergence. _Ultramicroscopy_ 190, 45–57 (2018). Article  Google Scholar

  * Han, Y. M. et al. Strain mapping of two-dimensional heterostructures with subpicometer precision. _Nano Lett._ 18, 3746–3751 (2018). Article  ADS  Google Scholar  * Nord, M. et al.

Three-dimensional subnanoscale imaging of unit cell doubling due to octahedral tilting and cation modulation in strained perovskite thin films. _Phys. Rev. Mater._ 3, 063605 (2019). Article

  Google Scholar  * Hoppe, W. Trace structure analysis, ptychography, phase tomography. _Ultramicroscopy_ 10, 187–198 (1982). Article  Google Scholar  * Gerchbeg, R. W. & Saxton, W. O. A

practical algorithm for the determination of phase from image and diffraction plane pictures. _Optik_ 35, 227–246 (1972). Google Scholar  * Cowley, J. M. Comments on ultra-high-resolution

STEM. _Ultramicroscopy_ 87, 1–4 (2001). Article  Google Scholar  * Nellist, P. D. & Rodenburg, J. M. Beyond the conventional information limit — the relevant coherence function.

_Ultramicroscopy_ 54, 61–74 (1994). Article  Google Scholar  * Rodenburg, J. M. & Bates, R. H. T. The theory of superresolution electron-microscopy via Wigner-distribution deconvolution.

_Phil. Trans. R. Soc. Lond. A_ 339, 521–553 (1992). Article  ADS  Google Scholar  * Lupini, A. R., Oxley, M. P. & Kalinin, S. V. Pushing the limits of electron ptychography. _Science_

362, 399 (2018). Article  ADS  Google Scholar  * Yang, H. et al. Simultaneous atomic-resolution electron ptychography and _Z_-contrast imaging of light and heavy elements in complex

nanostructures. _Nat. Commun._ 7, 12532 (2016). Article  ADS  Google Scholar  * Van Dyck, D., Jinschek, J. R. & Chen, F.-R. ‘Big Bang’ tomography as a new route to atomic-resolution

electron tomography. _Nature_ 486, 243–246 (2012). Article  ADS  Google Scholar  * Verbeeck, J., Tian, H. & Schattschneider, P. Production and application of electron vortex beams.

_Nature_ 467, 301–304 (2010). Article  ADS  Google Scholar  * McMorran, B. J. et al. Electron vortex beams with high quanta of orbital angular momentum. _Science_ 331, 192–195 (2011).

Article  ADS  Google Scholar  * Grillo, V. et al. Generation of nondiffracting electron Bessel beams. _Phys. Rev. X_ 4, 011013 (2014). Google Scholar  * Uchida, M. & Tonomura, A.

Generation of electron beams carrying orbital angular momentum. _Nature_ 464, 737–739 (2010). Article  ADS  Google Scholar  * Rotunno, E. et al. Electron-beam shaping in the transmission

electron microscope: control of electron-beam propagation along atomic columns. _Phys. Rev. Appl._ 11, 044072 (2019). Article  ADS  Google Scholar  * Nguyen, D. T., Findlay, S. D. &

Etheridge, J. A menu of electron probes for optimising information from scanning transmission electron microscopy. _Ultramicroscopy_ 184, 143–155 (2018). Article  Google Scholar  * Zheng, C.

L. et al. Axicon lens for electrons using a magnetic vortex: the efficient generation of a Bessel beam. _Phys. Rev. Lett._ 119, 174801 (2017). Article  ADS  Google Scholar  * Verbeeck, J.

et al. Demonstration of a 2×2 programmable phase plate for electrons. _Ultramicroscopy_ 190, 58–65 (2018). Article  Google Scholar  * Tavabi, A. H. et al. Generation of electron vortices

using nonexact electric fields. _Phys. Rev. Res._ 2, 013185 (2020). Article  Google Scholar  * Tavabi, A. H. et al. Experimental demonstration of an electrostatic orbital angular momentum

sorter for electrons. _Phys. Rev. Lett._ 126 094802 (2021). Article  ADS  Google Scholar  * Grillo, V. et al. Observation of nanoscale magnetic fields using twisted electron beams. _Nat.

Commun._ 8, 689 (2017). Article  Google Scholar  * Edstrom, A., Lubk, A. & Rusz, J. Quantum mechanical treatment of atomic-resolution differential phase contrast imaging of magnetic

materials. _Phys. Rev. B_ 99, 174428 (2019). * Pohl, D. et al. Atom size electron vortex beams with selectable orbital angular momentum. _Sci. Rep._ 7, 934 (2017). Article  ADS  Google

Scholar  * Rusz, J. & Bhowmick, S. Boundaries for efficient use of electron vortex beams to measure magnetic properties. _Phys. Rev. Lett_. 111, 105504 (2013). * Ophus, C. et al.

Efficient linear phase contrast in scanning transmission electron microscopy with matched illumination and detector interferometry. _Nat. Commun._ 7, 10719 (2016). Article  ADS  Google

Scholar  * Guzzinati, G. et al. Probing the symmetry of the potential of localized surface plasmon resonances with phase-shaped electron beams. _Nat. Commun._ 8, 14999 (2017). Article  ADS 

Google Scholar  * Zeltmann, S. E. et al. Patterned probes for high precision 4D-STEM Bragg measurements. _Ultramicroscopy_ 209, 112890 (2020). Article  Google Scholar  * Guzzinati, G. et al.

Electron Bessel beam diffraction for precise and accurate nanoscale strain mapping. _Appl. Phys. Lett_. 114, 243501 (2019). * Stoger-Pollach, M., Schachinger, T., Biedermann, K. &

Beyer, V. Valence EELS below the limit of inelastic delocalization using conical dark field EFTEM or Bessel beams. _Ultramicroscopy_ 173, 24–30 (2017). Article  Google Scholar  * Nellist, P.

D. & Pennycook, S. J. Incoherent imaging using dynamically scattered coherent electrons. _Ultramicroscopy_ 78, 111–124 (1999). Article  Google Scholar  * Grillo, V. et al. Measuring the

orbital angular momentum spectrum of an electron beam. _Nat. Commun._ 8, 15536 (2017). Article  ADS  Google Scholar  * Rotunno, E. et al. Alignment of electron optical beam shaping elements

using a convolutional neural network. _Ultramicroscopy_ 228, 113338 (2021). Article  Google Scholar  * Shahriari, B., Swersky, K., Wang, Z., Adams, R. P. & Freitas, N. D. Taking the

human out of the loop: a review of Bayesian optimization. _Proc. IEEE_ 104, 148–175 (2016). Article  Google Scholar  * Silver, D. et al. A general reinforcement learning algorithm that

masters chess, shogi, and Go through self-play. _Science_ 362, 1140–1144 (2018). Article  ADS  MathSciNet  MATH  Google Scholar  * Wilkinson, M. D. et al. The FAIR guiding principles for

scientific data management and stewardship. _Sci. Data_ 3, 160018 (2016). Article  Google Scholar  * Collins, T. J. ImageJ for microscopy. _Biotechniques_ 43, 1S (2007). Article  MathSciNet

  Google Scholar  * Roels, J. et al. An interactive ImageJ plugin for semi-automated image denoising in electron microscopy. _Nat. Commun._ 11, 771 (2020). Article  ADS  Google Scholar  *

Somnath, S. et al. USID and Pycroscopy–Open frameworks for storing and analyzing spectroscopic and imaging data. Preprint at https://arxiv.org/abs/1903.09515 (2019). * Clausen, A. et al.

LiberTEM: Software platform for scalable multidimensional data processing in transmission electron microscopy. _J. Open. Source Softw._ 5, 2006 (2020). Article  ADS  Google Scholar  *

Savitzky, B. H. et al. py4DSTEM: Open Source Software for 4D-STEM Data Analysis. _Micros. Microanal._ 25, 124–125 (2019). Article  Google Scholar  * Koch, C. T. Determination of core

structure periodicity and point defect density along dislocations. PhD thesis, Arizona State University https://www.physik.hu-berlin.de/en/sem/images/koch02_phdthesis.pdf (2002). * Allen, L.

J., D’Alfonso, A. J. & Findlay, S. D. Modelling the inelastic scattering of fast electrons. _Ultramicroscopy_ 151, 11–22 (2015). Article  Google Scholar  * Barthel, J. Dr. Probe: a

software for high-resolution STEM image simulation. _Ultramicroscopy_ 193, 1–11 (2018). Article  Google Scholar  * Lobato, I. & Van Dyck, D. MULTEM: A new multislice program to perform

accurate and fast electron diffraction and imaging simulations using graphics processing units with CUDA. _Ultramicroscopy_ 156, 9–17 (2015). Article  Google Scholar  * Oelerich, J. O. et

al. STEMsalabim: a high-performance computing cluster friendly code for scanning transmission electron microscopy image simulations of thin specimens. _Ultramicroscopy_ 177, 91–96 (2017).

Article  Google Scholar  * Madsen, J. & Susi, T. abTEM: ab initio transmission electron microscopy image simulation. _Microsc. Microanal._ 26, 448–450 (2020). Article  ADS  Google

Scholar  * Pryor, A., Ophus, C. & Miao, J. A streaming multi-GPU implementation of image simulation algorithms for scanning transmission electron microscopy. _Adv. Struct. Chem. Imag._

3, 15 (2017). Article  Google Scholar  * Mortensen, J. J., Hansen, L. B. & Jacobsen, K. W. Real-space grid implementation of the projector augmented wave method. _Phys. Rev. B_ 71,

035109 (2005). Article  ADS  Google Scholar  * Sakaguchi, N., Matsumoto, S., Kunisada, Y. & Ueda, M. Interaction of localized surface plasmons of a silver nanosphere dimer embedded in a

uniform medium: scanning transmission electron microscopy electron energy-loss spectroscopy and discrete dipole approximation simulation. _J. Phys. Chem. C_ 123, 6735–6744 (2019). Article 

Google Scholar  * Mendis, B. G. An inelastic multislice simulation method incorporating plasmon energy losses. _Ultramicroscopy_ 206, 112816 (2019). Article  Google Scholar  * Brown, H. G.,

Ciston, J. & Ophus, C. Linear-scaling algorithm for rapid computation of inelastic transitions in the presence of multiple electron scattering. _Phys. Rev. Res._ 1, 033186 (2019).

Article  Google Scholar  * Tessmer, J., Singh, S., Gu, Y., El-Awady, J. A. & Graef, M. D. Scanning transmission electron microscopy image simulations of complex dislocation structures

generated by discrete dislocation dynamics. _Ultramicroscopy_ 219, 113124 (2020). Article  Google Scholar  * Thomas, P. J. & Midgley, P. A. Image-spectroscopy — II. The removal of plural

scattering from extended energy-filtered series by Fourier deconvolution. _Ultramicroscopy_ 88, 187–194 (2001). Article  Google Scholar  * Leapman, R. D., Rez, P. & Mayers, D. F. K, L,

and M shell generalized oscillator strengths and ionization cross sections for fast electron collisions. _J. Chem. Phys._ 72, 1232–1243 (1980). Article  ADS  Google Scholar  * Ahn, C. C.

& Rez, P. Inner shell edge profiles in electron energy loss spectroscopy. _Ultramicroscopy_ 17, 105–115 (1985). Article  Google Scholar  * Cueva, P., Hovden, R., Mundy, J. A., Xin, H. L.

& Muller, D. A. Data processing for atomic resolution electron energy loss spectroscopy. _Microsc. Microanal._ 18, 667–675 (2012). Article  ADS  Google Scholar  * Varela, M. et al.

Atomic-resolution imaging of oxidation states in manganites. _Phys. Rev. B_ 79, 085117 (2009). Article  ADS  Google Scholar  * Verbeeck, J., Van Aert, S. & Bertoni, G. Model-based

quantification of EELS spectra: Including the fine structure. _Ultramicroscopy_ 106, 976–980 (2006). Article  Google Scholar  * Bosman, M. et al. Two-dimensional mapping of chemical

information at atomic resolution. _Phys. Rev. Lett._ 99, 086102 (2007). Article  ADS  Google Scholar  * Bosman, M., Watanabe, M., Alexander, D. & Keast, V. Mapping chemical and bonding

information using multivariate analysis of electron energy-loss spectrum images. _Ultramicroscopy_ 106, 1024–1032 (2006). Article  Google Scholar  * Thersleff, T. et al. Chemically and

morphologically distinct grain boundaries in Ge-doped Cu2ZnSnSe4 solar cells revealed with STEM-EELS. _Mater. Des._ 122, 102–109 (2017). Article  Google Scholar  * Potapov, P. & Lubk, A.

Extraction of physically meaningful endmembers from STEM spectrum-images combining geometrical and statistical approaches. _Micron_ 145, 103068 (2021). Article  Google Scholar  * Bobynko,

J., MacLaren, I. & Craven, A. J. Spectrum imaging of complex nanostructures using DualEELS: I. Digital extraction replicas. _Ultramicroscopy_ 149, 9–20 (2015). Article  Google Scholar  *

Craven, A. J., Sala, B., Bobynko, J. & MacLaren, I. Spectrum imaging of complex nanostructures using DualEELS: II. Absolute quantification using standards. _Ultramicroscopy_ 186, 66–81

(2018). Article  Google Scholar  * Shiga, M. et al. Sparse modeling of EELS and EDX spectral imaging data by nonnegative matrix factorization. _Ultramicroscopy_ 170, 43–59 (2016). Article 

Google Scholar  * Nicoletti, O. et al. Three-dimensional imaging of localized surface plasmon resonances of metal nanoparticles. _Nature_ 502, 80–84 (2013). Article  ADS  Google Scholar  *

Lichtert, S. & Verbeeck, J. Statistical consequences of applying a PCA noise filter on EELS spectrum images. _Ultramicroscopy_ 125, 35–42 (2013). Article  Google Scholar  * LeBeau, J.

M., Findlay, S. D., Allen, L. J. & Stemmer, S. Quantitative atomic resolution scanning transmission electron microscopy. _Phys. Rev. Lett._ 100, 4 (2008). Article  Google Scholar  *

LeBeau, J. M., Findlay, S. D., Allen, L. J. & Stemmer, S. Standardless atom counting in scanning transmission electron microscopy. _Nano Lett._ 10, 4405–4408 (2010). Article  ADS  Google

Scholar  * Katz-Boon, H. et al. Three-dimensional morphology and crystallography of gold nanorods. _Nano Lett._ 11, 273–278 (2011). Article  ADS  Google Scholar  * Katz-Boon, H. et al.

Stability of crystal facets in gold nanorods. _Nano Lett._ 15, 1635–1641 (2015). Article  ADS  Google Scholar  * Erni, R., Heinrich, H. & Kostorz, G. Quantitative characterisation of

chemical inhomogeneities in Al–Ag using high-resolution _Z_-contrast STEM. _Ultramicroscopy_ 94, 125–133 (2003). Article  Google Scholar  * Jones, L. Quantitative ADF STEM: acquisition,

analysis and interpretation. in _14th European Workshop on Modern Developments and Applications in Microbeam Analysis_, _IOP Conference Series Materials Science and Engineering_ Vol. 109

(IOP Publishing, 2016). * Dwyer, C. et al. Sub-0.1 nm-resolution quantitative scanning transmission electron microscopy without adjustable parameters. _Appl. Phys. Lett._ 100, 4 (2012).

Article  Google Scholar  * Ishikawa, R., Lupini, A. R., Findlay, S. D. & Pennycook, S. J. Quantitative annular dark field electron microscopy using single electron signals. _Microsc.

Microanal._ 20, 99–110 (2014). Article  ADS  Google Scholar  * Sang, X. H. & LeBeau, J. M. Characterizing the response of a scintillator-based detector to single electrons.

_Ultramicroscopy_ 161, 3–9 (2016). Article  Google Scholar  * LeBeau, J. M. & Stemmer, S. Experimental quantification of annular dark-field images in scanning transmission electron

microscopy. _Ultramicroscopy_ 108, 1653–1658 (2008). Article  Google Scholar  * LeBeau, J. M., Findlay, S. D., Allen, L. J. & Stemmer, S. Position averaged convergent beam electron

diffraction: theory and applications. _Ultramicroscopy_ 110, 118–125 (2010). Article  Google Scholar  * Verbeeck, J., Beche, A. & Van den Broek, W. A holographic method to measure the

source size broadening in STEM. _Ultramicroscopy_ 120, 35–40 (2012). Article  Google Scholar  * Maunders, C., Dwyer, C., Tiemeijer, P. C. & Etheridge, J. Practical methods for the

measurement of spatial coherence — a comparative study. _Ultramicroscopy_ 111, 1437–1446 (2011). Article  Google Scholar  * Haider, M., Uhlemann, S. & Zach, J. Upper limits for the

residual aberrations of a high-resolution aberration-corrected STEM. _Ultramicroscopy_ 81, 163–175 (2000). Article  Google Scholar  * Kuramochi, K. et al. Effect of chromatic aberration on

atomic-resolved spherical aberration corrected STEM images. _Ultramicroscopy_ 110, 36–42 (2009). Article  Google Scholar  * Zheng, C. L. & Etheridge, J. Measurement of chromatic

aberration in STEM and SCEM by coherent convergent beam electron diffraction. _Ultramicroscopy_ 125, 49–58 (2013). Article  Google Scholar  * Voyles, P. M., Grazul, J. L. & Muller, D. A.

Imaging individual atoms inside crystals with ADF-STEM. _Ultramicroscopy_ 96, 251–273 (2003). Article  Google Scholar  * Sang, X. H., Oni, A. A. & LeBeau, J. M. Atom column indexing:

atomic resolution image analysis through a matrix representation. _Microsc. Microanal._ 20, 1764–1771 (2014). Article  ADS  Google Scholar  * Yankovich, A. B. et al. Picometre-precision

analysis of scanning transmission electron microscopy images of platinum nanocatalysts. _Nat. Commun_. 4155 (2014). * Voyles, P. M., Muller, D. A., Grazul, J. L., Citrin, P. H. &

Gossmann, H. J. L. Atomic-scale imaging of individual dopant atoms and clusters in highly n-type bulk Si. _Nature_ 416, 826–829 (2002). Article  ADS  Google Scholar  * Ishikawa, R., Lupini,

A. R., Findlay, S. D., Taniguchi, T. & Pennycook, S. J. Three-dimensional location of a single dopant with atomic precision by aberration-corrected scanning transmission electron

microscopy. _Nano Lett._ 14, 1903–1908 (2014). Article  ADS  Google Scholar  * Mevenkamp, N. et al. Poisson noise removal from high-resolution STEM images based on periodic block matching.

_Adv. Struct. Chem. Imag._ 1, 19 (2015). Article  Google Scholar  * Bals, S., Van Aert, S., Van Tendeloo, G. & Avila-Brande, D. Statistical estimation of atomic positions from exit wave

reconstruction with a precision in the picometer range. _Phys. Rev. Lett._ 96, 4 (2006). Article  Google Scholar  * De Backer, A., van den Bos, K. H. W., Van den Broek, W., Sijbers, J. &

Van Aert, S. StatSTEM: an efficient approach for accurate and precise model-based quantification of atomic resolution electron microscopy images. _Ultramicroscopy_ 171, 104–116 (2016).

Article  Google Scholar  * So, Y. G. & Kimoto, K. Effect of specimen misalignment on local structure analysis using annular dark-field imaging. _J. Electron. Microsc._ 61, 207–215

(2012). Article  Google Scholar  * Schmid, H., Okunishi, E., Oikawa, T. & Mader, W. Structural and elemental analysis of iron and indium doped zinc oxide by spectroscopic imaging in

Cs-corrected STEM. _Micron_ 43, 49–56 (2012). Article  Google Scholar  * Fan, L., Zhang, F., Fan, H. & Zhang, C. Brief review of image denoising techniques. _Vis. Comput. Indust. Biomed.

Art_ 2, 7 (2019). Article  Google Scholar  * Ziatdinov, M. et al. Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking

local transformations. _ACS Nano_ 11, 12742–12752 (2017). Article  Google Scholar  * Ziatdinov, M., Nelson, C., Vasudevan, R. K., Chen, D. Y. & Kalinin, S. V. Building ferroelectric

from the bottom up: the machine learning analysis of the atomic-scale ferroelectric distortions. _Appl. Phys. Lett._ 115, 052902 (2019). Article  ADS  Google Scholar  * Lin, R., Zhang, R.,

Wang, C., Yang, X.-Q. & Xin, H. L. TEMImageNet and AtomSegNet deep learning training library and models for high-precision atom segmentation, localization, denoising, and

super-resolution processing of atom-resolution scanning TEM images. Preprint at https://arxiv.org/abs/2012.09093 (2020). * Ziatdinov, M., Dyck, O., Jesse, S. & Kalinin, S. V. Atomic

mechanisms for the Si atom dynamics in graphene: chemical transformations at the edge and in the bulk. _Adv. Funct. Mater._ 29, 1904480 (2019). Article  Google Scholar  * Ziatdinov, M. et

al. Building and exploring libraries of atomic defects in graphene: scanning transmission electron and scanning tunneling microscopy study. _Sci. Adv._ 5, eaaw8989 (2019). Article  ADS 

Google Scholar  * Lee, C. H. et al. Deep learning enabled strain mapping of single-atom defects in two-dimensional transition metal dichalcogenides with sub-picometer precision. _Nano Lett._

20, 3369–3377 (2020). Article  ADS  Google Scholar  * Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. _Science_ 313, 504 (2006). Article 

ADS  MathSciNet  MATH  Google Scholar  * Roccapriore, K. M., Ziatdinov, M., Cho, S. H., Hachtel, J. A. & Kalinin, S. V. Predictability of localized plasmonic responses in nanoparticle

assemblies. _Small_ 17, 2100181 (2021). Article  Google Scholar  * Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at https://arxiv.org/abs/1312.6114 (2013). *

Kingma, D. P. & Welling, M. An introduction to variational autoencoders. _Found. Trends Mach. Learn._ 12, 307–392 (2019). Article  MATH  Google Scholar  * Blei, D. M., Kucukelbir, A.

& McAuliffe, J. D. Variational inference: a review for statisticians. _J. Am. Stat. Assoc._ 112, 859–877 (2017). Article  MathSciNet  Google Scholar  * Muller, D. A. & Grazul, J.

Optimizing the environment for sub-0.2 nm scanning transmission electron microscopy. _J. Electron. Microsc._ 50, 219–226 (2001). Google Scholar  * Vonharrach, H. S. Instrumental factors in

high-resolution FEG STEM. _Ultramicroscopy_ 58, 1–5 (1995). Article  Google Scholar  * Couillard, M., Radtke, G. & Botton, G. A. Strain fields around dislocation arrays in a Σ9 silicon

bicrystal measured by scanning transmission electron microscopy. _Phil. Mag._ 93, 1250–1267 (2013). Article  ADS  Google Scholar  * Isakozawa, S., Tomonaga, S., Hashimoto, T. & Baba, N.

High-precision image-drift-correction method for EM images with a low signal-to-noise ratio. _Microscopy_ 63, 301–312 (2014). Article  Google Scholar  * Sang, X. H. & LeBeau, J. M.

Revolving scanning transmission electron microscopy: correcting sample drift distortion without prior knowledge. _Ultramicroscopy_ 138, 28–35 (2014). Article  Google Scholar  * Dycus, J. H.

et al. Accurate nanoscale crystallography in real-space using scanning transmission electron microscopy. _Microsc. Microanal._ 21, 946–952 (2015). Article  ADS  Google Scholar  * Jones, L.

et al. Smart Align — a new tool for robust non-rigid registration of scanning microscope data. _Adv. Struct. Chem. Imag._ 1, 8 (2015). Article  Google Scholar  * Ophus, C., Ciston, J. &

Nelson, C. T. Correcting nonlinear drift distortion of scanning probe and scanning transmission electron microscopies from image pairs with orthogonal scan directions. _Ultramicroscopy_ 162,

1–9 (2016). Article  Google Scholar  * Berkels, B. & Liebscher, C. H. Joint non-rigid image registration and reconstruction for quantitative atomic resolution scanning transmission

electron microscopy. _Ultramicroscopy_ 198, 49–57 (2019). Article  Google Scholar  * Yankovich, A. B. et al. Non-rigid registration and non-local principal component analysis to improve

electron microscopy spectrum images. _Nanotechnology_ 27, 364001 (2016). Article  Google Scholar  * Wang, Y. et al. Correcting the linear and nonlinear distortions for atomically resolved

STEM spectrum and diffraction imaging. _Microscopy_ 67, i114–i122 (2018). Article  Google Scholar  * Barcena-Gonzalez, G. et al. CDrift: an algorithm to correct linear drift from a single

high-resolution STEM image. _Microsc. Microanal._ 26, 913–920 (2020). Article  ADS  Google Scholar  * Sanchez, A. M. et al. An approach to the systematic distortion correction in

aberration-corrected HAADF images. _J. Microsc._ 221, 1–7 (2006). Article  MathSciNet  Google Scholar  * Braidy, N., Le Bouar, Y., Lazar, S. & Ricolleau, C. Correcting scanning

instabilities from images of periodic structures. _Ultramicroscopy_ 118, 67–76 (2012). Article  Google Scholar  * Ning, S. C. et al. Scanning distortion correction in STEM images.

_Ultramicroscopy_ 184, 274–283 (2018). Article  Google Scholar  * Roccapriore, K. M., Ziatdinov, M. & Kalinin, S. V. Identification and correction of temporal and spatial distortions in

scanning transmission electron microscopy. _Ultramicroscopy_ 229, 113337 (2021). Article  Google Scholar  * Sang, X. H. et al. Dynamic scan control in STEM: spiral scans. _Adv. Struct. Chem.

Imag_. 2, 6 (2016). * Li, X., Dyck, O., Kalinin, S. V. & Jesse, S. Compressed sensing of scanning transmission electron microscopy (STEM) with nonrectangular scans. _Microsc.

Microanal._ 24, 623–633 (2018). Article  ADS  Google Scholar  * Stevens, A. et al. A sub-sampled approach to extremely low-dose STEM. _Appl. Phys. Lett._ 112, 043104 (2018). Article  ADS 

Google Scholar  * Zobelli, A. et al. Spatial and spectral dynamics in STEM hyperspectral imaging using random scan patterns. _Ultramicroscopy_ 212, 112912 (2020). Article  Google Scholar  *

Beche, A., Goris, B., Freitag, B. & Verbeeck, J. Development of a fast electromagnetic beam blanker for compressed sensing in scanning transmission electron microscopy. _Appl. Phys.

Lett._ 108, 093103 (2016). Article  ADS  Google Scholar  * Velazco, A., Nord, M., Beche, A. & Verbeeck, J. Evaluation of different rectangular scan strategies for STEM imaging.

_Ultramicroscopy_ 215, 113021 (2020). * Dai, G. L. et al. Reference nano-dimensional metrology by scanning transmission electron microscopy. _Meas. Sci. Technol._ 24, 085001 (2013). Article

  ADS  Google Scholar  * Radaelli, P. G. & Cheong, S. W. Structural phenomena associated with the spin-state transition in LaCoO3. _Phys. Rev. B_ 66, 094408 (2002). Article  ADS  Google

Scholar  * Zhou, J. S. & Goodenough, J. B. Chemical bonding and electronic structure of _R_NiO3 (_R_ = rare earth). _Phys. Rev. B_ 69, 153105 (2004). Article  ADS  Google Scholar  *

Zhou, J. S. & Goodenough, J. B. Universal octahedral-site distortion in orthorhombic perovskite oxides. _Phys. Rev. Lett._ 94, 065501 (2005). Article  ADS  Google Scholar  * Taraci, J.

L. et al. Strain mapping in nanowires. _Nanotechnology_ 16, 2365–2371 (2005). Article  ADS  Google Scholar  * Hytch, M. J., Putaux, J. L. & Thibault, J. Stress and strain around

grain-boundary dislocations measured by high-resolution electron microscopy. _Phil. Mag._ 86, 4641–4656 (2006). Article  ADS  Google Scholar  * Chu, M. W., Szafraniak, I., Hesse, D., Alexe,

M. & Gosele, U. Elastic coupling between 90° twin walls and interfacial dislocations in epitaxial ferroelectric perovskites: a quantitative high-resolution transmission electron

microscopy study. _Phys. Rev. B_ 72, 174112 (2005). * Zhao, C. W., Xing, Y. M., Zhou, C. E. & Bai, P. C. Experimental examination of displacement and strain fields in an edge dislocation

core. _Acta Mater._ 56, 2570–2575 (2008). Article  ADS  Google Scholar  * Pennycook, S. J. & Nellist, P. D. _Scanning Transmission Electron Microscopy: Imaging and Analysis_ (Springer

Science & Business Media, 2011). * Sousa, A. A. & Leapman, R. D. Development and application of STEM for the biological sciences. _Ultramicroscopy_ 123, 38–49 (2012). Article  Google

Scholar  * Tagantsev, A. K., Cross, L. E. & Fousek, J. _Domains in Ferroic Crystals and Thin Films_ (Springer, 2010). * Salje, E. K. H. Ferroelasticity. _Contemp. Phys._ 41, 79–91

(2000). Article  ADS  Google Scholar  * Pan, X. Q., Kaplan, W. D., Ruhle, M. & Newnham, R. E. Quantitative comparison of transmission electron microscopy techniques for the study of

localized ordering on a nanoscale. _J. Am. Ceram. Soc._ 81, 597–605 (1998). Article  Google Scholar  * Jia, C. L. et al. Unit-cell scale mapping of ferroelectricity and tetragonality in

epitaxial ultrathin ferroelectric films. _Nat. Mater._ 6, 64–69 (2007). Article  ADS  Google Scholar  * Jia, C. L. et al. Atomic-scale study of electric dipoles near charged and uncharged

domain walls in ferroelectric films. _Nat. Mater._ 7, 57–61 (2008). Article  ADS  Google Scholar  * Chisholm, M. F., Luo, W. D., Oxley, M. P., Pantelides, S. T. & Lee, H. N. Atomic-scale

compensation phenomena at polar interfaces. _Phys. Rev. Lett_. 105, 197602 (2010). * Jia, C. L. et al. Oxygen octahedron reconstruction in the SrTiO3/LaAlO3 heterointerfaces investigated

using aberration-corrected ultrahigh-resolution transmission electron microscopy. _Phys. Rev. B_ 79, 081405 (2009). * Borisevich, A. et al. Mapping octahedral tilts and polarization across a

domain wall in BiFeO3 from _Z_-contrast scanning transmission electron microscopy image atomic column shape analysis. _ACS Nano_ 4, 6071–6079 (2010). Article  Google Scholar  * Kim, Y. M.

et al. Interplay of octahedral tilts and polar order in BiFeO3 Films. _Adv. Mater._ 25, 2497–2504 (2013). Article  Google Scholar  * He, Q. et al. Towards 3D Mapping of BO6 octahedron

rotations at perovskite heterointerfaces, unit cell by unit cell. _ACS Nano_ 9, 8412–8419 (2015). Article  Google Scholar  * Kim, Y. M. et al. Direct observation of ferroelectric field

effect and vacancy-controlled screening at the BiFeO3/LaxSr1-_x_MnO3 interface. _Nat. Mater._ 13, 1019–1025 (2014). Article  ADS  Google Scholar  * Borisevich, A. Y. et al. Interface dipole

between two metallic oxides caused by localized oxygen vacancies. _Phys. Rev. B_ 86, 140102 (2012). * Borisevich, A. Y. et al. Exploring mesoscopic physics of vacancy-ordered systems through

atomic scale observations of topological defects. _Phys. Rev. Lett_. 109, 065702 (2012). * Li, Q. et al. Quantification of flexoelectricity in PbTiO3/SrTiO3 superlattice polar vortices

using machine learning and phase-field modeling. _Nat. Commun_. 8, 1468 (2017). * Nelson, C. T. et al. Exploring physics of ferroelectric domain walls via Bayesian analysis of atomically

resolved STEM data. _Nat. Commun._ 11, 6361 (2020). Article  ADS  Google Scholar  * Rossell, M. D. et al. Atomic structure of highly strained BiFeO3 thin films. _Phys. Rev. Lett._ 108, 5

(2012). Article  Google Scholar  * Campanini, M., Erni, R., Yang, C. H., Ramesh, R. & Rossell, M. D. Periodic giant polarization gradients in doped BiFeO3 thin films. _Nano Lett._ 18,

717–724 (2018). Article  ADS  Google Scholar  * Campanini, M. et al. Atomic-resolution differential phase contrast STEM on ferroelectric materials: a mean-field approach. _Phys. Rev. B_ 101,

184116 (2020). Article  ADS  Google Scholar  * Midgley, P. A. & Dunin-Borkowski, R. E. Electron tomography and holography in materials science. _Nat. Mater._ 8, 271–280 (2009). Article

  ADS  Google Scholar  * Welborn, M., Tang, W. J., Ryu, J., Petkov, V. & Henkelman, G. A combined density functional and x-ray diffraction study of Pt nanoparticle structure. _J. Chem.

Phys._ 135, 014503 (2011). Article  ADS  Google Scholar  * Meredig, B. & Wolverton, C. A hybrid computational-experimental approach for automated crystal structure solution. _Nat.

Mater._ 12, 123–127 (2013). Article  ADS  Google Scholar  * Alkauskas, A., McCluskey, M. D. & Van de Walle, C. G. Tutorial: defects in semiconductors — combining experiment and theory.

_J. Appl. Phys._ 119, 181101 (2016). Article  ADS  Google Scholar  * Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O. & Walsh, A. Machine learning for molecular and materials

science. _Nature_ 559, 547–555 (2018). Article  ADS  Google Scholar  * Kolb, B., Lentz, L. C. & Kolpak, A. M. Discovering charge density functionals and structure-property relationships

with PROPhet: a general framework for coupling machine learning and first-principles methods. _Sci. Rep._ 7, 1192 (2017). Article  ADS  Google Scholar  * Schuett, O. & VandeVondele, J.

Machine learning adaptive basis sets for efficient large scale density functional theory simulation. _J. Chem. Theory Comput._ 14, 4168–4175 (2018). Article  Google Scholar  * Sen, F. G. et

al. Sen, F. G. et al. Atomistic Simulations of Grain Boundaries in CdTe in _2015 IEEE 42nd Photovoltaic Specialists Conference_ (IEEE, 2015). * Schwenker, E. et al. An autonomous microscopy

workflow for structure determination from atomic-resolution images. _Microsc. Microanal._ 24, 510–511 (2018). Article  Google Scholar  * Wei, J. et al. Direct imaging of atomistic grain

boundary migration. _Nat. Mater._ 20, 951–955 (2021). Article  ADS  Google Scholar  * Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error

visibility to structural similarity. _IEEE Trans. Image Process._ 13, 600–612 (2004). Article  ADS  Google Scholar  * Liu, T.-J., Lin, Y.-C., Lin, W. & Kuo, C. C. J. Visual quality

assessment: recent developments, coding applications and future trends. _APSIPA Trans. Signal. Inf. Process._ 2, e4 (2013). Article  Google Scholar  * Lin, J. Y., Liu, T., Wu, E. C. &

Kuo, C. J. in _Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific_ 1–5 (IEEE, 2014). * Guo, J. L. et al. Effect of selenium and chlorine

co-passivation in polycrystalline CdSeTe devices. _Appl. Phys. Lett._ 115, 50 (2019). Article  Google Scholar  * Krivanek, O. L. et al. Atom-by-atom structural and chemical analysis by

annular dark-field electron microscopy. _Nature_ 464, 571–574 (2010). Article  ADS  Google Scholar  * Treacy, M. M. J. _Z_ dependence of electron scattering by single atoms into annular

dark-field detectors. _Microsc. Microanal._ 17, 847–858 (2011). Article  ADS  Google Scholar  * Yamashita, S. et al. Atomic number dependence of _Z_ contrast in scanning transmission

electron microscopy. _Sci. Rep._ 8, 12325 (2018). Article  ADS  Google Scholar  * Li, H. et al. Atomic structure and dynamics of single platinum atom interactions with monolayer MoS2. _ACS

Nano_ 11, 3392–3403 (2017). Article  Google Scholar  * Wang, S. et al. Preferential Pt nanocluster seeding at grain boundary dislocations in polycrystalline monolayer MoS2. _ACS Nano_ 12,

5626–5636 (2018). Article  Google Scholar  * Zan, R., Bangert, U., Ramasse, Q. & Novoselov, K. S. Metal−graphene interaction studied via atomic resolution scanning transmission electron

microscopy. _Nano Lett._ 11, 1087–1092 (2011). Article  ADS  Google Scholar  * Wang, S. et al. In situ atomic-scale studies of the formation of epitaxial Pt nanocrystals on monolayer

molybdenum disulfide. _ACS Nano_ 11, 9057–9067 (2017). Article  Google Scholar  * Li, X. et al. Precursor design for high density single pt atom sites on MoS2: enhanced stability at elevated

temperatures and reduced 3D clustering. _Chem. Mater._ 32, 2541–2551 (2020). Article  Google Scholar  * Tai, K.-L. et al. Phase variations and layer epitaxy of 2D PdSe2 grown on 2D

monolayers by direct selenization of molecular Pd precursors. _ACS Nano_ 14, 11677–11690 (2020). Article  Google Scholar  * Sinha, S. et al. Atomic structure and defect dynamics of monolayer

lead iodide nanodisks with epitaxial alignment on graphene. _Nat. Commun._ 11, 823 (2020). Article  ADS  Google Scholar  * Poh, S. M. et al. Molecular beam epitaxy of highly crystalline

MoSe2 on hexagonal boron nitride. _ACS Nano_ 12, 7562–7570 (2018). Article  Google Scholar  * Gallagher, J. C. et al. Epitaxial growth of iridate pyrochlore Nd2Ir2O7 films. _Sci. Rep._ 6,

22282 (2016). Article  ADS  Google Scholar  * Gerkman, M. A. et al. Direct imaging of individual molecular binding to clean nanopore edges in 2D monolayer MoS2. _ACS Nano_ 14, 153–165

(2020). Article  Google Scholar  * Cai, C., Han, S., Wang, Q. & Gu, M. Direct observation of yolk–shell transforming to gold single atoms and clusters with superior oxygen evolution

reaction efficiency. _ACS Nano_ 13, 8865–8871 (2019). Article  Google Scholar  * Sun, Y. et al. Subunit cell–level measurement of polarization in an individual polar vortex. _Sci. Adv._ 5,

eaav4355 (2019). Article  ADS  Google Scholar  * Ohyama, J. et al. Atomic-scale insight into the structural effect of a supported Au catalyst based on a size-distribution analysis using

Cs-STEM and morphological image-processing. _J. Catal._ 335, 24–35 (2016). Article  Google Scholar  * Melzer, D. et al. Atomic-scale determination of active facets on the MoVTeNb oxide M1

phase and their intrinsic catalytic activity for ethane oxidative dehydrogenation. _Angew. Chem. Int. Ed._ 55, 8873–8877 (2016). Article  Google Scholar  * Qiao, B. et al. Single-atom

catalysis of CO oxidation using Pt1/FeOx. _Nat. Chem._ 3, 634–641 (2011). Article  Google Scholar  * Liu, J. et al. In situ tracing of atom migration in Pt/NiPt hollow spheres during

catalysis of CO oxidation. _Chem. Commun._ 50, 1804–1807 (2014). Article  Google Scholar  * Gerkman, M. A., Sinha, S., Warner, J. H. & Han, G. G. D. Direct imaging of photoswitching

molecular conformations using individual metal atom markers. _ACS Nano_ 13, 87–96 (2019). Article  Google Scholar  * Lee, J. K. et al. Metal atom markers for imaging epitaxial molecular

self-assembly on graphene by scanning transmission electron microscopy. _ACS Nano_ 13, 7252–7260 (2019). Article  Google Scholar  * Kengmana, E. S., Lee, J. K., Li, X., Warner, J. H. &

Han, G. G. D. Self-assembly of Bowlic supramolecules on graphene imaged at the individual molecular level using heavy atom tagging. _Small_ 16, 2002860 (2020). Article  Google Scholar  *

Henninen, T. R., Bon, M., Wang, F., Passerone, D. & Erni, R. The structure of sub-nm platinum clusters at elevated temperatures. _Angew. Chem. Int. Ed._ 59, 839–845 (2020). Article 

Google Scholar  * Furnival, T., Leary, R. K. & Midgley, P. A. Denoising time-resolved microscopy image sequences with singular value thresholding. _Ultramicroscopy_ 178, 112–124 (2017).

Article  Google Scholar  * Wang, F., Henninen, T. R., Keller, D. & Erni, R. Noise2Atom: unsupervised denoising for scanning transmission electron microscopy images. _Appl. Microsc._ 50,

23 (2020). Article  Google Scholar  * Mirman, R. _Point Groups, Space Groups, Crystals, Molecules_ (World Scientific, 1999). * Glinchuk, M. D. & Stephanovich, V. A. Dynamic properties of

relaxor ferroelectrics. _J. Appl. Phys._ 85, 1722–1726 (1999). Article  ADS  Google Scholar  * Vugmeister, B. E. Polarization dynamics and formation of polar nanoregions in relaxor

ferroelectrics. _Phys. Rev. B_ 73, 174117 (2006). * Takenaka, H., Grinberg, I., Liu, S. & Rappe, A. M. Slush-like polar structures in single-crystal relaxors. _Nature_ 546, 391–395

(2017). Article  ADS  Google Scholar  * Binder, K. & Reger, J. D. Theory of orientational glasses models, concepts, simulations. _Adv. Phys._ 41, 547–627 (1992). Article  ADS  Google

Scholar  * Binder, K. & Young, A. P. Spin-glasses — experimental facts, theoretical concepts, and open questions. _Rev. Mod. Phys._ 58, 801–976 (1986). Article  ADS  Google Scholar  *

Dagotto, E. Complexity in strongly correlated electronic systems. _Science_ 309, 257–262 (2005). Article  ADS  Google Scholar  * Dagotto, E., Hotta, T. & Moreo, A. Colossal

magnetoresistant materials: the key role of phase separation. _Phys. Rep. Rev. Sec. Phys. Lett._ 344, 1–153 (2001). Google Scholar  * Blinc, R. et al. Local polarization distribution and

Edwards–Anderson order parameter of relaxor ferroelectrics. _Phys. Rev. Lett._ 83, 424–427 (1999). Article  ADS  Google Scholar  * Cross, L. E. Relaxor ferroelectrics. _Ferroelectrics_ 76,

241–267 (1987). Article  Google Scholar  * Cliffe, M. J. et al. Structural simplicity as a restraint on the structure of amorphous silicon. _Phys. Rev. B_ 95, 224108 (2017). * Keen, D. A.

& Goodwin, A. L. The crystallography of correlated disorder. _Nature_ 521, 303–309 (2015). Article  ADS  Google Scholar  * Cheetham, A. K., Bennett, T. D., Coudert, F. X. & Goodwin,

A. L. Defects and disorder in metal organic frameworks. _Dalton Trans._ 45, 4113–4126 (2016). Article  Google Scholar  * Dobigeon, N. & Brun, N. Spectral mixture analysis of EELS

spectrum-images. _Ultramicroscopy_ 120, 25–34 (2012). Article  Google Scholar  * Pennycook, S. J., Varela, M., Lupini, A. R., Oxley, M. P. & Chisholm, M. F. Atomic-resolution

spectroscopic imaging: past, present and future. _J. Electron. Microsc._ 58, 87–97 (2009). Article  Google Scholar  * Jiang, Y. et al. Electron ptychography of 2D materials to deep

sub-ångstrom resolution. _Nature_ 559, 343–349 (2018). Article  Google Scholar  * Belianinov, A. et al. Identification of phases, symmetries and defects through local crystallography. _Nat.

Commun_. 6, 7801 (2015). * Ziatdinov, M. et al. Atomic-scale observation of structural and electronic orders in the layered compound α-RuCl3. _Nat. Commun_. 7, 13774 (2016). * He, Q., Woo,

J., Belianinov, A., Guliants, V. V. & Borisevich, A. Y. Better catalysts through microscopy: mesoscale M1/M2 intergrowth in molybdenum-vanadium based complex oxide catalysts for propane

ammoxidation. _ACS Nano_ 9, 3470–3478 (2015). Article  Google Scholar  * Wu, Z. et al. A comprehensive survey on graph neural networks. _IEEE Trans. Neural Netw. Learn. Syst._ 32, 4–24

(2021). Article  MathSciNet  Google Scholar  * Kalinin, S. V., Dyck, O., Ghosh, A., Sumpter, B. G. & Ziatdinov, M. Unsupervised machine learning discovery of chemical transformation

pathways from atomically-resolved imaging data. Preprint at https://arxiv.org/abs/2010.09196 (2020). * Wicks, S. et al. Collective dynamics in nanostructured polycrystalline ferroelectric

thin films using local time-resolved measurements and switching spectroscopy. _Acta Mater._ 58, 67–75 (2010). Article  ADS  Google Scholar  * Vlcek, L., Maksov, A., Pan, M., Vasudevan, R. K.

& Kalinin, S. V. Knowledge extraction from atomically resolved images. _ACS Nano_ 11, 10313–10320 (2017). Article  Google Scholar  * Landau, L. D. & Lifshitz, E. M. _Statistical

Physics_ (Elsevier Science, 2013). * Kitaev, A. Anyons in an exactly solved model and beyond. _Ann. Phys._ 321, 2–111 (2006). Article  ADS  MathSciNet  MATH  Google Scholar  * Vlcek, L.,

Yang, S., Ziatdinov, M., Kalinin, S. & Vasudevan, R. Statistical physics-based framework and Bayesian inference for model selection and uncertainty quantification. _Microsc. Microanal._

25, 130–131 (2019). Article  ADS  Google Scholar  * Mani Prudhvi Valleti, S., Vlcek, L., Vasudevan, R. K. & Kalinin, S. V. Inversion of lattice models from the observations of

microscopic degrees of freedom: parameter estimation with uncertainty quantification. Preprint at https://arxiv.org/abs/1909.09244 (2019). * Chakraborty, M. et al. Reconstruction of the

interatomic forces from dynamic scanning transmission electron microscopy data. _J. Appl. Phys._ 127, 224301 (2020). Article  ADS  Google Scholar  * Nobakht, A. Y. et al. Reconstruction of

effective potential from statistical analysis of dynamic trajectories. _AIP Adv_. 10, 065034 (2020). * Schepman, A. M. H., Vandervoort, J. A. P., Kramer, J. & Mellema, J. E. Getting

started with a scanning-transmission electron-microscope coupled to a small computer. _Ultramicroscopy_ 3, 265–269 (1978). Article  Google Scholar  * Strahm, M. & Butler, J. H. Fast

digital data acquisition and online processing system for an HB5 scanning-transmission electron-microscope. _Rev. Sci. Instrum._ 52, 840–848 (1981). Article  ADS  Google Scholar  * Andree,

P. J., Mellema, J. E. & Ruigrok, R. W. H. Discrimination of heavy and light-elements in a specimen by use of scanning-transmission electron-microscopy. _Ultramicroscopy_ 17, 237–241

(1985). Article  Google Scholar  * Haider, M., Epstein, A., Jarron, P. & Boulin, C. A versatile, software configurable multichannel stem detector for angle-resolved imaging.

_Ultramicroscopy_ 54, 41–59 (1994). Article  Google Scholar  * Yang, Y. S. et al. Deciphering chemical order/disorder and material properties at the single-atom level. _Nature_ 542, 75–79

(2017). Article  ADS  Google Scholar  * Kalinin, S. V., S., B. G. & Archibald, R. K. Big–deep–smart data in imaging for guiding materials design. _Nat. Mater._ 14, 973–980 (2015).

Article  ADS  Google Scholar  * Spurgeon, S. R. et al. Towards data-driven next-generation transmission electron microscopy. _Nat. Mater_. 20, 274–279 (2021). * Vasilevsky, N. A., Minnier,

J., Haendel, M. A. & Champieux, R. E. Reproducible and reusable research: are journal data sharing policies meeting the mark? _PeerJ_ 5, e3208 (2017). Article  Google Scholar  * Molloy,

J. C. The open knowledge foundation: open data means better science. _PLoS Biol._ 9, e1001195 (2011). Article  Google Scholar  * Woelfle, M., Olliaro, P. & Todd, M. H. Open science is a

research accelerator. _Nat. Chem._ 3, 745–748 (2011). Article  Google Scholar  * Hanisch, R. J., Gilmore, I. S. & Plant, A. L. Improving reproducibility in research: the role of

measurement science. _J. Res. Natl Inst. Stand. Technol._ 124, 124024 (2019). Article  Google Scholar  * Blaiszik, B. et al. The materials data facility: data services to advance materials

science research. _JoM_ 68, 2045–2052 (2016). Article  Google Scholar  * Jain, A. et al. Commentary: the Materials Project: a materials genome approach to accelerating materials innovation.

_APL Mater_. 1, 011002 (2013). * Iudin, A., Korir, P. K., Salavert-Torres, J., Kleywegt, G. J. & Patwardhan, A. EMPIAR: a public archive for raw electron microscopy image data. _Nat.

Methods_ 13, 387–388 (2016). Article  Google Scholar  * Berman, H., Henrick, K. & Nakamura, H. Announcing the worldwide Protein Data Bank. _Nat. Struct. Biol._ 10, 980–980 (2003).

Article  Google Scholar  * Guo, S. et al. Direct mapping of ion diffusion times on LiCoO2 surfaces with nanometer resolution. _J. Electrochem. Soc._ 158, A982–A990 (2011). Article  Google

Scholar  * Nikiforov, M. P. et al. Temperature-dependent phase transitions in zeptoliter volumes of a complex biological membrane. _Nanotechnology_ 22, 055709 (2011). * Kalinin, S. V. et al.

Big, deep, and smart data in scanning probe microscopy. _ACS Nano_ 10, 9068–9086 (2016). Article  Google Scholar  * Komsa, H. P. et al. Two-dimensional transition metal dichalcogenides

under electron irradiation: defect production and doping. _Phys. Rev. Lett_. 109, 035503 (2012). * Mishra, R., Ishikawa, R., Lupini, A. R. & Pennycook, S. J. Single-atom dynamics in

scanning transmission electron microscopy. _MRS Bull._ 42, 644–652 (2017). Article  ADS  Google Scholar  * Toma Susi, J. C. M. & Kotakoski, J. Manipulating low-dimensional materials down

to the level of single atoms with electron irradiation. _Ultramicroscopy_ 180, 163–172 (2017). Article  Google Scholar  * Dyck, O. et al. Building structures atom by atom via electron beam

manipulation. _Small_ 14, 1801771 (2018). * Dyck, O., Jesse, S. & Kalinin, S. V. A self-driving microscope and the atomic forge. _MRS Bull._ 44, 669–670 (2019). Article  ADS  Google

Scholar  * Kelley, K. P. et al. Fast scanning probe microscopy via machine learning: non-rectangular scans with compressed sensing and Gaussian process optimization. _Small_ 16, 2002878

(2020). Article  MathSciNet  Google Scholar  * Stevens, A. et al. Applying compressive sensing to TEM video: a substantial frame rate increase on any camera. _Adv. Struct. Chem. Imag_. 1, 10

(2015). * Kovarik, L., Stevens, A., Liyu, A. & Browning, N. D. Implementing an accurate and rapid sparse sampling approach for low-dose atomic resolution STEM imaging. _Appl. Phys.

Lett_. 109, 164102 (2016). * Jesse, S. et al. Direct atomic fabrication and dopant positioning in Si using electron beams with active real-time image-based feedback. _Nanotechnology_ 29,

255303 (2018). * Gulrajani, I. & Lopez-Paz, D. In search of lost domain generalization. Preprint at https://arxiv.org/abs/2007.01434 (2020). * Kalinin, S. V. et al. Automated and

autonomous experiments in electron and scanning probe microscopy. _ACS Nano_ 15, 12604–12627 (2021). Article  Google Scholar  * Wilson, A. G., Hu, Z., Salakhutdinov, R. & Xing, E. P. in

_Artificial Intelligence and Statistics_ 370–378 (PMLR, 2021). * Zheng, H. M. et al. Observation of transient structural-transformation dynamics in a Cu2S nanorod. _Science_ 333, 206–209

(2011). Article  ADS  Google Scholar  * Klie, R. F., Ito, Y., Stemmer, S. & Browning, N. S. Observation of oxygen vacancy ordering and segregation in perovskite oxides. _Ultramicroscopy_

86, 289–302 (2001). Article  Google Scholar  * Jang, J. H. et al. In situ observation of oxygen vacancy dynamics and ordering in the epitaxial LaCoO3 system. _ACS Nano_ 11, 6942–6949

(2017). Article  Google Scholar  * Hoehl, D. et al. Temperature-dependence of electron-beam induced epitaxial crystallization of silicon. _Phys. Status Solidi A_ 122, K35–K37 (1990). Article

  ADS  Google Scholar  * Jencic, I., Bench, M. W., Robertson, I. M. & Kirk, M. A. Electron-beam-induced crystallization of isolated amorphous regions in Si, Ge, GaP, and GaAs. _J. Appl.

Phys._ 78, 974–982 (1995). Article  ADS  Google Scholar  * Remeika, M. & Bezryadin, A. Sub-10 nanometre fabrication: molecular templating, electron-beam sculpting and crystallization of

metallic nanowires. _Nanotechnology_ 16, 1172–1176 (2005). Article  ADS  Google Scholar  * Bae, I. T., Zhang, Y. W., Weber, W. J., Higuchi, M. & Giannuzzi, L. A. Electron-beam induced

recrystallization in amorphous apatite. _Appl. Phys. Lett_. 90, 021912 (2007). * Zhang, Y. et al. Ion-induced damage accumulation and electron-beam-enhanced recrystallization in SrTiO3.

_Phys. Rev. B_ 72, 094112 (2005). * Huang, P. Y. et al. Imaging atomic rearrangements in two-dimensional silica glass: watching Silica’s Dance. _Science_ 342, 224–227 (2013). Article  ADS 

Google Scholar  * Kurasch, S. et al. Atom-by-atom observation of grain boundary migration in graphene. _Nano Lett._ 12, 3168–3173 (2012). Article  ADS  Google Scholar  * Yang, Z. Q. et al.

Direct observation of atomic dynamics and silicon doping at a topological defect in graphene. _Angew. Chem. Int. Edit._ 53, 8908–8912 (2014). Article  Google Scholar  * Krasheninnikov, A. V.

& Banhart, F. Engineering of nanostructured carbon materials with electron or ion beams. _Nat. Mater._ 6, 723–733 (2007). Article  ADS  Google Scholar  * Lingerfelt, D. B. et al.

Nonadiabatic effects on defect diffusion in silicon-doped nanographenes. _Nano Lett._ 21, 236–242 (2021). Article  ADS  Google Scholar  * Susi, T., Meyer, J. C. & Kotakoski, J.

Quantifying transmission electron microscopy irradiation effects using two-dimensional materials. _Nat. Rev. Phys._ 1, 397–405 (2019). Article  Google Scholar  * Krasheninnikov, A. V. &

Nordlund, K. Ion and electron irradiation-induced effects in nanostructured materials. _J. Appl. Phys._ 107, 071301 (2010). * Egerton, R. F. Radiation damage to organic and inorganic

specimens in the TEM. _Micron_ 119, 72–87 (2019). Article  Google Scholar  * Jesse, S. et al. Atomic-level sculpting of crystalline oxides: toward bulk nanofabrication with single atomic

plane precision. _Small_ 11, 5895–5900 (2015). Article  Google Scholar  * Mousavi, S. S., Schukat, M. & Howley, E. in _Proc. of SAI Intelligent Systems Conf._ 426–440 (Springer, 2021). *

Schrittwieser, J. et al. Mastering Atari, Go, chess and shogi by planning with a learned model. _Nature_ 588, 604–609 (2020). Article  ADS  Google Scholar  * Wagner, T. & Raunser, S.

The evolution of SPHIRE-crYOLO particle picking and its application in automated cryo-EM processing workflows. _Commun. Biol._ 3, 61 (2020). Article  Google Scholar  * Kalinin, S. V., Dyck,

O., Jesse, S. & Ziatdinov, M. Exploring order parameters and dynamic processes in disordered systems via variational autoencoders. _Sci. Adv._ 7, eabd5084 (2021). Article  ADS  Google

Scholar  Download references ACKNOWLEDGEMENTS This work is based upon work supported by the US Department of Energy (DOE), Office of Science, Basic Energy Sciences (BES), Materials Sciences

and Engineering Division (S.V.K., M.P.O. and A.R.L.) and was performed and partially supported (M.Z.) at Oak Ridge National Laboratory’s Center for Nanophase Materials Sciences (CNMS), a US

Department of Energy, Office of Science User Facility. V.G. acknowledges the support of the European Union Horizon 2020 Research and Innovation Programme under grant agreement no. 766970

Q-SORT (H2020-FETOPEN-1-2016-2017), no. 964591 Smart-electrons (H2020-FETOPEN-1-2016-2017) and no. 101035013 MINEON (H2020-FETOPEN-03-2018-2019-2020 – FET Innovation Launchpad). SuperSTEM

(D.K.) is the UK National Facility for Advanced Electron Microscopy funded by the Engineering and Physical Sciences Research Council (EPSRC). P.M.V. acknowledges support from DOE BES

(DE-FG02-08ER46547). G.G.D.H. and X.L. acknowledge support from Brandeis NSF MRSEC, Bioinspired Soft Materials, DMR-2011846. Work at the Molecular Foundry (C.O.) was supported by the Office

of Science, Office of Basic Energy Sciences, of the US Department of Energy under contract no. DE-AC02-05CH11231. E.S. and M.K.Y.C. acknowledges support from the Center for Nanoscale

Materials (DOE SUF) under contract no. DE-AC02-06CH11357. C.O. and M.K.Y.C. acknowledge support from DOE Early Career Research Awards. N.S. acknowledges support from the JSPS KAKENHI (grants

20H05659 and 19H05788). J.E. acknowledges the support of the Australian Research Council Discovery Project (grants DP150104483 and DP160104679). The authors are extremely grateful to K.

More (from CNMS) for careful reading and editing of the manuscript. AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak

Ridge, TN, USA Sergei V. Kalinin, Andrew R. Lupini, Mark P. Oxley & Maxim Ziatdinov * NCEM, Molecular Foundry, Lawrence Berkeley National Laboratory, Berkeley, CA, USA Colin Ophus *

Department of Materials Science and Engineering, University of Wisconsin-Madison, Madison, WI, USA Paul M. Voyles * Electron Microscopy Center, EMPA – Swiss Federal Laboratories for

Materials Science and Technology, Dubendorf, Switzerland Rolf Erni * SuperSTEM laboratory, Sci-Tech Daresbury Campus, Daresbury, UK Demie Kepaptsoglou * Department of Physics, University of

York, York, UK Demie Kepaptsoglou * CNR-NANO, Modena, Italy Vincenzo Grillo * Center for Nanoscale Materials, Argonne National Laboratory, Lemont, IL, USA Eric Schwenker & Maria K. Y.

Chan * Materials Science and Engineering Department, Northwestern University, Evanston, IL, USA Eric Schwenker * Monash Centre for Electron Microscopy and Department of Materials Science and

Engineering, Monash University, Clayton, Victoria, Australia Joanne Etheridge * Department of Chemistry, Brandeis University, Waltham, MA, USA Xiang Li & Grace G. D. Han * Computational

Sciences and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, TN, USA Maxim Ziatdinov * Institute of Engineering Innovation, School of Engineering, The University of Tokyo,

Tokyo, Japan Naoya Shibata * Department of Materials Science and Engineering, University of Tennessee, Knoxville, TN, USA Stephen J. Pennycook * School of Physical Sciences and CAS Key

Laboratory of Vacuum Sciences, University of Chinese Academy of Sciences, Beijing, China Stephen J. Pennycook Authors * Sergei V. Kalinin View author publications You can also search for

this author inPubMed Google Scholar * Colin Ophus View author publications You can also search for this author inPubMed Google Scholar * Paul M. Voyles View author publications You can also

search for this author inPubMed Google Scholar * Rolf Erni View author publications You can also search for this author inPubMed Google Scholar * Demie Kepaptsoglou View author publications

You can also search for this author inPubMed Google Scholar * Vincenzo Grillo View author publications You can also search for this author inPubMed Google Scholar * Andrew R. Lupini View

author publications You can also search for this author inPubMed Google Scholar * Mark P. Oxley View author publications You can also search for this author inPubMed Google Scholar * Eric

Schwenker View author publications You can also search for this author inPubMed Google Scholar * Maria K. Y. Chan View author publications You can also search for this author inPubMed Google

Scholar * Joanne Etheridge View author publications You can also search for this author inPubMed Google Scholar * Xiang Li View author publications You can also search for this author

inPubMed Google Scholar * Grace G. D. Han View author publications You can also search for this author inPubMed Google Scholar * Maxim Ziatdinov View author publications You can also search

for this author inPubMed Google Scholar * Naoya Shibata View author publications You can also search for this author inPubMed Google Scholar * Stephen J. Pennycook View author publications

You can also search for this author inPubMed Google Scholar CONTRIBUTIONS Introduction (S.J.P. and M.P.O.); Experimentation (J.E., C.O., A.R.L., D.K., R.E. and V.G.); Results (P.M.V., X.L.

and G.G.D.H.); Applications (N.S., E.S. and M.K.Y.C.); Reproducibility and data deposition (C.O. and M.Z.); Limitations and optimizations (C.O. and S.V.K.); Outlook (C.O., M.Z. and S.V.K.);

Overview of the Primer (S.V.K.). CORRESPONDING AUTHOR Correspondence to Sergei V. Kalinin. ETHICS DECLARATIONS COMPETING INTERESTS The authors declare no competing interests. PEER REVIEW

PEER REVIEW INFORMATION _Nature Reviews Methods Primers_ thanks Jamie Warner and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. ADDITIONAL

INFORMATION PUBLISHER’S NOTE Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. RELATED LINKS ABTEM:

https://doi.org/10.12688/openreseurope.13015.1 ATOMAI: https://github.com/pycroscopy/atomai ESPRIT FROM BRUKER:

https://www.bruker.com/en/products-and-solutions/elemental-analyzers/eds-wds-ebsd-SEM-Micro-XRF/software-esprit-family.html GATAN MICROSCOPY SUITE:

https://www.gatan.com/products/tem-analysis/gatan-microscopy-suite-software GITHUB: https://github.com/ HYPERSPY: https://doi.org/10.5281/zenodo.592838 IMAGEJ: https://imagej.nih.gov/ij/

LABVIEW: https://www.ni.com/en-us/shop/labview.html MATLAB: https://www.mathworks.com/products/matlab.html OCTAVE: https://www.gnu.org/software/octave/index PYTEMLIB:

https://github.com/pycroscopy/pyTEMlib STEMTOOLS TOOLKIT: https://github.com/pycroscopy/stemtool VELOX FROM THERMO FISHER SCIENTIFIC:

https://www.thermofisher.com/uk/en/home/electron-microscopy/products/software-em-3d-vis/velox-software.html SUPPLEMENTARY INFORMATION SUPPLEMENTARY INFORMATION GLOSSARY * Aberration An

imperfection in the electron optics of a microscope. * Coherent imaging Measurements where the local contrast is dominated by the phase alignment of the electron wavefronts: constructive

interference (in phase) leads to higher signals and destructive inference (out of phase) leads to lower signals. * Incoherent imaging When the coherence length of the electron waves is

smaller than the resolution element of the measurement, the total signal is given incoherently by the sum of individual electron wavefunction intensities, and the relative phase of these

wavefronts does not affect the measured intensity. * Contrast The spatial variation of intensity. * Z-contrast imaging A scanning transmission electron microscopy-high-angle annular

dark-field imaging method, where the image contrast scales roughly monotonically with the atomic number _Z_ of the atom(s) being imaged, approximately as _Z_1.7. * Ptychography A method of

generating images from many coherent diffraction patterns formed at different probe positions in the STEM. It is also widely used in X-ray scattering experiments. * Tilt series tomography By

tilting the specimen and recording projected images at different angles, computer algorithms can be used to reconstruct the 3D sample structure. * Transmission modes Imaging modes in

electron microscopy where the electron beam passes through the specimen. * Differential phase contrast A method that measures the change in the convergent beam diffraction pattern as a

function of probe position using either a segmented or a pixelated detector. These changes can be related to the local change in the sample’s potential and corresponding fields. * Phonons A

quantized collective vibration of atoms in a crystalline sample, which can be excited by the electron beam and characterized by scanning transmission electron microscopy-electron energy loss

spectroscopy or diffraction measurements. * Plasmons A quantized collective oscillation of electrons relative to the fixed ions in a sample, which can be excited by the electron beam and

characterized by scanning transmission electron microscopy-electron energy loss spectroscopy. * Core-loss edges Excitation of inner-shell electrons (ionization) by the electron beam, where

the energy loss can be probed by scanning transmission electron microscopy-electron energy loss spectroscopy for features referred to as ‘edges’. * Dynamical scattering A term commonly used

in electron microscopy to describe the multiple scattering of the incident electron probe as it propagates through the specimen. * Electron optical elements Electromagnetic lenses used to

focus or otherwise shape the electron beam. * Electron holography A technique for viewing the phase of the exit surface wavefunction using the interference of a scattered and unscattered

electron beam. * Azimuthal phase gradient (APG). A wavefunction where the phase is linearly proportional to the angle in polar coordinates, and the total phase shift is an integer multiple

of 2π for each revolution (see orbital angular momentum). * Orbital angular momentum (OAM). Orbital angular momenta are quanta given by the number of multiples of 2π in the phase of an

electron beam, per angular revolution in polar coordinates (see azimuthal phase gradient). * Electron energy loss near edge structure The intensity variation of the electron energy loss

spectroscopy signal as a function of energy loss near the onset of the core-loss signal. * L23 ratio The ratio of the L3 to L2 peaks formed by the transition of the 2_p_3/2 and 2_p_1/2

electrons to empty states. * Dark count rate (_D_). This is the mean value of a scanning transmission electron microscopy image acquired with the beam blanked preferably near the gun by, for

example, closing the gun vacuum valve. * Gain (_G_). Adjustment to ensure that the measured signal covers the optimal range of the amplifier. * Faraday cup A conductive cup that can capture

charged free particles, with which the electron beam current can be estimated by integrating the recorded signal. * Residual The difference between the fitted image and the experimental

image after atom location. * Latent variables A variable that is not directly observable, often obtained using variational auto encoders. * Latent spaces A vector space spanned by the latent

variables. * Evidence lower boundary The lower bound of the probability of observing a particular result for a given model. * Penrose structures Local structural units that, when displaced

and rotated, can fully tile space, but do not have periodic translational symmetry. Such atomic structures can be found in quasicrystals. * Electron beam irradiation This occurs when an

electron beam induces changes in a specimen due to energy transfer, often called beam damage. * Dwell time The time period of the data collection in each pixel. * Inferential biases The

assumptions and constraints implemented in the structure of the network, loss function or training set that impose specific limitations on the outputs. * Exploration Uncertainty

minimization. * Exploitation Balancing exploration and pursuing target functionalities. * Out-of-distribution data When observational conditions change between experiments, precluding a

direct comparison of data between experiments. * Distribution shift In machine learning, this shift occurs when training and test sets do not come from the same distribution. * Knock-on

damage thresholds The energy of the incident electron required to remove an atom from the crystal lattice. RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE THIS

ARTICLE Kalinin, S.V., Ophus, C., Voyles, P.M. _et al._ Machine learning in scanning transmission electron microscopy. _Nat Rev Methods Primers_ 2, 11 (2022).

https://doi.org/10.1038/s43586-022-00095-w Download citation * Accepted: 14 January 2022 * Published: 03 March 2022 * DOI: https://doi.org/10.1038/s43586-022-00095-w SHARE THIS ARTICLE

Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided

by the Springer Nature SharedIt content-sharing initiative