Abstract: In this study, we explore the possibility to use deep learning for the reconstruction of phase images from 4D scanning transmission electron microscopy (4D-STEM) data. The process can be divided into two main steps. First, the complex electron wave function is recovered for a convergent beam electron diffraction pattern (CBED) using a convolutional neural network (CNN). Subsequently, a corresponding patch of the phase object is recovered using the phase object approximation. Repeating this for each scan position in a 4D-STEM dataset and combining the patches by complex summation yields the full-phase object. Each patch is recovered from a kernel of 3x3 adjacent CBEDs only, which eliminates common, large memory requirements and enables live processing during an experiment. The machine learning pipeline, data generation, and the reconstruction algorithm are presented. We demonstrate that the CNN can retrieve phase information beyond the aperture angle, enabling super-resolution imaging. The image contrast formation is evaluated showing a dependence on the thickness and atomic column type. Columns containing light and heavy elements can be imaged simultaneously and are distinguishable. The combination of super-resolution, good noise robustness, and intuitive image contrast characteristics makes the approach unique among live imaging methods in 4D-STEM.

Abstract: A real-time image reconstruction method for scanning transmission electron microscopy (STEM) is proposed. With an algorithm requiring only the center of mass of the diffraction pattern at one probe position at a time, it is able to update the resulting image each time a new probe position is visited without storing any intermediate diffraction patterns. The results show clear features at high spatial frequency, such as atomic column positions. It is also demonstrated that some common post-processing methods, such as band-pass filtering, can be directly integrated in the real-time processing flow. Compared with other reconstruction methods, the proposed method produces high-quality reconstructions with good noise robustness at extremely low memory and computational requirements. An efficient, interactive open source implementation of the concept is further presented, which is compatible with frame-based, as well as event-based camera/file types. This method provides the attractive feature of immediate feedback that microscope operators have become used to, for example, conventional high-angle annular dark field STEM imaging, allowing for rapid decision-making and fine-tuning to obtain the best possible images for beam-sensitive samples at the lowest possible dose.

Abstract: Transmission electron microscopy (TEM) is a technique that uses an electron beam to analyze materials. This analysis is based on the interaction between the electron beam and the sample, such as photon emission and electron diffraction pattern, to name a few. Sample damage, however, also occurs when such interaction alters the structure of the sample. To ensure information from the undamaged material can be acquired, the electron expense to probe the material is thus limited. In this work, we propose efficient methods for acquiring and processing the information originating from the electron-sample interaction so that the study of the material and the conducting of the TEM experiment can be less hindered by the limited dose usage. In the first part of the work, the relationship between the scattering of the electron and the local physical property of the sample is studied. Based on this relationship, two reconstruction schemes are proposed capable of producing high-resolution images at low-dose conditions. Besides, the proposed reconstructions are not restricted to complete datasets but instead work on pieces of data, therefore allowing live feedback during data acquisition. Such feature of the methods allows the whole TEM experiment to be carried out under low dose conditions and thus further reduces possible beam damage on the studied material. In the second part of the work, we discuss our approach to modulating the electron beam and its benefits. An electrostatic device that can alter the wavefront of the passing electron wave is introduced and characterized. The beam-modulation ability is demonstrated by creating orthogonal beam sets, and applications that exploit the adaptability of the wave modulator are demonstrated with both simulation and experiments.