Abstract: The simulation of transmission electron microscopy (TEM) images or diffraction patterns is often required to interpret their contrast and extract specimen features. This is especially true for high-resolution phase-contrast imaging of materials, but electron scattering simulations based on atomistic models are widely used in materials science and structural biology. Since electron scattering is dominated by the nuclear cores, the scattering potential is typically described by the widely applied independent atom model. This approximation is fast and fairly accurate, especially for scanning TEM (STEM) annular dark-field contrast, but it completely neglects valence bonding and its effect on the transmitting electrons. However, an emerging trend in electron microscopy is to use new instrumentation and methods to extract the maximum amount of information from each electron. This is evident in the increasing popularity of techniques such as 4D-STEM combined with ptychography in materials science, and cryogenic microcrystal electron diffraction in structural biology, where subtle differences in the scattering potential may be both measurable and contain additional insights. Thus, there is increasing interest in electron scattering simulations based on electrostatic potentials obtained from first principles, mainly via density functional theory, which was previously mainly required for holography. In this Review, we discuss the motivation and basis for these developments, survey the pioneering work that has been published thus far, and give our outlook for the future. We argue that a physically better justified ab initio description of the scattering potential is both useful and viable for an increasing number of systems, and we expect such simulations to steadily gain in popularity and importance.