DeepMind and scholars from Imperial College London are focusing on using neural network methods to approximate solutions to the classic Schrödinger equation in quantum mechanics. Following the proposal of FermiNet in 2020, the team's latest achievement - solving quantum excited states, was published in Science. 

After AlphaFold 3 sparked a biological revolution, DeepMind is about to start working on quantum mechanics. 

01 Research result

On August 22, their latest research result, FermiNet, was published in Science, using neural networks to accurately calculate quantum excited states.

Paper address: 

https://www.science.org/doi/abs/10.1126/science.adn0137

The paper introduces an algorithm for estimating the excited states of quantum systems through variational Monte Carlo, but instead transforms the problem into a problem of finding the ground state of an extended system, making it very suitable for neural network analysis. The authors verified the accuracy of this method by running it on two different neural networks: FermiNet and Psiformer. FerminNet was also jointly proposed by DeepMind and Imperial College London in 2020, and the paper was published in the journal Physical Review Research.

This paper shows how deep learning can help solve the basic equations of quantum mechanics, and the innovative network architecture FermiNet proposed is very suitable for modeling the quantum states of large collections of electrons (i.e., the basic building blocks of chemical bonds). More importantly, FermiNet demonstrated for the first time how to use deep learning methods to calculate the energy of atoms and molecules based on first principles with enough accuracy to play a practical role. This is not only an important basic scientific problem, but also may have practical uses in the future, such as using computers for simulation or prototyping before making new materials and chemical syntheses. In addition, FermiNet may have a wide range of uses in protein folding, glass dynamics, lattice quantum chromodynamics and other fields. FeimiNet related code has been published on GitHub for reference in related research in the fields of computational physics and computational chemistry.

02 Quantum mechanics equations

In addition to FermiNet, Psiformer, a self-attention-based architecture released by DeepMind in 2022, remains the most accurate AI method to date for solving quantum mechanics equations.

Paper address: https://arxiv.org/abs/2211.13672

03 A brief history of quantum mechanics

Nothing is more confusing than the four words "quantum mechanics". After all, this is a field that even Planck was confused about.

For example, the dead and alive cat in Schrödinger's thought experiment, and something that classical physics cannot describe at all - the simultaneous existence of particle and wave properties.

In the classical model, the nucleus is located in the middle, and a fixed number of electrons orbit around fixed orbits, as rigorous and orderly as the solar system.

But in quantum systems, particles such as electrons do not have such precise orbits at all, and their positions are described by "probability clouds". What does "probability cloud" mean? This model tells us that electrons in atoms do not have fixed motion trajectories or exact positions, and can only use "electron probability density" to describe the probability of their appearance in a certain area. However, the probability cloud model only gives us "probability" and cannot really determine whether the electron is in a certain area at any particular moment.

This situation is beyond the scope of human comprehension, so the famous physicist Richard Feynman declared: "If you think you understand quantum mechanics, then you don't understand quantum mechanics at all." Although it is difficult to understand intuitively and difficult to explain in words, the core content of the theory can be described in very concise mathematical language. So the "Talk is cheap, show me your code" often heard in the field of computer science has a similar phrase in theoretical physics - "Shut up and calculate." Among the core equations, the most famous is the Schrödinger equation, which is sufficient to describe the behavior of all familiar substances at the atomic and nuclear levels.

Covalent bonds in chemistry, or the various counterintuitive properties of superconductors, superfluids, lasers and semiconductors are all the result of quantum interactions between electrons, and can all be described by the Schrödinger equation. In the 1920s, after the relevant rules of quantum mechanics emerged, scientists could for the first time use detailed theories to describe the underlying principles of chemical reactions. For example, by establishing corresponding equations for different molecules and solving the energy of the system, it is possible to find out which molecules are in a stable state and which molecules will react spontaneously. However, the ideal is beautiful, but the reality is very skinny. When scientists actually started to calculate, they found that only the equations of the hydrogen atom could be solved - the others were too complicated to calculate. Therefore, one of the founders of quantum mechanics, physicist Paul Dirac, said this in 1929, which is still applicable today: "The basic physical laws necessary to describe most of the mathematical theories of physics and chemistry as a whole are completely known. The only difficulty is that if these laws are applied accurately, the equations will be too complex to be solved." "So, we need to develop approximate methods that can apply quantum mechanics." After Dirac, physicists used an approximate method to assign electrons to a specific orbit, and the shape of each orbit was obtained by averaging all other orbits. This "mean field" method only assigns one orbit to each electron, so the description of the actual behavior of electrons is very incomplete, but it is still a feasible method, and the error in estimating the total energy of molecules is about 0.5%.

However, a method with an error of 0.5% is still not enough. You should know that the molecular bond energy only accounts for a small part of the total energy of the system. To correctly predict whether a molecule is in a stable state, you need to rely on the energy difference of 0.001%, or 0.02% of the remaining relevant energy. Compared with the electron mean field, various calculation methods developed in recent decades have made progress, but they cannot always achieve satisfactory accuracy and calculation efficiency at the same time.

04 Neural networks tailored for fermions

The challenge of representing quantum systems is that you must assign a probability to each possible electron position, which creates an extremely large configuration space. For example, the number of possible electron configurations for silicon atoms alone is greater than the number of atoms in the universe combined. This is where deep neural networks come in. In the past few years, neural networks have made great progress in representing complex, high-dimensional probability distributions, and can be trained in an efficient and scalable way. Their ability to fit high-dimensional functions in the field of AI may also be used to represent quantum wave functions.

Pauli Exclusion Principle

When dealing with electrons, there is another issue to consider - electrons must obey the Pauli exclusion principle, which means that they cannot be in the same space at the same time. Because electrons belong to "fermions", their wave functions must be antisymmetric. If the positions of two electrons are swapped, the wave function will be multiplied by -1. This means that if two electrons overlap each other, the wave function (and the probability of that configuration) will be zero. This means that we need to develop a new neural network that accepts inputs of antisymmetric structures - this is FermiNet. The determinant of a matrix happens to have this "antisymmetric" property: if two rows are swapped, the output is multiplied by -1, just like the wave function of a fermion. Therefore, in most quantum chemistry methods, "antisymmetry" is introduced through the determinant. The Slater determinant adopts this idea: each electron in the system is evaluated using a single-electron function, and all the results are packed into a matrix, the determinant of which is the antisymmetric wave function.

Slater determinant: Each curve is a slice of one of the orbitals shown above. When electrons 1 and 2 swap places, a row swap occurs in the Slater determinant, and the wave function is multiplied by -1, ensuring that the Pauli exclusion principle is followed.

Deep neural networks are often more efficient at representing complex functions than linear combinations of basis functions. In FermiNet, every function that goes into the determinant is a function of all electrons, which goes far beyond methods that build functions using only a single electron or two electrons (such as the Slater determinant). In addition, each electron in FermiNet corresponds to a separate information flow. But if there is no interaction between these flows, FermiNet's expressive power will not be any better than the Slater determinant. Therefore, the information of all flows in each layer of the network is averaged before it is passed to the information flow of the next layer, which is similar to the way graph neural networks aggregate information at each layer. Therefore, unlike the Slater determinant, if the neural network layers become wide enough, FermiNet can act as a universal function approximator. This means that if trained correctly, FermiNet can fit a nearly exact solution to the Schrödinger equation.

Variational Quantum Monte Carlo

If we want to accurately fit FermiNet by minimizing the energy of the system, we need to evaluate the wave function for all possible electron configurations. Obviously, we cannot calculate the exact value directly, but can only estimate it approximately. We randomly select electron configurations, evaluate the local energy of each electron configuration, and minimize it by adding them together. Instead of directly calculating the true energy minimum, this estimation method is called the Monte Carlo method, which is a bit like a gambler rolling the dice over and over again. Although the result is only an approximation, we can always roll the dice again if we need to make it more accurate. Where do these samples of electron configurations come from? Since the square of the wave function gives the probability of observing a certain particle configuration at any position, it is most convenient to generate samples from the wave function itself - essentially simulating the behavior of observing particles. While most neural networks are trained based on external data, in FermiNet, the input data used for training is generated by the neural network itself. This means that we do not need any additional training data other than specifying the structure and position of the atomic nucleus. This idea is called "variational quantum Monte Carlo" (VMC). Although it was proposed in the 1960s, it is generally considered a low-cost and inaccurate calculation method. By replacing the Slater determinant with FermiNet, VMC has been successfully "renovated" and its accuracy has been greatly improved.

05 Experimental Results

We started with simple, well-studied systems: atoms in the first row of the periodic table (hydrogen to neon). These are small systems with ≤10 electrons and are simple enough—even simple enough to be handled by the most accurate and expensive (exponentially scaling) methods. We found that FermiNet outperformed similar VMC methods by a wide margin—often reducing the error by half or more relative to methods that grow exponentially in computational complexity. On larger systems, exponentially scaling methods were too complex, so a coupled cluster method was used as a baseline. Coupled cluster methods work well for molecules with stable configurations, but have difficulty dealing with situations where chemical bonds are stretched or broken, which is crucial to understanding chemical reactions. In addition, while this method is much more computationally efficient than exponentially scaling methods, the computational complexity of the method used in the experiment still grows at the seventh power of the number of electrons, so it can only be used for molecules of moderate size. The next stage is to apply FermiNet to progressively larger molecules, from lithium hydride to bicyclobutane. "For the smallest molecules, FermiNet captures 99.8% of the difference between the coupled cluster energy and the energy obtained from a single Slater determinant. For bicyclobutane, the largest system we have ever studied, with 30 electrons, FermiNet still captures 97% or more of the correlation energy, which is a huge achievement considering the simplicity of the method we applied."

06 New method for calculating excited states

In addition to FermiNet, proposed in 2020, in the latest results published in Science, DeepMind proposed a solution to one of the most difficult challenges in computational quantum chemistry - understanding how molecules transition between excited states. FermiNet initially focused on the ground state of molecules, that is, given a set of atomic nuclei, find the lowest energy arrangement of electrons around them. However, when molecules and materials are excited by a large amount of energy, such as light or high temperature, electrons may enter a higher energy state - the excited state. Excited states are the basis for understanding the interaction of matter with light, and different molecules and materials absorb/release the exact amount of energy, which is equivalent to their unique fingerprint. Understanding the principles in this regard affects the performance of technologies such as solar panels, LEDs, semiconductors, photocatalysts, and also plays a key role in biological processes involving light, such as photosynthesis and vision. Accurately calculating excited state energies is more challenging than calculating ground state energies. Even the gold benchmark method for ground state chemistry, such as the coupled clusters mentioned above, can have errors tens of times larger when calculating excited state energies. Although we hope to extend FermiNet's work to excited states, at present, neural networks cannot compete with state-of-the-art methods. Therefore, this paper proposes a new method for calculating excited states that is more powerful and more general than previous methods and can be applied to any type of mathematical model, including FermiNet and other neural networks. It works by finding the ground state of an extended system with additional particles, so existing optimization algorithms can be used with minor modifications. The paper was verified on a wide range of benchmarks and achieved very ideal experimental results. On a small and complex molecule such as diatomic carbon (carbon dimer), the method achieved a mean absolute error (MAE) of 4meV, while the previous gold standard method had an error of 20meV, which is equivalent to five times closer to the experimental results.

Diatomic carbon diagram

In addition, the authors tested it on some of the most challenging systems in computational chemistry, such as when two electrons are excited at the same time. The results showed that the error was only 0.1 eV compared to the most demanding and complex calculations completed to date.

Reference: https://deepmind.google/discover/blog/ferminet-quantum-physics-and-chemistry-from-first-principles/

View more at EASELINK