A Maximum Entropy Principle in Deep Thermalization
and in Hilbert-Space Ergodicity

The first ensemble of states we study is the temporal ensemble, consisting of quantum states generated by time evolution of an initial state $\ket{\Psi_{0}}$ under a Hamiltonian $H$ [Fig. 3(a)], with probabilities being uniform on the time interval $t\in[-\tau/2,\tau/2]$.

This ensemble consists of the trajectory of a quantum state as it evolves under $H$. The statistical properties of the trajectory quantify its degree of ergodicity in Hilbert space, as our results will make clear.

Conventionally, quantum ergodicity is formulated in terms of the energy eigenstates of a system [68, 69, 70]. While this applies to time-independent Hamiltonian evolution or Floquet dynamics, it is not applicable to quantum dynamics in general. Furthermore, the notion of ergodicity is fundamentally dynamical in nature, and its definition in terms of static quantities such as energy eigenstates illustrates a tension in our understanding of quantum ergodicity. Recent works [39, 40] challenge this formulation. Specifically, they show that the ergodicity of certain time-dependent Hamiltonian dynamics cannot be formulated using conventional approaches based on the statistical properties of (quasi)-energy eigenstates, but instead can be characterized by the statistical properties of their trajectories. The central idea of Refs. [39, 40] is to test if a time-evolved state uniformly visits all ambient points in the Hilbert space, a property dubbed (complete) Hilbert-space ergodicity.

Our work generalizes the notion of Hilbert-space ergodicity [39] to systems with energy conservation. The presence of energy conservation explicitly disallows ergodicity over all of Hilbert space. Nevertheless, we find that many-body dynamics under time-independent Hamiltonian evolution is still Hilbert-space ergodic, in the sense that the evolved states uniformly explore their available Hilbert space, which is constrained due to energy conservation. In order to understand the effects of energy conservation, consider the simple case of a single qubit initially polarized in the $x$-direction, subject to a magnetic field along the $z$-axis. Over time, the qubit periodically traces out an equatorial circle on the Bloch sphere, instead of uniformly covering the sphere. In contrast, quantum dynamics without such conservation laws, specifically a quasiperiodically driven system, can uniformly explore Hilbert space [39].

As in classical dynamics [71], we find that a quantum many-body system may be Hilbert-space ergodic without being quantum chaotic (i.e. being integrable). We show that Hilbert-space ergodicity holds as long as the system satisfies the no-resonance conditions (Definition 3), colloquially the absence of higher order resonances in the spectrum (Definition 3). This condition is widely assumed to be true in chaotic systems [69, 17, 72, 13], and we numerically observe that it is also true in certain integrable systems such as the XXZ model, but not in others such as the transverse field Ising model.

To prove our claim of Hilbert-space ergodicity, we show that in the $\tau\rightarrow\infty$ infinite interval limit, the temporal ensemble is equal to the random phase ensemble [46, 73]. This is the ensemble of states with fixed amplitudes $|c_{E}|\equiv|\langle E|\Psi_{0}\rangle|$ in some basis $\{|E\rangle\}$, here the energy eigenbasis, and with complex phases $\text{arg}(c_{E})$ independently random from the uniform distribution $\text{Unif}([0,2\pi))$. In a time-evolved state, the amplitudes $|c_{E}|^{2}$ are fixed by the initial state and Hamiltonian, and the phases are the only degrees of freedom.

The above Theorem is somewhat of a folklore result [72, 70, 46], often stated without proof. Here we provide a full statement and proof, for several reasons. Firstly, Theorem 1 has a natural interpretation in terms of a maximum entropy principle. The random phase ensemble has maximum ensemble entropy [Eq. (2)] on this restricted Hilbert space, and in Sec. IV.2, we quantify the rate of convergence of the temporal ensemble to the random phase ensemble with increasing interval $\tau$. Next, this leads to a novel type of universality in the many-body setting, which we outline below.

Unlike the above example of a two-level system, the exponentially larger Hilbert space in many-body systems leads to behaviour much more akin to ergodicity in the full Hilbert space. Specifically, we show that the trajectory appears pseudorandom in the following sense: in the limit of large Hilbert space dimension $D$, the temporal ensemble is statistically equal to the Haar ensemble, deformed as follows

where $\rho_{d}\equiv\sum_{E}|E\rangle\!\langle E|\Psi_{0}\rangle\!\langle\Psi_{0}|E\rangle\!\langle E|$ is the diagonal ensemble. By “statistically equal,” we mean that Eq. (4) is not true on the level of wavefunctions, but the moments $\rho^{(k)}$ of the ensembles are very close. To show this, we develope a simple analytic expression for the moments of the temporal ensemble that holds in the $D\rightarrow\infty$ limit (Sec. IV.1). One consequence of this pseudorandomness is our finding that the measurement probabilities, upon appropriate rescaling, universally follow the Porter-Thomas distribution (below, and Sec. IV.3), a hallmark of random quantum states which had been observed in quantum dynamics without conservation laws [74]. We discuss information-theoretic aspects of this result and also prove an analog of the ergodic theorem, a result in the theory of dynamical systems [75], where we relate the statistics defined over time and configuration space (discussed below and in Sec. IV.5).

Not only do temporal ensembles satisfy the maximum entropy principle, so do projected ensembles. Projected ensembles, introduced in a modern context by Refs. [28, 29] (and previously studied in Ref. [76]), are ensembles of states obtained from a single, larger many-body state $\ket{\Psi}$. To do so, we partition the system into two parts $A$ and $B$, such that the Hilbert space factorizes (for simplicity of discussion) as $\mathcal{H}=\mathcal{H}_{A}\otimes\mathcal{H}_{B}$. We then measure $B$ in a basis $\{\ket{z_{B}}\}$. Each measurement outcome $\ket{z_{B}}$ projects $\ket{\Psi}$ into a distinct pure state $\ket{\Psi(z_{B})}\in\mathcal{H}_{A}$ [Fig. 1(d)], defined as

where the normalization constant is the probability of obtaining the outcome $z_{B}$, $p(z_{B})\equiv\bra{\Psi}(\mathbb{I}_{A}\otimes\outerproduct{z_{B}}{z_{B}})\ket{\Psi}$. This defines the projected ensemble:

For concreteness, we primarily study global states $\ket{\Psi}=\exp(-iHt)\ket{\Psi_{0}}$ obtained from time-evolution by a generic, ergodic many-body Hamiltonian $H$, which rapidly reach thermal equilibrium at a temperature set by the energy of the initial state [51, 52]. We will also study eigenstates of $H$ that exhibit similar thermal properties.

In Refs. [28, 29], it was found that when the state $\ket{\Psi}$ is obtained from time-evolution under a chaotic many-body Hamiltonian at infinite effective temperature, the projected ensemble of small subsystems is very well-approximated by the Haar ensemble. This phenomenon is known as deep thermalization [30]. This has since been rigorously shown in a variety of settings, including deep random circuits [29, 36], dual-unitary models [77, 30, 31, 32, 78], free fermion models [33], or Hamiltonian dynamics with certain assumptions [34]. Various aspects including the rate of deep thermalization [30, 31, 78, 36], the effect of symmetries [35, 37], and the presence of magic in the global state [38] have been investigated. The majority of what we know so far is restricted to the setting of infinite temperature dynamics. In this work, we set out to study the projected ensembles formed in finite temperature dynamics.

At finite temperature, projected ensembles are not close to the Haar ensemble. To see this, note that the first moment of the projected ensemble $\rho^{(1)}_{\mathcal{E}}\equiv\sum_{z_{B}}p(z_{B})\outerproduct*{\Psi(z_{B})}{\Psi(z_{B})}=\text{tr}_{B}[\outerproduct{\Psi}{\Psi}]$ is precisely equal to the reduced density matrix (RDM). In turn, only at infinite temperature is the RDM equal to the first moment of the Haar ensemble, the maximally mixed state $\mathbb{I}_{A}/D_{A}$. This follows from a standard result in quantum thermalization [69] that the reduced density matrix $\rho_{A}$ is well described by a Gibbs state, i.e. $\rho_{A}\propto\exp(-\beta H_{A})$, where $H_{A}$ is the Hamiltonian truncated to the subsystem $A$, and $\beta^{-1}$ is an effective temperature. It has remained an open question as to how to describe these projected ensembles at finite temperature.

In this work, we find that a maximum entropy principle still applies: under an “energy non-revealing” condition on the measurement basis $\{|z_{B}\rangle\}$ which we shall discuss below, the projected ensembles are described by the ensembles that have maximum entropy, over all ensembles with a given first moment. These ensembles have been previously studied and were termed Scrooge ensembles [79, 80, 81] for their unique information-theoretic properties³³3As a technical point, Scrooge ensembles only have maximum entropy when they are regarded as ensembles of unnormalized states [80], see Sec. III.. Each density matrix $\rho$ uniquely defines a Scrooge ensemble $\text{Scrooge}(\rho)$, and our result is that at finite temperatures, the projected ensemble is the Scrooge ensemble corresponding to the appropriate thermal RDM.

The Scrooge ensemble may be essentially understood as a deformation of the Haar ensemble such that its first moment equals $\rho$ [79] [Fig. 4(b)]. Specifically, we can form the Scrooge ensemble by first sampling Haar-random states $\ket{\psi}\sim\text{Haar}$, then deforming them by a given density matrix $\rho$, i.e.

The probability $p(\Phi)$ is further weighted by the norm $\bra{\psi}\rho\ket{\psi}$, leading to the distribution of states

In Sec. V.3, we give closed form expressions for the resultant probability distribution function $p(\Phi)$, as well as its $k$-th moments.

Scrooge ensembles do not always describe the projected ensemble. They only do so in the special case when the measurement basis $z_{B}$ is uncorrelated with the energy, defined precisely in Sec. VI.4. Under these circumstances, we can treat the projected states $\ket{\Psi(z_{B})}$ as independent, identical samples from a distribution. However, this is not always true: when $z_{B}$ is correlated with the subsystem energy, it is also correlated with the projected state $\ket*{\Psi(z_{B})}$, such that the energies $E_{A}\equiv\langle\Psi(z_{B})|H_{A}|\Psi(z_{B})\rangle$ and $E_{B}\equiv\langle z_{B}|H_{B}|z_{B}\rangle$ are anticorrelated. Under these general conditions, we introduce the generalized Scrooge ensemble to describe the projected ensemble (Section VI). Each projected state $|\Psi(z_{B})\rangle$ should be treated as a sample from a different Scrooge ensemble for each $z_{B}$, i.e. where the first moment is not the RDM $\rho_{A}$, but is a $z_{B}$-dependent mixed state $\bar{\rho}(z_{B})$, which we may determine by the time-average of the projected state, $\mathbb{E}_{t}[|\Psi(z_{B},t)\rangle\!\langle\Psi(z_{B},t)|]$. Therefore, the generalized Scrooge ensemble is more accurately a collection of ensembles, where the projected ensemble is comprised of exactly one sample from each ensemble, but we will abuse notation and refer to it as an ensemble. This is our second key result.

When the measurement basis $\{z_{B}\}$ is uncorrelated with the global energy, the generalized Scrooge ensemble reduces to the Scrooge ensemble (Claim 1). Specifically, this happens when the mixed state $\bar{\rho}(z_{B})$ is independent of $z_{B}$ (up to a rescaling), giving a concrete definition for “energy non-revealing” bases.

We prove this result using our knowledge of the temporal ensemble. We show that the temporal trajectory of the projected state $|\Psi(z_{B},t)\rangle$ is a Scrooge ensemble (here, because of the subsystem projection, there are no hard constraints on the amplitudes, as there were in the case of the global state). We then utilize our ergodic theorem (Sec. IV.5) to translate our results for a distribution over time to a distribution over the outcomes $z_{B}$.

While Theorem 4 concerns the time-evolved states $\ket{\Psi(t)}$, we numerically observe that the generalized Scrooge ensemble also describes the projected ensembles obtained from energy eigenstates, suggesting that our maximum entropy principle holds for a broader class of many-body states.

In projected ensembles, the maximum entropy principle sheds light on the nature of correlations between system and bath. Specifically, the projected ensemble reveals correlations between the subsystem $A$ and its complement $B$, which we quantify in terms of the mutual information. We find that the nontrivial correlations between them are minimal and basis independent. Here, nontrivial correlations mean any amount of correlation (mutual information) that is beyond the thermal correlations that remain after infinite-time averaging. In other words, any meaningful correlations between $A$ and $B$ originate only from energy conservation, and other symmetries if they exist. Below and in Sec. V.8 and VI.3, we discuss consequences of these correlations for tasks such as communication and compression.

Below, we summarize several broader points that we found applicable to both the temporal and projected ensemble.

Ergodic theorem — Remarkably, the temporal and projected ensembles are closely related. We prove that a version of the ergodic theorem holds in many-body dynamics. This enables us to translate statements about distributions over time (temporal ensemble) into statements about distributions over measurement outcomes, at fixed, typical points in time. In particular, we use this to prove Theorem 4, a statement about the projected ensembles at typical points in time.

Operational meaning of ensemble entropy — In this work, we use the ensemble entropy to define our maximum entropy principle. We also identify its operational interpretation: the ensemble entropy quantifies the information required to classically represent an ensemble of states. Our maximally entropic ensembles are therefore the most difficult to compress, formalizing our intuition that the states generated by natural, chaotic many-body dynamics lack overall structure that can be exploited for compression.

Specifically, the ensemble entropy quantifies the number of classical bits required to store an ensemble of states up to numerical precision $\epsilon$ [Fig. 2(a)]. This should be contrasted with the minimum number of qubits required to store the ensemble in quantum memory, which was established by Schumacher to depend only on the first moment $\rho_{\mathcal{E}}$ of the ensemble through the von Neumann entropy $S(\rho_{\mathcal{E}})$ [82].

Information-theoretic consequences — Not only are our ensembles of states maximally difficult to classically store, they are also maximally difficult to distinguish by measurement, as quantified by their accessible information [79, 63]. The accessible information quantifies the maximum rate of classical information that can be transmitted by sending states from the ensemble over a quantum channel (in terms of bits of information per channel use) [Fig. 2(b)]. It is upper-bounded by the well-known Holevo bound [56], and lower-bounded by Josza, Robb, and Wooters [79]. They found that the Scrooge ensemble achieves this lower bound, dubbed the subentropy $Q(\rho)$ [Eq. (61)]. They dubbed the ensemble the “Scrooge ensemble” because it is uniquely “stingy” in the amount of information revealed under measurement. Therefore, the projected ensemble not only has maximum ensemble entropy, it also minimizes the amount of information that can be extracted. These information theoretic properties are directly measurable [Fig. 4(f) and Fig. 5(d)] but have not yet been observed in experiment.

In the case of temporal ensembles, the accessible information quantifies the amount of information that can be gained about the evolution time of the state. We find that it takes on a universal value $(1-\gamma)/\ln 2\approx 0.61$, where $\gamma$ is the Euler-Mascheroni constant [Fig. 3(d)].

Measurable signature: universal Porter-Thomas distribution — Our maximum entropy principle selects distributions of states that are “closest” to the Haar ensemble, satisfying certain constraints. The relationship between our ensembles of states to the Haar ensemble gives rise to a measurable signature of our maximum entropy principle: the Porter-Thomas (PT) distribution [83]. Originally identified in nuclear physics settings, the PT distribution has received renewed attention in many-body physics. This distribution represents the distribution of overlaps between Haar random states $\ket{\psi}$ with a fixed (but arbitrary) state $\ket{\Phi}$, i.e. the probability $p(\Phi)\equiv|\langle\Phi|\psi\rangle|^{2}$ of measuring $\ket{\Phi}$ from a Haar-random state $\ket{\psi}$ [84]. Not only is the PT distribution a measurable signature of Haar-random states, we find that it is also present in many natural many-body states. In companion work [41], we experimentally and theoretically investigate the emergence of the Porter-Thomas distribution in global quantities, its relationship to local quantities, and how the Porter-Thomas distribution is modified in the presence of noise, using our results to learn about the noise in an experiment.

The rest of the paper is organized as follows. In Section III, we provide an overview of the central object of our work — ensembles of pure states and their information-theoretic and measurable properties. We then proceed to study our two ensembles of states: the temporal ensemble (Section IV) and the projected ensemble in a special and a general setting (Sections V and VI). Finally, in Section VII we discuss the PT distribution in the properties of many-body eigenstates.

The ensemble entropy, which quantifies our maximum entropy principle, is defined with reference to the Haar (or Gaussian) ensemble. These ensembles are unique in being invariant under arbitrary unitary transformations and have been the subject of considerable study as paradigmatic distributions of random quantum states [48].

Random unnormalized states: the Gaussian ensemble — The Gaussian ensemble is a probability distribution over unnormalized states, where each component $\tilde{\Psi}_{z}\equiv\innerproduct*{z}{\tilde{\Psi}}$ is an independent complex Gaussian variable.

The normalization above is chosen so that the average norm is $\mathbb{E}[\innerproduct*{\tilde{\Psi}}{\tilde{\Psi}}]=1$.

Random normalized states: the Haar ensemble — The Haar ensemble is the unique distribution of normalized states that is invariant under any unitary transformation $U$. It is naturally related to the Gaussian ensemble: one can sample from the Haar ensemble by sampling states $\ket{\Psi}$ from the Gaussian ensemble, then normalizing them. This induces the measure

The Haar ensemble has been widely studied in quantum information science and utilized for a myriad of applications, see e.g. Ref. [48]. In particular, the moments of the Haar ensemble play a key role, which we now discuss.

An ensemble of states contains a large amount of information, beyond that of its density matrix. In this work, we systematically study and quantify ensembles of states by their higher-order moments.

Higher order quantities have received less attention in many-body physics because they have been inaccessible to traditional experiments, requiring a large number of controlled repetitions in order to access higher order statistical properties. Modern quantum devices address these limitations with both high repetition rates and a high degree of controllability [28, 85].

In this work, we find that not only do the average properties (first moments) thermally equilibrate, so too do the higher moments of our natural ensembles of states. We first define the $k$-th moments.

The $k$-th moment of an ensemble $\mathcal{E}$ is, for discrete ensembles

for continuous ensembles is

and for unnormalized ensembles we have (denoting the moments with tildes)

The $k$-th moment of an ensemble can be used to describe the expectation value of any linear operator $O^{(k)}\!:\!\mathcal{H}^{\otimes k}\rightarrow\mathcal{H}^{\otimes k}$ which acts on $k$-copies of the Hilbert space. The ensemble average of $O^{(k)}$ can be obtained from the $k$-th moment: $\mathbb{E}_{\mathcal{\ket*{\Psi}\sim E}}[\bra{\Psi}^{\otimes k}O^{(k)}\ket{\Psi}^{\otimes k}]=\text{tr}[O^{(k)}\rho^{(k)}_{\mathcal{E}}]$. For example, $O^{(k)}=O^{\otimes k}$ describes the $k$-th moment of the expectation values $\mathbb{E}_{\mathcal{E}}[\bra{\Psi}O\ket{\Psi}^{k}]$. Here, we use $\mathbb{E}_{\mathcal{E}}[f(\Psi)]$ to denote the ensemble average $\int d\Psi P(\Psi)f(\Psi)$.

The $k$-th moment can be thought of as a generalization of the density matrix. Namely, the first moment $\rho^{(1)}=\sum_{j}p_{j}\outerproduct{\Psi_{j}}{\Psi_{j}}$ is the conventional density matrix, and describes the average state of the ensemble. All higher moments $\rho^{(k)}_{\mathcal{E}}$ are likewise positive semi-definite operators and, for ensembles of normalized states, have unit trace.

Moments of Haar ensemble — The higher moments of the Gaussian and Haar ensembles are particularly simple. The moments of the Haar ensemble have analytical expression [48]:

with similar expression for the moments of the Gaussian ensemble. Here, $\text{Perm}(\sigma)$ is the operator that permutes $k$ states by the permutation $\sigma\in S_{k}$, i.e. $\text{Perm}(\sigma)[\ket*{\Psi_{1}}\otimes\cdots\otimes\ket*{\Psi_{k}}]=\ket*{\Psi_{\sigma(1)}}\otimes\cdots\otimes\ket*{\Psi_{\sigma(k)}}$. This result follows from the Schur-Weyl duality [86]. Its simplicity enables the design of useful quantum algorithms such as classical shadow tomography [87, 88], as well as to analytical solutions of models of quantum dynamics such as random quantum circuits [64]. We will find that our ensembles of states have $k$-th moments with similar structure as the Haar $k$-th moments, enabling phenomena such as the emergence of the Porter-Thomas distribution.

Trace distance of higher moments — Higher moments also enable the systematic and quantitative comparison of two ensembles of states. Specifically, we compare their $k$-th moments through their trace distance.

where the trace (or nuclear) norm $\|\cdot\|_{*}$ of a matrix is the sum of the absolute values of its eigenvalues. In particular, the trace distance sets an upper bound on how well any $k$-copy observable $O^{(k)}$ can distinguish two ensembles of states [63].

Refs. [28, 29] found that with increasing system size, projected ensembles of infinite effective temperature states — either obtained from quench dynamics of simple initial states or chosen from high energy eigenstates — have closer $k$-th moments to the Haar moment (in other words, projected ensembles form better approximate $k$-designs). Analogously, we find that the $k$-th moments of the projected and (generalized) Scrooge ensembles converge in trace distance with increasing system size [Fig. 4(b) and Fig. 5(b)].

The central claim of our work is that natural many-body dynamics generates ensembles of states with maximum entropy. The entropy in question, the ensemble entropy, differs from the conventional von Neumann entropy $S(\rho)\equiv-\tr[\rho\log(\rho)]$. This is not suitable as an entropy of an ensemble of states, since it only depends on the first moment $\rho$, and there are many ensembles of states which share the same average state $\rho$. Instead, our ensemble entropy is the Shannon entropy of the distribution of states over Hilbert space. For continuous distributions, this entropy must be defined relative to a reference distribution.

The ensemble entropy is non-positive, achieving zero only when the two distributions are equal. See Refs. [53, 54] for similar attempts to quantify the entropy of state ensembles.

Connection to von Neumann entropy — While the ensemble entropy may seem an unconventional notion of entropy, in fact it can be related to the von Neumann entropy of higher moments of the ensemble. For finite ensembles $\mathcal{E}=\{p_{j},|\Psi_{j}\rangle\}$, the quantity

converges to the Shannon entropy $-\sum_{j}p_{j}\log_{2}p_{j}$, which in turn converges to the ensemble entropy (plus a divergent constant), when the discrete ensemble is appropriately selected (see below). This follows from the fact that with increasing number of copies $k$, two states $\ket{\psi}^{\otimes k},\ket{\phi}^{\otimes k}$ become increasingly distinguishable (orthogonal), since $|\innerproduct{\phi}{\psi}|^{2k}$ quickly vanishes when $|\innerproduct{\phi}{\psi}|\neq 1$. Therefore, for any discrete ensemble $\{p_{j},\ket{\Psi_{j}}\}$, the quantities $p_{j}$ and $\ket{\Psi_{j}}^{\otimes k}$ converge to the eigenvalues and eigenvectors of $\rho^{(k)}$ respectively. A similar argument can be made for continuous distributions, and we obtain $\text{Ent}(\mathcal{E})=\text{lim}_{k\rightarrow\infty}S(\rho^{(k)}_{\mathcal{E}})+C$, with a divergent constant $C$.

Ensemble entropy and information compression — The ensemble entropy has a simple operational meaning: it quantifies the minimum number of bits required to store the states in the ensemble in classical memory, up to a fixed precision $\epsilon$. If storing a single such state requires $N_{\epsilon}$ classical bits, we find that storing a large number of states $M$ drawn from the ensemble $\mathcal{E}$ requires $M[N_{\epsilon}+\text{Ent}(\mathcal{E})]$ classical bits (note that $\text{Ent}(\mathcal{E})\leq 0$), as illustrated in Fig. 2(a). Our maximum entropy principle indicates that the required storage for many-body states generated by time evolution is maximal.

To see this, note that storing states up to accuracy $\epsilon$ amounts to partitioning the Hilbert space in a uniform $\epsilon$-net, i.e. into a discrete set of points which are a distance $\epsilon$ apart [3]. To store the states in the ensemble, each state in the continuous Hilbert space is assigned to one of these discrete points. Storing these states thus amounts to storing these labels — they may either be stored directly or they may be compressed according to how often the labels appear. In the former case, each label requires $N_{\epsilon}$ bits, if there are $2^{N_{\epsilon}}$ points in the net. In the latter, the optimal rate of compression is given by the relative entropy between the discrete probabilities on this $\epsilon$-net and the uniform distribution [3]. In the limit of small $\epsilon$, the discrete and continuous distributions converge, and therefore the overall amount of compression achieved is equal to the ensemble entropy $\text{Ent}(\mathcal{E})$ per state, and the overall number of bits required to store $M$ states is $M[N_{\epsilon}+\text{Ent}(\mathcal{E})]$.

Finally, we note that this task is different from the task of storing an ensemble of states in quantum memory, i.e. compressing the states into a smaller number of qubits. This can be done by Schumacher compression [82], with compression rate given by the von Neumann entropy of the first moment $S(\rho_{\mathcal{E}})$. In contrast, the task of classical approximate storage depends on all moments of the ensemble.

In addition to the ensemble entropy, we study another information-theoretic property: the mutual information, which quantifies the information-theoretic correlations between two variables. We find that not only is the ensemble entropy maximized in our ensembles, the mutual information is also minimized. These two quantities are a priori unrelated, and lead to distinct operational consequences.

The mutual information $I(X;Z)$ between random variables $X$ and $Z$ is defined as

This can be interpreted as the information gain (reduction in entropy) of the variable $X$ from measurements of the variable $Z$. In this work, we take the logarithm to base 2, which quantifies the mutual information in units of bits.

The mutual information is relevant in several information theoretic tasks, including quantifying the maximum rate of information transmission through a noisy classical channel [79], where the variables $X$ and $Z$ are here the channel input and output. Below, we focus on the significance of mutual information to a quantum communication task.

Holevo studied the use of a noiseless quantum channel to transmit classical information [56]. Given an ensemble of (possibly mixed) states $\mathcal{E}=\{p_{x},\rho_{x}\}$, Holevo considered a task where one party (Alice) transmits a classical message $\{x_{1},x_{2},\dots\}$ to another (Bob), by sending a sequence of states $\{\rho_{x_{1}},\rho_{x_{2}},\dots\}$. Bob decodes this message using measurements $Z$ on the received states. When the quantum states $\ket*{\psi_{x_{i}}}$ are not orthogonal, the symbols $x_{i}$ cannot be unambiguously determined: the maximum classical transmission rate is therefore given by the accessible information

the mutual information maximized over complete measurement bases $Z$.

Holevo provided an upper bound for the accessible information: the Holevo-$\chi$ quantity $\chi(\mathcal{E})\equiv S(\rho_{\mathcal{E}})-\sum_{x}p_{x}S(\rho_{x})$ [56]. In an ensemble of pure states, the Holevo-$\chi$ quantity is equal to the von Neumann entropy $\chi(\mathcal{E})=S(\rho_{\mathcal{E}})$. For a given first moment $\rho_{\mathcal{E}}$, the Holevo bound is saturated when the ensemble of states is formed using the eigenbasis of $\rho_{\mathcal{E}}$, i.e. $\mathcal{E}=\{\lambda_{j},\ket{j}\}$.

Josza, Robb and Wooters [79] subsequently established a lower bound for the accessible information, dubbed the subentropy $Q(\rho_{\mathcal{E}})$. That is, for any ensemble of pure states $\mathcal{E}$ with a given first moment $\rho_{\mathcal{E}}$, the accessible information is bounded by:

Like the von Neumann entropy, the subentropy only depends on the first moment $\rho_{\mathcal{E}}$. For any ensemble $\mathcal{E}$, the subentropy is defined as the mutual information $I(X;Z)$ averaged over all complete projective measurement bases $Z$. This is equivalent to averaging over unitaries $U$ which rotate measurement bases and has expression [79, 89]

where $\lambda_{j}$ are the eigenvalues of $\rho_{\mathcal{E}}$. More recently, the subentropy has received attention in studies of the dynamics of information in many-body systems [61].

Josza, Robb and Wooters established that the Scrooge ensemble saturates this lower bound (Section V, [79]). Scrooge ensembles are therefore ensembles with minimum accessible information (or, the most information “stingy” ensembles), leading to their name.

(Generalized) Scrooge ensembles describe projected ensembles, therefore our projected ensembles have minimal accessible information. That is, in natural many-body systems, it is maximally hard to predict the measurement outcome of one subsystem from measurements on the other.

Finally, we turn to a measurable signature of our maximum entropy principle, the Porter-Thomas (PT) distribution. Originally studied in nuclear physics [83], the PT distribution has received renewed attention due to its presence in Haar-random many-body states, approximated by the output of deep random quantum circuits [84].

Concretely, a positive random variable $X$ follows the Porter-Thomas distribution with mean $\mu$ [i.e. $X\sim\text{PT}(\mu)$] if its probability distribution function is

In particular, the overlap of states $\ket{\Psi}$ drawn from the Haar random distribution to any fixed state $\ket{\Phi}$ follows a PT-distribution with $\mu=1/D$ [74]

Notably, this is a statement about the probability distribution of probabilities $|\langle\Psi|\Phi\rangle|^{2}$. While relatively unknown in physical settings, this has received attention in the statistical literature under several names, including the fingerprint and histogram of the histogram [90, 91]. The fingerprint is sufficient to describe the properties of a distribution that do not depend on its labels, most notably its Shannon entropy.

The probability-of-probabilities can be studied in modern quantum experiments. For example, the PT distribution expected from Haar-random states has been predicted and subsequently experimentally verified in deep random quantum circuits [84, 74] and many-body states at infinite temperature [29]. Even though the ensembles we study in this work are not the Haar ensemble, we find that the maximum entropy principle leads to the same PT distribution, when observables are appropriately normalized. In companion work, we experimentally and theoretically study this object in detail [41].

In this work, we shall be interested in the moments of the random phase ensemble. Here, we provide a simplified, approximate form of the $k$-th moments, which enables their further analysis.

Theorem 2 explicitly demonstrates how the higher order moments of the temporal ensemble depend on the first moment $\rho_{d}$. Specifically, the moments of the temporal ensemble have a structural resemblance to the moments of the Haar ensemble [Eq. (16)], up to a correction which is typically exponentially small, formalizing our intuition of maximally entropic ensembles as “distorted” Haar ensembles. This asymptotic behavior will also be common to projected ensembles.

Theorem 1 studies the properties of temporal ensembles over infinite time intervals. Here, we investigate how temporal ensembles over finite time intervals converge to their infinite-interval values. We analytically study its convergence rate in random-matrix models and present numerical evidence that the above convergence rate also holds for realistic Hamiltonians. Specifically, we study the convergence of the finite time $k$-th moment $\rho^{(k)}_{\text{Temp.}}(\tau)$ to its infinite-time limit $\rho^{(k)}_{\text{Temp.}}\equiv\rho^{(k)}_{\text{Temp.}}(\tau=\infty)$. This has expression:

We consider the convergence both in terms of the trace norm and Frobenius norm, with the latter being analytically easier to study. For the first moment, i.e. $k=1$, we numerically observe that the distance in Frobenius and trace norms both decay as $\tau^{-1}$ (see App. F). This holds for a generic chaotic model, the mixed field Ising model (the main model we study in this work, Sec. IV.4) as well as for Hamiltonians sampled at random from the Gausian Unitary Ensemble (GUE). We attribute this asymptotic $1/\tau$ scaling to the $1/x$ asymptotic behavior of the $\text{sinc}(x)$ function in Eq. (34). In Appendix F, we support these numerical findings by analytical calculations in random matrix theory, showing that the average of the squared Frobenius norm decays as $\tau^{-2}$. This suggests that the average Frobenius norm decays as $\tau^{-1}$, consistent with our numerical observations.

Interestingly, for higher order moments $k=2,3$, we observe different behavior. For the mixed field Ising model as well as individual samples from the GUE, the Frobenius distance decays at intermediate times as $\tau^{-1/2}$ before a crossover time, after which it decays as $\tau^{-1}$ [Fig. 3(b)]. This crossover happens in times which are exponentially long in system size, which we attribute to the presence of a minimum distance of gaps $E_{i}-E_{j}+E_{k}-E_{l}$ (for $k=2$) for each individual Hamiltonian. While such minimum distance is exponentially small in system size, it has a finite, nonzero value $\Delta_{\min}$ for finite-sized systems. Only at times much longer than the inverse of this minimum gap does the asymptotic $1/(\Delta_{\min}\tau)$ behavior become clear.

Notably, we find that for the first moment, $k=1$, the GUE ensemble mean closely resembles the behavior of individual instances and decays as $\tau^{-1}$. In contrast, for higher moments, $k=2$, the behavior of individual samples and the sample mean differ. With an increasing number of samples, the crossover is less pronounced. We relate these findings to statistical properties of the eigenvalues of GUE Hamiltonians. It is well known that the eigenvalues exhibit level repulsion, i.e. the probability to find two distinct eigenvalues close to each other $E_{i}-E_{j}\approx 0$ is vanishing. In contrast, as suggested by Ref. [98] and the analytical and numerical results presented in Appendix F, we find that the gaps $E_{i}-E_{j}+E_{k}-E_{l}$ of eigenvalues exhibit only weak repulsion, leading to the crossover timescale growing with the size of the sample.

An immediate consequence of Theorem 2 is the emergence of the Porter-Thomas distribution (Sec. III.5) in the probabilities of measuring a given outcome $z$, as a function of time. This result was originally presented in Ref. [94]. For clarity, we denote this distribution as $p_{z}(t)\equiv p(z,t)\equiv\innerproduct{z}{\Psi(t)}\!\innerproduct{\Psi(t)}{z}$ ⁷⁷7The subscript $z$ indicates that the measurement outcome $z$ is being fixed and the distribution $p_{z}(t)$ is taken over time $t$., which can be experimentally estimated by repeated measurements of the state $\ket{\Psi(t)}$.

Unlike local observables, whose expectation values quickly equilibrate in time— $\langle O(t)\rangle$ has exponentially small relative fluctuations about its equilibrium value $\mathbb{E}_{t}[\langle O(t)\rangle]$[17]— global observables such as the projector $|z\rangle\langle z|$ never equilibrate. Theorem 3 indicates that they have $O(1)$ relative fluctuations which follow a universal, exponential distribution. We utilized this result in Ref. [94] to propose a protocol for many-body benchmarking in analog quantum simulators. Here, we will use the PT distribution as a signature of our maximum entropy principle; it will be a common signature among the temporal ensemble, projected ensemble and beyond.