Generalization despite overfitting in quantum machine learning models

We are interested in functions $g$ defined on a finite interval that may be written in terms of a linear combination of Fourier basis functions or modes $e^{i2\pi kx}$ (each describing a complex sinusoid with integer-valued frequency $k$) weighted by their corresponding Fourier coefficients $\hat{g}_{k}$:

We restrict our attention to well-behaved functions that are sufficiently smooth and continuous to be expressed in this form. We will now set up a simple learning problem whose basic components – the model and target function to be learned – can be expressed as Fourier series with only few non-zero coefficients, and define concepts such as overparametrization and interpolation.

Consider a machine learning problem in which data is generated by a target function of the form

which only contains frequencies in the discrete, integer-valued spectrum

for some odd integer $n_{0}$. We call functions of this form band-limited with bandwidth $n_{0}/2$ (that is, $|k|<n_{0}/2$ for all frequencies $k\in\Omega_{n_{0}}$). The bandwidth limits the complexity of a function, and will therefore be important for exploring scenarios where a complex model is used to overfit a less complex target function. We are provided with $n$ noisy samples $y_{j}=g(x_{j})+\epsilon$ of the target function $g$ evaluated at points $x_{j}=j/n$ spaced uniformly on interval $[0,1]$ (we will assume input data has been rescaled to $[0,1]$ without loss of generality), where $n>n_{0}$ and we require $n$ to be odd for simplicity.

The model class we consider likewise contains band-limited Fourier series, and since we are interested in interpolating models, we always assume that they have enough modes to interpolate the noisy training data, namely:

Similarly to Eq. (3) we define the spectrum

Following Ref. [9], this model class has two components: The set of weighted Fourier basis functions $\sqrt{\nu_{k}}e^{i2\pi kx}$ describe a feature map applied to $x$ for some set of feature weights $\nu_{k}\in\mathbb{R}^{+}$, while the trainable weights $\alpha_{k}\in\mathbb{C}$ are optimized to ensure that $f$ interpolates the data. The theory of trigonometric polynomial interpolation [32] ensures that $f$ can always interpolate the training data for some choice of trainable weights $\alpha_{k}$ under these conditions. In the following, we will therefore usually consider the $\alpha_{k}$ as being determined by the data and interpolation condition, while the $\nu_{k}$ serve as our “turning knobs” to create different settings in which to study spiking properties and benign overfitting. We call the model class described in Eq. (4) overparameterized when the degree of the Fourier series is much larger than the degree of the target function, $d\gg n_{0}$, in which case the model has many more frequencies available to perform fitting than there are present in the underlying signal to fit.

Note that one can rewrite the Fourier series of Eq. (4) in a linear form

where

From this perspective, optimizing $f$ amounts to learning trainable weights $\boldsymbol{\alpha}$ by performing regression on observations $\phi(x)$ sampled from a random Fourier features model [33] for which $\nu_{k}$ and $\phi(x)_{k}$ are precisely the eigenvalues and eigenfunctions of a shift-invariant kernel [34].

To complete the problem setup, we have to impose one more constraint. Consider that $\exp(i2\pi kx_{j})$ with frequency $k<n/2$ and uniformly spaced points $x_{j}$ is equal to $\exp(i2\pi k^{\prime}x_{j})$ for any choice of alias frequency $k^{\prime}=k\mod n$. The presence of these aliases means that the model class described in Eq. (6) contains many interpolating solutions in the overparameterized regime. Motivated by prior work exploring benign overfitting for linear features [6], Fourier features [9, 35], and other nonlinear features [36, 37], we will study the minimum-$\ell_{2}$ norm interpolating solution,

Minimizing the $\ell_{2}$ norm is a typical choice for imposing a penalty on complex functions (regularization) in the underparameterized regime, though we will see that this intuition does not carry over to the overparameterized regime. The remainder of this section will explore how this learning problem results in a trade-off in interpolating Fourier models: Overparameterization introduces alias frequencies that increase the error in fitting simple target functions but can also reduce error by absorbing noise into high-frequency modes with spiking behavior.

To better understand the trade-off that overparametrization – or in our case, a much larger number of Fourier modes than needed to interpolate the data – introduces between fitting noise and generalization error, we revisit two extreme cases explored in Ref. [9], involving a pure-noise signal and a noiseless signal.

The opposing effects of higher-frequency modes in overparameterized models in the cases discussed above hint at a trade-off in model performance that depends on the underlying signal and the feature weights of the Fourier feature map. Returning to the more general case of input samples $y_{j}=g(x_{j})+\epsilon_{j}$, in Appendix A we show that the task of fitting uniformly spaced samples using weighted Fourier features may be transformed into a linear regression problem, thereby generalizing the results of [9] to derive the following general solution to the minimum-$\ell_{2}$ interpolating problem of Eq. (9):

where $\hat{y}_{k}$ is the discrete Fourier transform of $y_{k}$ and $k\in\Omega_{n_{0}}$, where

denotes the set of alias frequencies of $k$ appearing in the overparameterized model with spectrum $\Omega_{d}$. The optimal model is then expressed as

Recalling that our model $f$ is trained on $n$ noisy samples $(y_{0},\dots,y_{n-1})$ of the target function $g$, we are interested in the squared error of the model $f$ averaged over (noisy) samples over the input domain,

and we call $L$ the generalization error of $f$, as it captures the behavior of $f$ with respect to $g$ over the entire input domain $x\in[0,1]$ instead of just the uniformly spaced training points $x_{j}$. In Appendix A we derive

We use this generalization error now to explore two interesting behaviors of the interpolating model in our setting: The tradeoff between noise absorption and signal recovery exemplified by the cases in Sec. 2.2, and the ability of an overparameterized Fourier features model to benignly overfit the training data.

The first behavior involves a trade-off in the generalization error $L(f^{opt})$ between absorbing noise (reducing var) and capturing the target function signal (reducing $\textsc{bias}^{2}$) that recovers and generalizes the behavior of the two cases in Sec. 2.2. This trade-off is controlled by three components: The noise variance $\sigma^{2}$, the input signal Fourier coefficients $\hat{g}_{k}$, and the distribution of feature weights $\sqrt{\nu_{k}}$. As described in the two cases above, when $\sigma^{2}\rightarrow 0$ (signal only) the variance term var vanishes and the model is biased for any choice of nonzero $\nu_{k}$ where $k>n$. Conversely, when $\hat{g}\rightarrow 0$ (noise only) the bias term $\textsc{bias}^{2}$ vanishes, and the variance term is minimized by choosing uniform $\nu_{k}$ for all $k\in\Omega_{d}$.

The second interesting behavior occurs when the generalization error of the overparameterized model decreases at a nearly optimal rate as the number of samples $n$ increases, known as benign overfitting. Prior work on benign overfitting in linear regression studied scenarios where the distribution of input data varied with the dimensionality of data and size of the training set in such a way that the excess generalization error of the overparameterized model (compared to a simple model) vanished [6]. However, since the dimensionality of the input data for our model is fixed, we instead consider sequences of feature weights that vary with respect to the problem parameters ($n_{0}$, $n$, $d$) in a way that results in $\textsc{bias}^{2}$ and var vanishing as $n\rightarrow\infty$. In this case, by fitting an increasing number of samples $n$ using such a sequence of feature weights, the overparameterized model both perfectly fits the training data and generalizes well for unseen test points, and therefore exhibits benign overfitting.

These behaviors are exemplified by a straightforward choice of feature weights that incorporate some prior knowledge of the spectrum $\Omega_{n_{0}}$ available to the target function $g$. For all $k\in\Omega_{n_{0}}$, let $\nu_{k}=c/n_{0}$ for some positive $c$ and normalize the feature weights so that $\sum_{k\in\Omega_{k}}\nu_{k}=1$. We show in Appendix A.3.1 that the error terms of $L(f^{opt})$ scale as

Thus, as long as the dimension of the overparameterized Fourier features model grows strictly faster than $n$ (i.e., $d=\omega(n)$), the model exhibits benign overfitting. In Appendix A.3.2 we demonstrate how this simple example actually captures the mechanisms of benign overfitting for much more general choices of feature weights. Fig. 3 summarizes this behavior and provides an example of the bias-variance tradeoff that occurs for overparameterized models. In particular, Fig. 3a exemplifies the setting in which benign overfitting occurs, wherein the feature weights of the Fourier features model are strongly concentrated over frequencies in $\Omega_{n_{0}}$ but extend over a large range of alias frequencies for each $k\in\Omega_{n_{0}}$.

The generalization behavior described here is fundamentally different from many generalization guarantees typically found in statistical learning theory. While prior work has derived guarantees for the generalization of quantum models by constructing bounds on the complexity of the model class [25], Eqs. 20-21 demonstrate that generalization may occur as the complexity (i.e. dimension) of a model grows arbitrarily large.

So far, we have reviewed the Fourier perspective on fitting periodic functions in a classical setting and extended the analysis to characterize benign overfitting. However, if we can link the basic components of quantum models to the terms appearing in the error of Eq. (19), then we will be able to study a similar trade-off in the error of overparameterized quantum models and the conditions necessary for benign overfitting. The remainder of this work is devoted to showing that analogous mechanisms exist in certain quantum machine learning models, and to studying the choices of feature weights for which quantum models can exhibit tradeoffs in generalization error and benign overfitting.

To explicitly highlight the role of $R(k)$ in controlling the feature weights $\nu_{k}^{opt}$ of the optimized quantum model, we will now simplify the model by using an equal superposition input state $|\Gamma\rangle=\frac{1}{\sqrt{d}}\sum_{j=0}^{d-1}|j\rangle$ and by restricting the set of observables considered during optimization. If we fix every entry of the observable with respect to elements in a partition $R(k)$ to be proportional to some complex constant $M(k)$:

then we can simplify the quantum model of Eq. (25) to

Comparing Eq. (33) to Eq. (4) we identify a direct correspondence between the trainable weights $\alpha_{k}$ in the classical model with $M(k)$, as well as a correspondence between the feature weights $\sqrt{\nu_{k}}$ and the the degeneracy $|R(k)|$ of the quantum model. Making the substitutions $\alpha_{k}\rightarrow M(k)$, $\sqrt{\nu_{k}}\rightarrow\sqrt{|R(k)|}/d$, one can verify that $\sum_{k\in\Omega}|M(k)|^{2}=\lVert M\rVert_{F}^{2}$ for this restricted choice of $M$ and so the solution to the optimization of Eq. (28) is essentially the same as that of the classical problem in Eq. (15).

The crucial property of the simplified model is that the degeneracy $|R(k)|$ – and hence the combinatorial structure introduced by the data encoding Hamiltonian’s eigenvalues – completely controls the trade-off in the generalization error (Eq. (19)). We can hence study different types of partitions $R(k)$ to show a direct effect of the data-encoding unitary $S(x)$ on the fitting and generalization error behaviors for this simplified, overparameterized quantum model.

To study these behaviors we will now consider specific families of $H$ which we call encoding strategies since the choice of $H$ completely determines how the data is encoded into the quantum model. While $R(k)$ and $\Omega$ may be computed for an arbitrary $S(x)$ using brute-force combinatorics, some encoding strategies lead to particularly simple solutions. We have derived a few such examples of simple degeneracies and spectra for different encoding strategies in Appendix C and present the results in Table 1. These choices highlight the extreme variation in $\Omega$ resulting from minor changes to $S(x)$, for example $|\Omega|\propto n_{q}$ for the “Hamming” encoding strategy compared to $|\Omega|\propto 3^{n_{q}}$ for the “Ternary” encoding strategy. These examples also highlight the limitations in constructing Hamiltonians with specific properties such as uniform $|R(k)|$ or evenly-spaced frequencies in $\Omega$.

Since the feature weights $\nu_{k}$ of the Fourier modes are fixed by the choice of the data-encoding unitary, we can understand a choice of $S(x)$ as providing a structural bias of a quantum model towards different overfitting behaviors, and conversely the choices of feature weights available to quantum models are limited and are particular to the structure of the associated quantum circuit. Figure 4 shows distributions for feature weights arising from the example encoding strategies presented in Table 2, and demonstrates a broad connection between the degeneracies $|R(k)|$ of the model (giving rise to feature weights $\nu_{k}^{opt}$) and the generalization error $L(f^{opt})$.

We now return to the general quantum model of Eq. (25) to understand how different choices of the state preparation unitary $U$ (giving rise to input state $|\Gamma\rangle$) affect the feature weights $\nu_{k}^{opt}$ of the general quantum model, thereby influencing benign overfitting of the target function. While Eq. (31) differs from the correspondence $\nu_{k}=|R(k)|/d^{2}$ that we observed for the simplified quantum model, we see that the feature weights of the optimal quantum model still depend heavily on the degeneracies $|R(k)|$. For instance, the average $\nu_{k}^{opt}$ with respect to Haar-random $d$-dimensional input states $|\Gamma\rangle$ is proportional to $|R(k)|$ whenever $k\neq 0$:

where $\delta_{i}^{j}$ denotes the Kronecker delta. Furthermore, in Appendix B we observe that the variance of $\nu_{k}^{opt}$ around its average tends to be small for encoding strategies considered in this work, for instance scaling like $\mathcal{O}\left(d^{-3}\right)$ for the Binary encoding strategy and $\mathcal{O}\left(d^{-4}\right)$ for the Golomb encoding strategy. This demonstrates that the feature weights of generic quantum models (i.e., ones for which $U$ is randomly sampled) will be dominated by the degeneracy $|R(k)|$ introduced by the data-encoding operation $S(x)$.

Despite the behavior of $\nu_{k}$ being dominated by $|R(k)|$ in an average sense, there are specific choices of $U$ for which the feature weights deviate significantly from this average. We will now use one such choice of $U$ to provide a concrete example of an interpolating quantum model that exhibits benign overfitting. Suppose we choose $U$ such that the elements of $|\Gamma\rangle$ are given by

for $j\in[d]$, and some integers $0<c_{1}<c_{2}<d$ and amplitudes $a,b$ subject to normalization. We show that given a band-limited target function $g$ with access to spectrum $\Omega_{n_{0}}$, there is a specific choice of $c_{1},c_{2}$ dependent on $d$ and $n_{0}$ for which the interpolating quantum model $f^{opt}$ of Eq. (30) also benefits from vanishing generalization error in the limit of many samples, namely we show in Appendix D that

Thus, by perfectly fitting the training data and exhibiting good generalization in the $n\rightarrow\infty$ limit, the quantum model exhibits benign overfitting. This behavior is outlined in Figure 5, which highlights the role that $|\Gamma\rangle$ plays in concentrating the feature weights $\nu_{k}$ within the spectrum $\Omega_{n_{0}}$ of $g$ while preserving a long tail that provides the model with low-variance “spiky” behavior in the vicinity of noisy samples. In contrast, the feature weights for the Binary encoding strategy with a uniform input state has little support on $\Omega_{n_{0}}$ and resulting in a large bias error.

The above discussion shows how the input state amplitudes $\gamma_{i}$ provide additional degrees of freedom with which the feature weights $\nu_{k}^{opt}$ can be tuned in order to modify the generalization behavior of the interpolating quantum model, and to exhibit benign overfitting in particular. It is therefore worthwhile to consider what other kinds of feature weights $\nu_{k}$ might be prepared by some choice of input state $|\Gamma\rangle$. We may use simple counting arguments to demonstrate the restrictions in designing particular distributions of feature weights. Suppose we define $\Omega_{+}=\{k:k\in\Omega,k>0\}$ containing the positive frequencies of a quantum model. Then the introduction of an arbitrary input state $|\Gamma\rangle$ provides us with $2^{n_{q}}-1$ free parameters with which to tune $|\Omega_{+}|$-many terms in the distribution of $\nu_{k}^{opt}$ (subject to $\nu_{k}=\nu_{-k}$ and $\sum_{k}\nu_{k}=1$). Clearly, there are distributions of feature weights $\nu_{k}^{opt}$ that can not be achieved for models where $|\Omega_{+}|\geq 2^{n_{q}}$.

Conversely, the condition $|\Omega_{+}|<2^{n_{q}}$ does not necessarily mean that we can thoroughly explore the space of possible feature weights by modifying the input state $|\Gamma\rangle$. For example, consider the Hamming encoding strategy for which the number of free parameters controlling the distribution of feature weights $\nu_{k}^{opt}$ is $|\Omega_{+}|=n_{q}$, which is exponentially smaller the number of parameters in $|\Gamma\rangle$. While this might suggest significant freedom in adjusting $\nu_{k}^{opt}$, the opposite is true: For any choice of input state $|\Gamma\rangle$, there is another state of the form

that achieves exactly the same distribution of feature weights $\nu_{k}$. In Eq. (37), $|\Phi_{i}\rangle$ describes a uniform superposition over all computational basis state bitstrings with weight $i$, and so the distribution of $\nu_{k}^{opt}$ actually only depends on $n_{q}+1$ real parameters $\phi_{i}$, $i=0,1,\dots,n_{q}$, and the feature weights are invariant under any operations in $U$ that preserve $|\Phi_{i}\rangle$ (see Appendix B). An example of such operations are the particle-conserving unitaries well-known in quantum chemistry, which act to preserve the number of set bits (i.e., the Hamming weight) when each bit represents a mode occupied by a particle in second quantization [46, 47]. This example demonstrates how symmetry in the data-encoding Hamiltonian (e.g. Refs. [48, 49]) can have a profound influence on the ability to prepare specific distributions of feature weights $\nu_{k}^{opt}$, and consequently affect the generalization and overparameterization behavior of the associated quantum models.