REFINED DISTRIBUTIONAL LIMIT THEOREMS
FOR COMPOUND SUMS

The classification of compound sums, where shorthand for “compound sum with regular summation and independence” is “$\mathcal{R\hskip-0.5pt/I}$  sum”, for “compound sum with irregular summation and independence” is “$\mathcal{I\hskip-0.5pt/I}$  sum”, for “compound sum with regular summation and dependence” is “$\mathcal{R\hskip-0.5pt/D}$  sum”, and for “compound sum with irregular summation and dependence” is “$\mathcal{I\hskip-0.5pt/D}$  sum”, is shown in Fig. 1. Under natural regularity conditions, compound sums “with regular summation” are always proper, while compound sums “with irregular summation” are either proper, or defective⁷⁷7Recall (see [37]) that a random variable is defective, with defect $1-p$, if it takes the value $\infty$ with probability $p>0$. Otherwise, it is proper..

The following example shows that finding the defect of $\mathcal{I\hskip-0.5pt/I}$  sum is not easy.

Although the expression (2.3) for defect of ${S}_{N_{t}}$ is complicated⁸⁸8This is a particular case of Cramér’s famous result on the probability of ultimate ruin., the conditions under which ${S}_{N_{t}}$ is proper or defective, are simple. On the one hand, if $\mathsf{E}T_{1}>0$, then (see Theorem 2 in [37], Chapter XII) the ordinary sums ${V}_{n}$ increase a.s.  to $+\infty$, as $n\to\infty$. Consequently, for any $t$ the summation limit $N_{t}$ is a.s.  finite and ${S}_{N_{t}}$ is a.s.  less than $+\infty$, whence proper. On the other hand, if $\mathsf{E}T_{1}<0$, then the ordinary sums ${V}_{n}$ decrease a.s.  to $-\infty$, as $n\to\infty$. With a positive probability, the maximum $M=\max_{n\in\mathsf{N}}{V}_{n}$ is finite and less than $t>0$ large enough, whence $N_{t}$ and ${S}_{N_{t}}$ are defective.

In renewal theory, $p$–components $X_{i}$, $i=1,2,\dots$, (typically positive) are called “rewards in the moments of renewals” and $s$–components $T_{i}$, $i=1,2,\dots$, (always positive) are called “intervals between successive renewals”. This interpretation yields a variety of species of $\mathcal{R\hskip-0.5pt/I}$  sums. In particular, the total number of renewals before time $t$, defined as $\widetilde{N}_{t}=\sup\{n\geqslant 1:{V}_{n}\leqslant t\}$, or $0$, if $T_{1}>t$, comes to the fore. Focus shifts from $N_{t}$ to $\widetilde{N}_{t}$, which coincides with $\widehat{N}_{t}=N_{t}-1$ trivially related to $N_{t}$.

When moving from relatively simple $\mathcal{R\hskip-0.5pt/I}$  sums to more complicated $\mathcal{I\hskip-0.5pt/I}$ and $\mathcal{I\hskip-0.5pt/D}$  sums, both of them with “irregular summation”, the difference between altered (with summation limit $\widetilde{N}_{t}$) and non–altered (with summation limit $N_{t}$) compound sums becomes non–trivial: $\widetilde{N}_{t}$ is no more trivially related to $N_{t}$ and the altered compound sums ${S}_{\widetilde{N}_{t}}=\sum_{i=1}^{\widetilde{N}_{t}}X_{i}$ and ${S}_{\widehat{N}_{t}}=\sum_{i=1}^{\widehat{N}_{t}}X_{i}$ are not bounded to coincide.

As for $\mathcal{R\hskip-0.5pt/D}$  sums with Markov dependence, which generalize $\mathcal{R\hskip-0.5pt/I}$  sums in the same way as Markov renewal theory generalizes renewal theory, dependence is introduced by embedded Markov chain $\{\xi_{i}\}_{i\geqslant 0}$. In such $\mathcal{R\hskip-0.5pt/D}$  sums, $s$–components $T_{i}$, $i=1,2,\dots$, are the time intervals between jumps of $\{\xi_{i}\}_{i\geqslant 0}$ and $p$–components $X_{i}$, $i=1,2,\dots$, which depend on $\{\xi_{i}\}_{i\geqslant 0}$, are rewards in the moments of renewals.

In this way, the variety of species of compound sums echoes the diversity of renewal processes. There are⁹⁹9We refer to [21], Section 2.2, or [22], Section 9.2. There are terminological collisions, e.g., in [45], Chapter 5, Section 7, this process is called delayed renewal process. modified, equilibrium, and ordinary renewal processes. If $T_{1}$ is distributed differently than $T_{i}\overset{d}{=}T$, $i=2,3,\dots$, then the renewal process is called modified. When all intervals between renewals, including the first one, are identically distributed, the renewal process is called ordinary. Furthermore, if

where $F_{T}$ denotes c.d.f.  of $T$, then it is called equilibrium, or stationary.

Regarding compound sums, modified are those in which a finite number of elements of the basis $(T_{i},X_{i})$, $i=1,2,\dots$, e.g., the first element $(T_{1},X_{1})$, differ from all the others. Unlike in renewal theory which models technical repairable systems where the first repair does not necessarily occur at the starting time zero, this modification looks awkward for $\mathcal{R\hskip-0.5pt/I}$  sums: it boils down to a very special case of non–identically distributed summands. The similar modification of $\mathcal{R\hskip-0.5pt/D}$  sums with Markov dependence comes down to the choice of the initial distribution of the embedded Markov chain $\{\xi_{i}\}_{i\geqslant 0}$ and is much more sensible.

It is noteworthy that, under natural regularity conditions, switching within the species of compound sums, e.g., from altered to unaltered compound sums and vice versa, does not affect the main–term approximations, but affects refinements such as Edgeworth’s expansions.

In words, the renewal function¹⁰¹⁰10The asterisk denotes the convolution operator. $\mathsf{U}(t)=\sum_{n=0}^{\infty}F_{T}^{*n}(t)$, $t>0$, where $F_{T}$ denotes c.d.f.  of a positive random variable $T$, is the expected number of renewals in the time interval $(0,t)$, with the origin counted as renewal epoch¹¹¹¹11In [22], [45], the renewal function is defined as $\mathsf{H}(t)=\mathsf{U}(t)-1$, $t>0$ (or $\mathsf{E}\widehat{N}_{t}$), which differs from $\mathsf{U}(t)$ by one. In [37], the function $\mathsf{U}(t)$, $t>0$, is called the (ordinary) renewal process. In [22], [45], the renewal process is referred to as the continuous–time random process $\{\widehat{N}_{t}\}_{t\geqslant 0}$, i.e., the random number of renewals in the time interval $(0,t)$, with the origin not being counted as a renewal epoch.. As a formula, this is $\mathsf{U}(t)=1+\mathsf{E}\widehat{N}_{t}$. Together with altered $\mathcal{R\hskip-0.5pt/I}$  sum ${S}_{\widehat{N}_{t}}$, called in renewal theory “cumulated rewards”, it is built on $\mathcal{R\hskip-0.5pt/I}$  basis $(T_{i},X_{i})\overset{d}{=}(T,X)$, $i=1,2,\dots$, with i.i.d.  components and $T$ positive.

The elementary renewal theorem (see, e.g., [41]), states that for $0<\mathsf{E}\,T\leqslant\infty$ the approximation¹²¹²12When $\mathsf{E}\,T=\infty$, the ratio $t/\mathsf{E}\,T$ is replaced by $0$.

holds. The corresponding expansion up to vanishing term is

called refined elementary renewal theorem.

The elementary renewal theorem is widely known: see, e.g., Equality (3) in [21], Section 4.2, or Equality (17) in [22], Section 9.2, or main formula in point (b) in [45], Chapter 5, Section 6. In [22], p. 345, it is noted that “a rigorous proof $\dots$ is possible under very weak assumptions about the distribution of $T$, but requires difficult Tauberian arguments and will not be attempted here”. It can also be obtained (with some caution with regard to the definition of renewal function) from Theorem 1 in [37], Chapter XI, the proof of which is based on the renewal equation.

The refined elementary renewal theorem and the similar results for higher–order power moments of $\widehat{N}_{t}$ and ${S}_{\widehat{N}_{t}}$, including expansions up to vanishing terms, were studied (see, e.g., [21], Section 4.5, and [3]) by various methods.

We outline a proof (see details in [65], Chapter 4) that does not compete with the proofs mentioned above in the sense of elegance or minimality of conditions. The advantage of this proof, based on the use of refined CLT and consisting of steps  A–F, is that it will be routinely carried over to much more complex settings.

Step  A:  use of fundamental identity.  The identity

is obvious. If the probability density function (p.d.f.) $f_{T}$ exists (we assume this for simplicity of presentation), then (3.3) can be rewritten as

Step  B:  reduction of range of summation.  At this step, we cut out from the sum on the right side of (3.3) those terms that correspond to small $n$ (i.e., $n<n_{t}$ with properly selected $n_{t}\to\infty$, as $t\to\infty$). This is intuitively clear: for large $t$, the random variable $\widehat{N}_{t}$ is likely to be large. Therefore, the probabilities $\mathsf{P}\big{\{}\widehat{N}_{t}=n\big{\}}$ for small $n$ are small. Technically, it is done using the well–known probability inequalities, such as Markov’s inequality.

Step  C:  reduction of range of integration.  At this step, relying on the moment conditions, we cut out from the integral in (3.4) that part that corresponds to large $z$ (i.e., $z>z_{t}$ with properly selected $z_{t}\to\infty$, as $t\to\infty$). This is also intuitively clear: any single random variable, including $T_{n+1}$, is small compared to the sum $\sum_{i=1}^{n}T_{i}$, as $n$ is sufficiently large.

Step  D:  application of refined CLT.  Writing $\mu_{T}=\mathsf{E}\,T$, $\sigma_{T}^{2}=\mathsf{D}\,T$ and switching to standardized random variables $\tilde{T}_{i}=(T_{i}-\mu_{T})/\sigma_{T}$, $i=1,2,\dots$, we have

where $\lambda_{T,n}{(t-z)}=(t-z-\mu_{T}n)/(\sigma_{T}\sqrt{n})$. Assuming that natural moment conditions are satisfied, we apply Berry–Esseen’s estimate (see Theorem 11 in [80], Chapter VII, § 2) in the proof of (3.1) and Edgeworth’s expansion (see Theorem 17 in [80], Chapter VII, § 3) in the proof of (3.2). In both cases, these results are taken with non–uniform remainder terms.

Step  E:  analysis of approximating term.  In the proof of (3.1), where Berry–Esseen’s estimate was used, the approximating term is

where $\varphi_{\left({0},{1}\right)}$ denotes p.d.f.  of standard normal distribution. In the proof of (3.2), where Edgeworth’s expansion was used, the approximating term is the sum of

Using Riemann summation formula with nodal points $\lambda_{T,n}{(t)}{}<\lambda_{T,n-1}{(t)}{}<\dots<\lambda_{T,2}{(t)}{}<\lambda_{T,1}{(t)}{}$, we seek to reduce $\mathcal{I}_{t}$ to ${t}/{\mathsf{E}\,T}$ with the required accuracy, and $\mathcal{I}_{t}^{\,{\dagger}}+\mathcal{I}_{t}^{\,{\ddagger}}$ to $t/\mathsf{E}\,T+(\mathsf{D}T-(\mathsf{E}\,T)^{2})/(2\,(\mathsf{E}\,T)^{2})$ with the required accuracy. This is a tedious but fairly standard classical analysis left to the reader.

Step  F:  analysis of remainder term.  In the proof of (3.1), the remainder term is

In the proof of (3.2), the remainder term is

with $\delta_{n}\to 0$, as $n\to\infty$. The former is $o(t)$, $t\to\infty$. The latter is $o(1)$, $t\to\infty$, which is shown by direct analytical methods. This is also a tedious but fairly standard classical analysis left to the reader.

Moving from renewal function (or power moments of $\widehat{N}_{t}$) to the distribution of altered $\mathcal{R\hskip-0.5pt/I}$  sum ${S}_{\widehat{N}_{t}}=\sum_{i=1}^{\widehat{N}_{t}}X_{i}$, called in renewal theory “cumulated rewards”, we use the shorthand notation $\mu_{X}=\mathsf{E}X$, $\sigma_{X}^{2}=\mathsf{D}X$, $\varkappa_{XT}^{\,(i,j)}=\mathsf{E}((X-\mu_{X})^{i}(T-\mu_{T})^{j})$ for $i$ and $j$ integers, and $\varkappa_{XT}=\varkappa_{XT}^{\,(1,1)}$. We put

By $\varPhi_{\left({0},{1}\right)}(x)$, we denote c.d.f.  of standard normal distribution.

Both Theorems 3.1 and 3.2 are proved (see details in [65], Chapter 4) by means of practically the same advanced technique which was used in the proof of (3.1) and (3.2), respectively. It consists of steps A–F, which we will draw.

Step  A:  use of fundamental identity.  Equalities (3.3) and (3.4) are replaced by

which is the total probability formula.

Steps  B and C  remain essentially the same.

Step  D:  application of refined CLT.  The integrand in (3.5) is ready for application of (see [31]–[33]) two–dimensional hybrid integro–local (for density) CLT. In more detail, in the proof of Theorem 3.1, we use Berry–Esseen estimate, in the proof of Theorem 3.2, we use Edgeworth’s expansion in this CLT, both with non–uniform remainder terms.

Steps  E and F  remain nearly the same. Technically, they are more complicated than steps E and F in the proof of (3.1) and (3.2), but use essentially the same analytical techniques.

When studying altered $\mathcal{R\hskip-0.5pt/I}$  sum ${S}_{\widehat{N}_{t}}$, the main point of basic technique consists in approximating ${S}_{\widehat{N}_{t}}$ by the ordinary sum ${S}_{\,[\mathsf{E}\!\widehat{N}_{t}]}$, whose asymptotical normality is evident: the real number $[\mathsf{E}\!\widehat{N}_{t}]$ is (see (3.1)) of the order $t/\mathsf{E}\,T\to\infty$, as $t\to\infty$, and the summands are i.i.d.  In other words, although technically the basic technique applies Kolmogorov’s inequality for maximum of partial sums, its fundamental idea is to throw the difference $\mathcal{G}_{\,t}={S}_{\widehat{N}_{t}}-{S}_{\,[\mathsf{E}\!\widehat{N}_{t}]}$, called garbage, in the trash.

Constrained by this idea, the basic technique does not allow further refinements, such as in Theorems 3.1 and 3.2, since $\mathcal{G}_{\,t}$ is a random variable of order $t^{1/4}$: under natural regularity conditions, including $\mathsf{E}(T^{3})<\infty$, $\mathsf{E}(X^{3})<\infty$,

The proof of (3.6) (see details in [57] and [58]) by means of advanced technique follows the scheme sketched in the proof of (3.1) and (3.2) and Theorems 3.1 and 3.2. It consists of steps A–F, applies refined CLT with non–uniform remainder terms, and requires mere standard classical analysis at steps B, C, E, and F.

For greater clarity, note that p.d.f.  of the limit distribution in (3.6) is

its maximal value is

and its graph is drawn in Fig. 2.

A system is modular if its components can be separated and recombined, with the advantage of flexibility and variety in use. Breaking such a system into independent or nearly independent modules is done in order (see [10]) “to hide the complexity of each part behind an abstraction and interface”. Modularity is implemented in two ways: by splitting an orderly queue (for “queuing–type” models) by events that occur one after another in time, or by sorting spatial items (for “clustering–type” models) into loosely related and similar sets or groups.

An example of the latter is the development of the OS/360 operating system for the original IBM 360 line of computers. At first (see [16]), it was organized in a relatively indecomposable way. It was deemed that each programmer should be familiar with all the material, i.e., should have a copy of the workbook in his office. But over time, this led to a rapid growth of troubles. For example, even maintaining the workbook, whose volume constantly and significantly increased, began to take up significant time from each working day. Therefore, it became necessary to figure out whether programmers developing one particular module really need to know about the structure of the entire system.

It was concluded that, especially in large projects, the answer is negative and managers (quoting from [79]) “should pay attention to minimizing interdependencies. If knowledge is hidden or encapsulated within a module, that knowledge cannot affect, and therefore need not be communicated to, other parts of a system.” This idea has been framed as “information hiding”, a key concept in the modern object–oriented approach to computer programming.

While the modular approach is not needed for $\mathcal{R\hskip-0.5pt/D}$  sums, it is useful when studying $\mathcal{I\hskip-0.5pt/I}$, $\mathcal{R\hskip-0.5pt/D}$, and proper $\mathcal{I\hskip-0.5pt/D}$  sums. In “queuing–type” models, it is applied by splitting an orderly queue by events (see, e.g., [78]: “regenerative phenomena”, or the like) that occur one after another in time. In “clustering–type” models, it is applied by sorting spatial items into loosely related (and similar) sets or groups.

In this case, modular structure is defined by the random indices $0\equiv\mathbb{\ell}_{0}<\mathbb{\ell}_{1}<\mathbb{\ell}_{2}\dots$, where

The key point is the following “exact”¹⁵¹⁵15This means that, unlike, e.g., Dœblin’s dissection, it does not contain incomplete initial and final modules. partition of the proper $\mathcal{I\hskip-0.5pt/I}$  sum ${S}_{N_{t}}$, called Blackwell’s dissection¹⁶¹⁶16This dissection was introduced by Blackwell in [13].:

where $\mathcal{N}_{t}=\inf\big{\{}n>0:\sum_{i=1}^{n}{\mathcal{T}}_{\hskip 1.0pti}>t\big{\}}$, or $\infty$, if the set is empty, ${\mathcal{X}}_{\hskip 1.0pti}=\sum_{j=\mathbb{\ell}_{i-1}+1}^{\mathbb{\ell}_{i}}X_{j}$, and ${\mathcal{T}}_{\hskip 1.0pti}=\sum_{j=\mathbb{\ell}_{i-1}+1}^{\mathbb{\ell}_{i}}T_{j}>0$ by definition. Bearing in mind that $t$ can be exceeded only at a ladder index, equality (4.1) is obvious.

Since $({\mathcal{T}}_{\hskip 1.0pti},{\mathcal{X}}_{\hskip 1.0pti})\overset{d}{=}({\mathcal{T}}_{\hskip 1.0pt},{\mathcal{X}}_{\hskip 1.0pt})$, $i=1,2,\dots$, are i.i.d.  and ${\mathcal{T}}_{\hskip 1.0pt}$ is positive by definition, the sum on the right–hand side of (4.1) is a modular $\mathcal{R\hskip-0.5pt/I}$  sum which is examined as above. Using (4.1), nearly all results on asymptotic normality and its refinements available for $\mathcal{R\hskip-0.5pt/I}$  sums can be transferred to proper $\mathcal{I\hskip-0.5pt/I}$  sums.

In particular (see details in [65], Chapter 9), in Example 2.1 with $0<c<c^{\ast}$ the normal approximation under natural regularity conditions is

where $m_{{}_{\scriptscriptstyle\triangledown}}=\mu_{X}\mu_{T}^{-1}$ and $D_{{\hskip-1.0pt\scriptscriptstyle\triangledown}}^{2}=\mathsf{E}(\mu_{T}X-\mu_{X}T\,)^{2}\mu_{T}^{-3}$. Its refinements, i.e., Berry–Esseen’s estimate and Edgeworth’s expansion, are similar to those in Theorems 3.1 and 3.2 and are omitted in this presentation.

Regarding $m_{{}_{\scriptscriptstyle\triangledown}}$ and $D_{{\hskip-1.0pt\scriptscriptstyle\triangledown}}^{2}$ expressed above in the original (rather than modular) terms, the following should be noted. The refined normal approximation (4.2) obtained by modular approach is first written in terms of the modular random variables $\mathbb{\ell}$, ${\mathcal{T}}_{\hskip 1.0pt}$, and ${\mathcal{X}}_{\hskip 1.0pt}$. Converting them to the original terms requires additional effort. There are several ways to do this. First, Wald’s identities can bring a great release since certain combinations of moments of block random variables ${\mathcal{T}}_{\hskip 1.0pt}$ and ${\mathcal{X}}_{\hskip 1.0pt}$ can be represented through the moments of the original random variables $T$ and $X$. Second, moments of ladder index $\mathbb{\ell}$ and ladder modules ${\mathcal{T}}_{\hskip 1.0pt}$, ${\mathcal{X}}_{\hskip 1.0pt}$ can be calculated analytically, using (see, e.g., [37]) Spitzer’s sums.

For $c>c^{\ast}$, under natural regularity conditions (see details in [65], Chapter 9), quasi–normal approximation¹⁷¹⁷17This name emphasizes the presence of a normal distribution function. In risk theory, it is known as Cramér–Lundberg’s approximation. is

It follows from (4.3) that $\mathsf{P}\{{S}_{N_{t}}<\infty\}\approx\,\mathbb{C}\,e^{-\varkappa t}$, which is an asymptotic formula for defect (cf. (2.3)) of defective $\mathcal{I\hskip-0.5pt/I}$  sum ${S}_{N_{t}}$.

Here $\varkappa$ is a positive solution (w.r.t.  $x$) to the nonlinear equation¹⁸¹⁸18In risk theory, equation (4.4) is called Lundberg’s equation. Its positive solution $\varkappa$ is called Lundberg’s exponent, or adjustment coefficient.

and in terms of the associated random variables $\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu_{i}\overset{d}{=}\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu$ and $\mskip 3.59999mu\overline{\mskip-4.79999mu{X}\mskip-2.0mu}\mskip 1.5mu_{i}\overset{d}{=}\mskip 3.59999mu\overline{\mskip-4.79999mu{X}\mskip-2.0mu}\mskip 1.5mu$, $i=1,2,\dots$,

where ${\mskip 1.5mu\overline{\mskip-2.0mu{V}\mskip-2.40002mu}\mskip 1.5mu}_{n}=\sum_{i=1}^{n}\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu_{i}$. Recall (see, e.g., Example (b) in [37], Chapter XII, Section 4) that the associated random variables we define as follows. Starting with $\mathcal{I\hskip-0.5pt/I}$  basis $(T_{i},X_{i})\overset{d}{=}(T,X)$, $i=1,2,\dots$, we switch from $F_{TX}(t,x)=\mathsf{P}\big{\{}T\leqslant t,X\leqslant x\big{\}}$ to $F_{\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu\mskip 3.59999mu\overline{\mskip-4.79999mu{X}\mskip-2.0mu}\mskip 1.5mu}(t,x)=\mathsf{P}\big{\{}\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu\leqslant t,\mskip 3.59999mu\overline{\mskip-4.79999mu{X}\mskip-2.0mu}\mskip 1.5mu\leqslant x\big{\}}$ defined by the integral $\int_{-cx}^{t}\int_{0}^{x}e^{\varkappa u}\,F_{TX}(du,dv)$. The latter probability distribution is proper and $\mathsf{E}\,\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu>0$. Commonly used shorthand for it is $F_{\mskip 1.5mu\overline{\mskip-2.0mu{T}\mskip-1.20001mu}\mskip 0.90001mu\mskip 3.59999mu\overline{\mskip-4.79999mu{X}\mskip-2.0mu}\mskip 1.5mu}(dt,dx)=e^{\varkappa z}F_{TX}(dt,dx)$.