Second order ancillary: A differential view from continuity

Ancillaries are loved or hated, accepted or rejected, but typically ignored. Recent approximate ancillary methods (e.g., [28]) give a decomposition of the sample space rather than providing statistics on the sample space (e.g., [7, 26]). As a result, continuity gives the contour along which the variable directly measures the parameter and then gives the subcontour that provides measurement of a parameter of interest. This, in turn, enables the high accuracy of cumulant generating function approximations [9, 2] to extend to cover a wide generality of statistical models.

Ancillaries initially arose (see [10]) to examine the accuracy of the maximum likelihood estimate, then (see [11]) to calibrate the loss of information in the use of the maximum likelihood estimate and then (see [12]) to develop a key instance involving the configuration statistic. The configuration of a sample arises naturally in the context of sampling a location-scale model, where a standardized coordinate $z=(y-\mu)/\sigma$ has a fixed and known error distribution $g(z)$: the $i$th coordinate of the response thus has $f(y_{i};\mu,\sigma)=\sigma^{-1}g\{(y_{i}-\mu)/\sigma\}$. The configuration $a(y)$ of the sample is the plug-in estimate of the standardized residual,

where $(\hat{\mu},\hat{\sigma})$ is the maximum likelihood value for $(\mu,\sigma)$ or is some location-scale equivalent. Clearly, the distribution of $\hat{z}$ is free of $\mu$ and $\sigma$ as the substitution $y_{i}=\mu+\sigma z_{i}$ in (1) leads to the cancellation of dependence on $\mu$ and $\sigma$. This supports a common definition for an ancillary statistic $a(y)$, that it has a parameter-free distribution; other conditions are often added to seek sensible results.

More generally, the observed value of an ancillary identifies a sample space contour along which parameter change modifies the model, thus yielding the conditional model on the observed contour as the appropriate model for the data. The ancillary method is to use directly this conditional model identified by the data.

One approach to statistical inference is to use only the observed likelihood function $L^{0}(\theta)=L(\theta;y^{0})$ from the model $f(y;\theta)$ with observed data $y^{0}$. Inference can then be based on some simple characteristic of that likelihood. Alternatively, a weight function $w(\theta)$ can be applied and the composite $w(\theta)L(\theta)$ treated as a distribution describing the unknown $\theta$; this leads to a rich methodology for exploring data, usually, but unfortunately, promoted solely within the Bayesian framework.

A more incisive approach derives from an enriched model which is often available and appropriate. While the commonly cited model is just a set of probability distributions on the sample space, an enriched model can specifically include continuity of the model density function and continuity of coordinate distribution functions. An approach that builds on these enrichments can then, for example, examine the observed data $y^{0}$ in relation to other data points that have a similar shape of likelihood and are thus comparable, and can do even more. For the location-scale model, such points are identified by the configuration statistic; then, accordingly, the model for inference would be $f\{y\mid a(y)=a^{0};\theta\}$, where $a(y)$ is the configuration ancillary.

Exact ancillaries as just described are rather rare and seem limited to location-type models and simple variants. However, extensions that use approximate ancillaries (e.g., [18, 22]) have recently been broadly fruitful, providing approximation in an asymptotic sense. Technical issues can arise with approximate values for an increasing number of coordinates, but these can be managed by using ancillary contours rather than statistics; thus, for a circle, we use explicitly a contour $A=\{(x,y)=(a^{1/2}\cos t,a^{1/2}\sin t)\colon\ t\mbox{ in }[0,2\curpi)\}$ rather than using implicitly a statistic $x^{2}+y^{2}=a$.

We now assume independent coordinate distribution functions that are continuously differentiable with respect to the variable and the parameter; extensions will be discussed separately. Then, rather than working directly with a coordinate distribution function $u_{i}=F_{i}(y_{i};\theta)$, we will use the inverse, the quantile function $y_{i}=y_{i}(u_{i};\theta)$ which presents a data value $y_{i}$ in terms of a corresponding $p$-value $u_{i}$. For additional advantage, we could use a scoring variable $x$ in place of the $p$-value, for example, $x=\Phi^{-1}(u)$ or $x=F^{-1}(u;\theta_{0})$, where $\Phi(\cdot)$ is the standard Normal distribution function. We can then write $y=y(x;\theta)$, where a coordinate $y_{i}$ is presented in terms of the corresponding scoring variable $x_{i}$.

For the full response variable, let $y=y(x;\theta)=\{y_{1}(x_{1};\theta),\ldots,y_{n}(x_{n};\theta)\}^{\prime}$ be the quantile vector expressing $y$ in terms of the reference or scoring variable $x$ with its given distribution: the quantile vector records how parameter change affects the response variable and its distribution, as prescribed by the continuity of the coordinate distribution functions.

For an observed data point $y^{0}$, a convenient reference value $\hat{x}^{0}$ or the fitted $p$-value vector is obtained by solving the equation $y^{0}=y(x;\hat{\theta}^{0})$ for $x$, where $\hat{\theta}^{0}$ is the observed maximum likelihood value; for this, we assume regularity and asymptotic properties for the statistical model. The contour of the second order ancillary through the observed data point as developed in this paper is then given as the trajectory of the reference value,

to second order under parameter change, where $p$ here is the dimension of the parameter. A sample space point on this contour has, to second order, the same estimated $p$-value vector as the observed data point and special properties for the contours are available to second order.

The choice of the reference variable with given data has no effect on the contour: the reference variable could be Uniform, as with the $p$-value; or, it could be the response distribution itself for some choice of the parameter, say $\theta_{0}$.

For the location-scale example mentioned earlier, we have the coordinate quantile function $y_{i}=\mu+\sigma z_{i}$, where $z_{i}$ has the distribution $g(z)$. The vector quantile function is

where $1=(1,\ldots,1)^{\prime}$ is the ‘one vector.’ With the data point $y^{0}$, we then have the fitted $\hat{z}^{0}=(y^{0}-\hat{\mu}^{0}1)/\hat{\sigma}^{0}$. The observed ancillary contour to second order is then obtained from (2) by substituting $\hat{z}^{0}$ in the quantile (3):

with positive coefficient for the second vector. This is the familiar exact ancillary contour $a(y)=a^{0}$ from (1).

An advantage of the vector quantile function in the context of the enriched model mentioned above is that it allows us to examine how parameter change modifies the distribution and thus how it moves data points as a direct expression of the explicit continuity. In this sense, we define the velocity vector or vectors as $v(x;\theta)=(\partial/\partial\theta)y(x;\theta)=\partial y/\partial\theta$. In the scalar $\theta$ case, this is a vector recording the direction of movement of a point $y$ under $\theta$ change; in the vector $\theta$ case, it is a $1\times p$ array of such vectors in $\mathbb{R}^{n}$, $V(x;\theta)=\{v_{1}(x_{1};\theta),\ldots,v_{p}(x_{p};\theta)\},$ recording the separate effects from the parameter coordinates $\theta_{1},\ldots,\theta_{p}$. For the location-scale example, the velocity array is $V(z;\mu,\sigma)=(1,z),$ which can be viewed as a $1\times 2$ array of vectors in $\mathbb{R}^{n}$.

The ancillary contour can then be presented using a Taylor series about $y^{0}$ with coefficients given by the velocity and acceleration $V$ and $W$. For the location-scale example, the related acceleration vectors are equal to zero.

For more insight, consider the general scalar $\theta$ case and the velocity vector $v(x;\hat{\theta}^{0})$. For a typical coordinate, this gives the change $\mathrm{d}y=v(x;\hat{\theta}^{0})\,\mathrm{d}\theta$ in the variable as produced by a small change $\mathrm{d}\theta$ at $\hat{\theta}^{0}$. A re-expression of the coordinate variable can make these increments equal and produce a location model; the product of these location models is a full location model $g(y_{1}-\theta,\ldots,y_{n}-\theta)$ that precisely agrees with the initial model to first derivative at $\theta=\hat{\theta}^{0}$ (see [20, 1]). This location model then, in turn, determines a full location ancillary with configuration $a(y)=(y_{1}-\bar{y},\ldots,y_{n}-\bar{y})$. For the original model, this configuration statistic has first-derivative ancillarity at $\theta=\hat{\theta}^{0}$ and is thus a first order approximate ancillary; the tangent to the contour at the data point is just the vector $v(\hat{x}^{0};\hat{\theta}^{0})$. Also this contour can be modified to give second order ancillarity.

In a somewhat different way, the velocity vector $v(y^{0};\theta)$ at the data point $y^{0}$ gives information as to how data change at $y^{0}$ relates to parameter change at various $\theta$ values of interest. This allows us to examine how a sample space direction at the data point relates to estimated $p$-value and local likelihood function shape at various $\theta$ values; this, in turn, leads to quite general default priors for Bayesian analysis (see [21]).

In the presence of a cumulant generating function, the saddle-point method has produced highly accurate third order approximations for density functions (see [9]) and for distribution functions (see [25]). Such approximations are available in the presence of exact ancillaries [2] and extend widely in the presence of approximate ancillaries (see [18]). For third order accuracy, only second order approximate ancillaries are needed, and for such ancillaries, only the tangents to the ancillary contour at the data point are needed (see [18, 19]). With this as our imperative, we develop the second order ancillary for statistical inference.

Tangent vectors to an ancillary at a data point give information as mentioned above concerning a location model approximation at the data point. For a scalar parameter, these provide a vector field and integrate quite generally to give a unique approximate ancillary to second order accuracy. The resulting conditional model then provides definitive $p$-values by available theory; see, for example, [22]. For a vector parameter, however, the multiple vector fields may not satisfy the Frobenius conditions for integrability and thus may not define a function.

Under mild conditions, however, we show that such tangent vectors do generate a surface to second order without the Frobenius conditions holding. We show this in several steps. First, we obtain the coordinate quantile functions $y_{i}=y_{i}(x_{i};\theta).$ Second, we Taylor series expand the full vector quantile $y=(y_{1},\ldots,y_{n})$ in terms of the full reference variable $x=(x_{1},\ldots,x_{n})$ and the parameter $\theta=(\theta_{1},\ldots,\theta_{p})$ about data-based values, appropriately re-expressing coordinates and working to second order. Third, we show that this generates a partition with second order ancillary properties and the usual tangent vectors. The seeming need for the full Frobenius conditions is bypassed by finding that two integration routes need not converge to each other, but do remain on the same contour, calculating, of course, to second order.

This construction of an approximate ancillary is illustrated in Section 2 using the familiar example, the Normal-on-the circle from [13]; see also [8, 3, 20, 16]. The example, of course, does have an exact ancillary and the present procedure gives an approximation to that ancillary. In Section 3, we consider various examples that have exact and approximate ancillaries, and then in Sections 4 and 5, we present the supporting theory. In particular, in Section 4, we develop notation for a $p$-dimensional contour in $\mathbb{R}^{n}$, $A=\{y(x_{0};t)\colon\ t\mbox{ in }\mathbb{R}^{p}\},$ and use velocity and acceleration vectors to present a Taylor series with respect to $t$. Then, in Section 5, we consider a regular statistical model with asymptotic properties and use the notation from Section 4 to develop the second order ancillary contour through an observed data point $y^{0}$. The re-expression of individual coordinates, both of the variable and the parameter, plays an essential role in the development; an asymptotic analysis is used to establish the second order approximate ancillarity. Section 6 contains some discussion.

We illustrate the second order approximate ancillary with a simple nonlinear regression model, the Normal-on-the-circle example (see [13]). The model has a well-known exact ancillary. Let $y=(y_{1},y_{2})^{\prime}$ be Normal on the plane with mean $(\rho\cos\theta,\rho\sin\theta)^{\prime}$ and variance matrix $I/n$ with $\rho$ known. The mean is on a circle of fixed radius $\rho$ and the distribution has rotationally symmetric error with variances $n^{-1}$, suggesting an antecedent sample size $n$ for an asymptotic approach. The full $n$-dimensional case is examined as Example 2 in Section 3 and the present case derives by routine conditioning.

The distribution is a unit probability mass centered at $(\rho\cos\theta,\rho\sin\theta)^{\prime}$ on the circle with radius $\rho$. If rotations about the origin are applied to $(y_{1},y_{2})^{\prime}$, then the probability mass rotates about the origin, the mean moves on the circle with radius $\rho$ and an element of probability at a distance $r$ from the origin moves on a circle of radius $r$. The fact that the rotations move probability along circles but not between circles of course implies that probability on any circle about the origin remains constant: probability flows on the ancillary contours. Accordingly, we have that the radial distance $r=(y_{1}^{2}+y_{2}^{2})^{1/2}$ has a fixed $\theta$-free distribution and is thus ancillary.

The statistic $r(y)$ is the Fisher exact ancillary for this problem and Fisher recommended that inference be based on the conditional model, given the observed ancillary contour. This conditional approach has a long but uneven history; [17] provides an overview and [23] offer links with asymptotic theory. We develop the approximate second order ancillary and examine how it relates to the Fisher exact ancillary.

The model for the Normal-on-the-circle has independent coordinates, so we can invert the coordinate distribution functions and obtain the vector quantile function,

where the $x_{i}=\Phi^{-1}(u_{i})/n^{1/2}$ are independent normal variables with means $0$ and variances $n^{-1}$, and $\Phi$ is the standard Normal distribution function. We now examine the second order ancillary contour $A^{0}$ given by (2).

Let $y^{0}=(y_{1}^{0},y_{2}^{0})^{\prime}=(r^{0}\cos a^{0},r^{0}\sin a^{0})$ be the observed data point where $r^{0}$, $a^{0}$ are the corresponding polar coordinates; see Figure 1. For this simple nonlinear normal regression model, $\hat{\theta}^{0}=a^{0}$ is the angular direction of the data point. The fitted reference value $\hat{x}^{0}$ is the solution of the equation $y^{0}=y(x;\hat{\theta}^{0})=\rho(\cos a^{0},\sin a^{0})^{\prime}+(x_{1},x_{2})$, giving $\hat{x}^{0}=(\hat{x}_{1}^{0},\hat{x}_{2}^{0})^{\prime}=y^{0}-\rho(\cos a^{0},\sin a^{0})^{\prime}=y^{0}-\hat{y}^{0}$, where $\hat{y}^{0}=\rho(\cos a^{0},\sin a^{0})^{\prime}$ is the fitted value, which is the projection of the data point $y^{0}$ onto the circle. The observed ancillary contour is then

Figure 1 shows that $A^{0}=\{y(\hat{x}^{0};t)\colon\ t\mbox{ near }a^{0}\}$ is a translation, as shown by the arrow of a segment $S$ of the solution contour, from the fitted point $\hat{y}^{0}$ to the data point $y^{0}$.

The second order ancillary segment at $y^{0}$ does not lie on the exact ancillary surface $r(y_{1},y_{2})=r^{0}$. The tangent vector at the data point $y^{0}$ is $v=(\partial y/\partial t)|_{t=a^{0}}=(-\rho\sin a^{0},\rho\cos a^{0})^{\prime}$, which is the same as the tangent vector for the exact ancillary and which agrees with the usual tangent vector $v$ (see [22]). However, the acceleration vector is $w=(\partial^{2}/\partial t^{2})y|_{t=a^{0}}=(-\rho\sin a^{0},-\rho\cos a^{0})^{\prime}$, which differs slightly from that for the exact ancillary: the approximation has radius of curvature $\rho$, as opposed to $r^{0}$ for the exact, but the difference in moderate deviations about $y^{0}$ can be seen to be small and is second order.

The second order ancillary contour through $y^{0}$ can also be expressed in a Taylor series as $A^{0}=\{y^{0}+tv+wt^{2}/2\colon\ t\mbox{ near }0\}$; here, the acceleration vector $w$ is orthogonal to the velocity vector $v$. Similar results hold in wide generality when $y$ has dimension $n$ and $\theta$ has dimension $p$; further examples are discussed in the next section and the general development follows in Sections 4 and 5.

For the Normal-on-the-circle example, the exact ancillary contour was given as the observed contour of the radial distance $r(y_{1},y_{2})$: the contour is described implicitly. By contrast, the approximate ancillary was given as the trajectory of a point $y(\hat{x}^{0};t)$ under change of an index or mathematical parameter $t$: the contour is described explicitly. For the general context, the first approach has serious difficulties, as found even with nonlinear regression, and these difficulties arise with an approximate statistic taking an approximate value; see Example 2. Accordingly, we now turn to the second, the explicit approach, and develop the needed notation and expansions.

Consider a smooth one-dimensional contour through some point $y_{0}$. To describe such a contour in the implicit manner requires $n-1$ complementary statistics. By contrast, for the explicit method, we write $y=y(t)$, which maps a scalar $t$ into the sample space $\mathbb{R}^{n}$. More generally, for a $p$-dimensional contour, we have $y=y(t)$ in $\mathbb{R}^{n}$, where $t$ has dimension $p$ and the mapping is again into $\mathbb{R}^{n}$.

For such a contour, we define the row array $V(t)=(\mathrm{d}/{\mathrm{d}t^{\prime}})y(t)=\{v_{1}(t),\ldots,v_{p}(t)\}$ of tangent vectors, where the vector $v_{\alpha}(t)=(\mathrm{d}/\mathrm{d}t_{\alpha})y(t)$ gives the direction or gradient of $y(t)$ with respect to change in a coordinate $t_{\alpha}$. We are interested in such a contour near a particular point $y_{0}=y(t_{0})$; for convenience, we often choose $y_{0}$ to be the observed data point $y^{0}$ and the $t_{0}$ to be centered so that $t_{0}=0$. In particular, the array $V=V(t_{0})$ of tangent vectors at a particular data point $y_{0}$ will be of special interest. The vectors in $V$ generate a tangent plane $\mathcal{L}(V)$ at the point $y_{0}$ and this plane provides a linear approximation to the contour. Differential geometry gives length properties of such vectors as the first fundamental form:

this records the matrix of inner products for the vectors $V$ as inherited from the inner product on $\mathbb{R}^{n}$. A change in the parameterization $\tilde{t}=t(t)$ of the contour will give different tangent vectors $V$, the same tangent plane $\mathcal{L}(V)$ and a different, but corresponding, first fundamental form.

Now, consider the derivatives of the tangents $V(t)$ at $t_{0}$:

where $w_{\alpha\alpha^{\prime}}=(\partial^{2}/\partial t_{\alpha}\,\partial t_{\alpha^{\prime}})y(t)|_{t=t_{0}}$ is an acceleration or curvature vector relative to coordinates $t_{\alpha}$ and $t_{\alpha^{\prime}}$ at $t_{0}$. We regard the array $W$ as a $p\times p$ array of vectors in $\mathbb{R}^{n}$. We could have used tensor notation, but the approach here has the advantage that we can write the second degree Taylor expansion of $y(t)$ at $t_{0}=0$ as

which uses matrix multiplication for linearly combining the vectors in the arrays $V$ and $W$. Some important characteristics of the quadratic term in (5) are obtained by orthogonalizing the elements of $W$ to the tangent plane $\mathcal{L}(V)$, to give residuals

this uses the regression analysis projection matrix $P=V(V^{\prime}V)^{-1}V^{\prime}$. The full array $\tilde{W}$ of such vectors $\tilde{w}_{\alpha\alpha^{\prime}}$ is then written $\tilde{W}=W-PW=W-VH,$ where $H=(h_{\alpha\alpha^{\prime}})$ is a $p\times p$ array of elements $h_{\alpha\alpha^{\prime}}=(V^{\prime}V)^{-1}V^{\prime}w_{\alpha\alpha^{\prime}}$; an element $h_{\alpha\alpha^{\prime}}$ is a $p\times 1$ vector, which records the regression coefficients of $w_{\alpha\alpha^{\prime}}$ on the vectors $V$.

The array $\tilde{W}$ of such orthogonalized curvature vectors $\tilde{w}$ is the second fundamental form for the contour at the expansion point. Consider the Taylor expansion (5) and substitute $W=\tilde{W}+VH$:

where we note that $t$ and $t^{\prime}$ are being applied to the $p\times p$ arrays $H$ and $\tilde{W}$ by matrix multiplication, but the elements are $p\times 1$ vectors for $H$ and $n\times 1$ vectors for $\tilde{W}$, and these are being combined linearly. We can then write $y(t)=y_{0}+V\tilde{t}+\tilde{t}\tilde{W}\tilde{t}^{\prime}/2+\cdots$ and thus have the contour expressed in terms of orthogonal curvature vectors $\tilde{w}$ with the reparameterization $\tilde{t}=t+t^{\prime}Ht/2+\cdots$. When we use this in the asymptotic setting, we will have standardized coordinates and the reparameterization will take the form $\tilde{t}=t+t^{\prime}Ht/2n^{1/2}+\cdots$.

We have used the Normal-on-the-circle example to illustrate the proposed second order ancillary contour $\{y(\hat{x}^{0};t)\}$. Now, generally, let $f(y;\theta)$ be a statistical model with regularity and asymptotic properties as the data dimension $n$ increases: we assume that the vector quantile $y(x;\theta)$ has independent scalar coordinates and is smooth in both the reference variable $x$ and the parameter $\theta$; more general conditions will be considered subsequently. For the verification, we use a Taylor expansion of the quantile function in terms of both $x$ and $\theta$, and work from theory developed in [5] and [1]. The first steps involve the re-expression of individual coordinates of $y$, $x$, and $\theta$, and show that the proposed contours establish a partition on the sample space; the subsequent steps establish the ancillarity of the contours. (

• 1a)]

• (1a)

  Standardizing the coordinates. Consider the statistical model in moderate deviations about $(y^{0},\hat{\theta}^{0})$ to order $\mathrm{O}(n^{-1})$. For this, we work with coordinate departures in units scaled by $n^{-1/2}$. Thus, for the $i$th coordinate, we write $y_{i}=\hat{y}_{i}^{0}+\tilde{y}_{i}/n^{1/2}$, $x_{i}=\hat{x}_{i}^{0}+\tilde{x}_{i}/n^{1/2}$ and $\theta_{\alpha}=\hat{\theta}^{0}_{\alpha}+\tilde{\theta}_{\alpha}/n^{1/2}$; and for a modified $i$th quantile coordinate $\tilde{y}_{i}=\tilde{y}_{i}(\tilde{x}_{i},\hat{\theta})$, we Taylor expand to the second order, omit the subscripts and tildes for temporary clarity, and obtain $y=x+V\theta+(ax^{2}+2xB\theta+\theta^{\prime}W\theta)/2n^{1/2}$, where $V$ is the $1\times p$ gradient of $y$ with respect to $\theta$, $B$ is the $1\times p$ cross Hessian with respect to $x$ and $\theta$, $W$ is the $p\times p$ Hessian with respect to $\theta$ and vector–matrix multiplication is used for combining $\theta$ with the arrays.

• (1b)

  Re-expressing coordinates for a nicer expansion. We next re-express an $x$ coordinate, writing $\tilde{x}=x+ax^{2}/2n^{1/2}$, and then again omit the tildes to obtain the simpler expansion

  $$y=x+V\theta+(2xB\theta+\theta^{\prime}W\theta)/2n^{1/2}+\cdots,$$

  (6)

  to order $\mathrm{O}(n^{-1})$ for the modified $y$, $x$ and $\theta$, now in bounded regions about 0.

• (1c)

  Full response vector expansion. For the vector response $y=(y_{1},\ldots,y_{n})$ in quantile form, we can compound the preceding coordinate expansions and write $y=x+V\theta+(2x:B\theta+\theta^{\prime}W\theta)/2n^{1/2}+\cdots,$ where $y$ and $x$ are now vectors in $\mathbb{R}^{n}$, $V=(v_{1},\ldots,v_{p})=(v_{\alpha})$ and $B=(b_{1},\ldots,b_{p})=(b_{\alpha})$ are $1\times p$ arrays of vectors in $\mathbb{R}^{n}$, $W=(w_{\alpha\alpha^{\prime}})$ is a $p\times p$ array of vectors in $\mathbb{R}^{n}$ and $x\mbox{:}B$ is a $1\times p$ array of vectors $x\mbox{:}b$, where the $i$th element of the vector $x\mbox{:}b$ is the product $x_{i}b_{i}$ of the $i$th elements of the vectors $x$ and $b$.

• (1d)

  Eliminate the cross Hessian: scalar parameter case. The form of a Taylor series depends heavily on how the function and the component variables are expressed. For a particular coordinate of (6) in (1b), if we re-express the coordinate $y=\tilde{y}+c\tilde{y}^{2}/2n^{1/2}$ in terms of a modified $\tilde{y}$, substitute it in (6) and then, for notational ease, omit the tildes, we obtain $y+c(x+v\theta)^{2}/2n^{1/2}=x+v\theta+(2xb\theta+\theta^{2}w)/2n^{1/2}.$ To simplify this, we take the $x^{2}$ term over to the right-hand side and combine it with $x$ to give a re-expressed $x$, take the $\theta x$ term over to the right-hand side and choose $c$ so that $cv=b$ and, finally, combine the $\theta^{2}$ terms giving a new $w$. We then obtain ${y(x;\theta)=x+v\theta+\theta^{2}w/2n^{1/2}}$ with the cross Hessian removed; for this, if $v=0$, we ignore the coordinate as being ineffective for $\theta$. For the full response accordingly, we then have $y(x;\theta)=x+v\theta+w\theta^{2}/2n^{1/2}+\cdots$ to the second order in terms of re-expressed coordinates $x$ and $y$. The trajectory of a point $x$ is $A(x)=\{y(x;t)\}=\{x+vt+wt^{2}/2n^{1/2}+\cdots\}$ to the second order as $t$ varies.

• (1e)

  Scalar case: trajectories form a partition. In the standardized coordinates, the initial data point is $y^{0}=0$ with corresponding maximum likelihood value $\hat{\theta}^{0}=0$; the corresponding trajectory is $A(0)=\{vt+wt^{2}/2n^{1/2}+\cdots\}.$ For a general reference value $x$, but with $\hat{\theta}(x)=0$, the trajectory is $A(x)=\{x+vt+wt^{2}/2n^{1/2}+\cdots\}=x+A(0)$. The sets $\{A(x)\}$ with $\hat{\theta}(x)=0$ are all translates of $A(0)$ and thus form a partition.

  Consider an initial point $x_{0}$ with maximum likelihood value $\hat{\theta}(x_{0})=0$ and let $y_{1}=x_{0}+vt_{1}+wt_{1}^{2}/2n^{1/2}+\cdots$ be a point in the set $A(x_{0})=x_{0}+A(0)$. We calculate the trajectory $A(y_{1})$ of $y_{1}$ and show that it lies on $A(x_{0})$; the partition property then follows and the related Jacobian effect is constant. From the quantile function $y=x+v\theta+w\theta^{2}/2n^{1/2}$, we see that the $y$ distribution is a $\theta$-based translation of the reference distribution described by $x$. Thus the likelihood at $y_{1}$ is $l(y_{1}-v\theta-w\theta^{2}/2n^{1/2})$, in terms of the log density $l(x)$ near $x_{0}$. It follows that $y_{1}=x_{0}+vt_{1}+wt_{1}^{2}/2n^{1/2}$ has maximum likelihood value $\hat{\theta}(y_{1})=t_{1}$.

  Now, for the trajectory about $y_{1}$, we calculate derivatives

  $$\frac{\mathrm{d}y}{\mathrm{d}\theta}=v+w\theta/n^{1/2},\qquad\frac{\mathrm{d}^{2}y}{\mathrm{d}\theta^{2}}=w/n^{1/2},$$

  which, at the point $y_{1}=vt_{1}+wt^{2}_{1}/2n^{1/2}$ with $\theta=\hat{\theta}(y_{1})$, gives

  $$V(y_{1})=v+wt_{1}/n^{1/2},\qquad W(y_{1})=w/n^{1/2},$$

  to order $\mathrm{O}(n^{-1})$. We thus obtain the trajectory of the point $y_{1}$:

  	$\displaystyle A(y_{1})$	$\displaystyle=$	$\displaystyle\{x_{0}+vt_{1}+wt_{1}^{2}/2n^{1/2}+(v+wt_{1}/n^{1/2})t+wt^{2}/2n^{1/2}\}$
  		$\displaystyle=$	$\displaystyle\{x_{0}+vT+wT^{2}/2n^{1/2}\}$

  under variation in $t$. However, with $T=t_{1}+t$, we have just an arbitrary point on the initial trajectory. Thus the mapping $y\to A(y)$ is well defined and the trajectories generate a partition, to second order in moderate derivations in $\mathbb{R}^{n}$. In the standardized coordinates, the Jacobian effect is constant.

• (1f)

  Vector case: trajectories form a partition. For the vector parameter case, we again use standardized coordinates and choose a parameterization that gives orthogonal curvature vectors $w$ at the observed data point $y^{0}$. We then examine scalar parameter change on some line through $\hat{\theta}(y^{0})$. For this, the results above give a trajectory and any point on it reproduces the trajectory under that scalar parameter. Orthogonality ensures that the vector maximum likelihood value is on the same line just considered. These trajectories are, of course, part of the surface defined by $\{Vt+t^{\prime}Wt/2n^{1/2}\}$. We then use the partition property of the individual trajectories as these apply perpendicular to the surface; the surfaces are thus part of a partition. We can then write the trajectory of a point $x$ as a set

  $$A(x)=\{x+Vt+t^{\prime}Wt/2n^{1/2}+\cdots\colon\ t\}=x+A(0)$$

  (7)

  in a partition to the second order in moderate deviations.

• (2a)

  Observed information standardization. With moderate regularity, and following [18] and [23], we have a limiting Normal distribution conditionally on $y^{0}+\mathcal{L}(V)$. We then rescale the parameter at $\hat{\theta}^{0}$ to give identity observed information and thus an identity variance matrix for the Normal distribution to second order. We also have a limiting Normal distribution conditionally on $y^{0}+\mathcal{L}(V,W)$; for this, we linearly modify the vectors in $W$ by rescaling and regressing on $\mathcal{L}(V)$ to give distributional orthogonality to $\hat{\theta}$ and identity conditional variance matrix to second order.

• (2b)

  The trajectories are ancillary: first derivative parameter change. We saw in the preceding section that key local properties of a statistical model were summarized by the tangent vectors $V$ and the curvature vectors $W$, and that the latter can, to advantage, be taken to be orthogonal to the tangent vectors. These vectors give local coordinates for the model and can be replaced by an appropriate subset if linear dependencies are present.

  First, consider the conditional model given the directions corresponding to the span $y^{0}+\mathcal{L}\{V,W\}$. From the ancillary expansion (7), we have that change of $\theta$ to the second order moves points within the linear space $y^{0}+\mathcal{L}\{V,W\}$; accordingly, this conditioning is ancillary. Then, consider the further conditioning to an alleged ancillary contour, as described by (7). Also, let $y_{0}$ be a typical point having $\hat{\theta}(y_{0})=\hat{\theta}^{0}$ as the corresponding maximum likelihood value; $y_{0}$ is thus on the observed maximum likelihood contour.

  Now, consider a rotationally symmetric Normal distribution on the $(x,y)$ plane with mean $\theta$ on the $x$ axis and let $a=y+cx^{2}/2$ be linear in $y$ with a quadratic adjustment with respect to $x$. Then $a=a(x,y)$ is first-derivative ancillary at $\theta=0$. For this, we assume, without loss of generality, that the standard deviations are unity. The marginal density for $a$ is then

  $$f(a;\theta)=\int_{-\infty}^{\infty}\phi(x-\theta)\phi(a-cx^{2}/2)\,\mathrm{d}x,$$

  which is symmetric in $\theta$; thus $(\mathrm{d}/\mathrm{d}\theta)f(a;\theta)|_{\theta=0}=0,$ showing that the distribution of $a$ is first-derivative ancillary at $\theta=0$ or, more intuitively, that the amount of probability on a contour of $a$ is first-derivative free of $\theta$ at $\theta=0$. Of course, for this, the $y$-spacing between contours of $a$ is constant.

  Now, more generally, consider an asymptotic distribution for $(x,y)$ that is first order rotationally symmetric Normal with mean $\theta$ on the $y=0$ plane; this allows $\mathrm{O}(n^{-1/2})$ cubic contributions. Also, consider an $s$-dimensional variable $a=y+Q(x)/2n^{1/2}$ which is a quadratic adjustment of $y$. The preceding argument extends to show that $a(y)$ is first-derivative ancillary: the two $\mathrm{O}(n^{-1/2})$ effects are zero and the combination is of the next order.

• (2c)

  Trajectories are ancillary: parameter change in moderate deviations. Now, consider a statistical model $f(y;\theta)$ with data point $y^{0}$ and assume regularity, asymptotics and smoothness of the quantile functions. We examine the parameter trajectory $\{y(\hat{x}^{0};t)\}$ in moderate deviations under change in $t$. From the preceding paragraph, we then have first-derivative ancillarity at $\theta=\hat{\theta}=0$. But this holds for each expansion in moderate deviations and we thus have ancillarity in moderate deviations. The key here has been to use the expansion form about the point that has $\hat{\theta}$ equal to the parameter value being examined.