This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

State-space analysis of a continuous gravitational wave source with a pulsar timing array: inclusion of the pulsar terms

Tom Kimpson1,2, Andrew Melatos1,2, Joseph O’Leary1,2, Julian B. Carlin1,2, Robin J. Evans3, William Moran3, Tong Cheunchitra1,2, Wenhao Dong1,2, Liam Dunn1,2, Julian Greentree3, Nicholas J. O’Neill1,2, Sofia Suvorova3, Kok Hong Thong1,2, Andrés F. Vargas1,2
1School of Physics, University of Melbourne, Parkville, VIC 3010, Australia
2OzGrav, University of Melbourne, Parkville, VIC 3010, Australia
3Department of Electrical and Electronic Engineering, University of Melbourne, Parkville, VIC 3010, Australia
Contact e-mail: tom.kimpson@unimelb.edu.au
(Last updated )
Abstract

Pulsar timing arrays can detect continuous nanohertz gravitational waves emitted by individual supermassive black hole binaries. The data analysis procedure can be formulated within a time-domain, state-space framework, in which the radio timing observations are related to a temporal sequence of latent states, namely the intrinsic pulsar spin frequency. The achromatic wandering of the pulsar spin frequency is tracked using a Kalman filter concurrently with the pulse frequency modulation induced by a gravitational wave from a single source. The modulation is the sum of terms proportional to the gravitational wave strain at the Earth and at every pulsar in the array. Here we generalize previous state-space formulations of the pulsar timing array problem to include the pulsar terms; that is, we copy the pulsar terms from traditional, non-state-space analyses over to the state-space framework. The performance of the generalized Kalman filter is tested using astrophysically representative software injections in Gaussian measurement noise. It is shown that including the pulsar terms corrects for previously identified biases in the parameter estimates (especially the sky position of the source) which also arise in traditional matched-filter analyses that exclude the pulsar terms. Additionally, including the pulsar terms decreases the minimum detectable strain by 14%14\%. Overall, the study verifies that the pulsar terms do not raise any special extra impediments for the state-space framework, beyond those studied in traditional analyses. The inspiral-driven evolution of the wave frequency at the Earth and at the retarded time at every pulsar in the array is also investigated.

keywords:
gravitational waves – methods: data analysis – pulsars: general
pubyear: 2024pagerange: State-space analysis of a continuous gravitational wave source with a pulsar timing array: inclusion of the pulsar termsG

1 Introduction

Nanohertz gravitational waves (GWs) are produced by the inspiral of supermassive black hole binaries (SMBHBs; Rajagopal & Romani, 1995; Jaffe & Backer, 2003; Wyithe & Loeb, 2003; Sesana, 2013; McWilliams et al., 2014; Ravi et al., 2015; Burke-Spolaor et al., 2019; Sykes et al., 2022). They modulate sinusoidally the times of arrival (TOAs) at the Earth of radio pulses from pulsars. A pulsar timing array (PTA; Tiburzi, 2018; Verbiest et al., 2021) simultaneously measures the TOAs from multiple pulsars to search for a coincident GW signature. Congruent evidence from PTA observations has been presented (Agazie et al., 2023a; Antoniadis et al., 2023a; Reardon et al., 2023; Xu et al., 2023) for a stochastic GW background, which arises from the incoherent superposition of multiple SMBHB sources (Allen, 1997; Sesana et al., 2008; Christensen, 2019; Renzini et al., 2022). SMBHBs that are sufficiently close and massive may be resolvable individually but no definitive detection has been claimed so far (Jenet et al., 2004; Zhu et al., 2014; Babak et al., 2016; Arzoumanian et al., 2023; Antoniadis et al., 2023c).

The modulation in a pulsar’s TOAs due to a GW has two contributions: an “Earth term” and “pulsar terms” which are proportional to the GW strain at the Earth and every pulsar respectively. Whilst the Earth term is phase coherent between all pulsars, the pulsar terms have uncorrelated phases. Consequently the pulsar terms are typically considered as a source of self-noise and dropped from many — although not all — standard PTA analyses (e.g. Sesana & Vecchio, 2010; Babak & Sesana, 2012a; Petiteau et al., 2013; Zhu et al., 2015; Taylor et al., 2016; Goldstein et al., 2018; Charisi et al., 2023). It is acknowledged in standard analyses that dropping the pulsar terms leads to biases in the inferred parameters and reduces the detection probability, but both impacts are modest (Zhu et al., 2016; Chen & Wang, 2022; Kimpson et al., 2024).

Kimpson et al. (2024) (K24 hereafter) introduced a state-space method for the detection and parameter estimation of continuous GWs from individual SMBHBs, which self-consistently tracks the intrinsic, achromatic timing noise in PTA pulsars (e.g. Shannon & Cordes, 2010a; Lasky et al., 2015; Caballero et al., 2016; Goncharov et al., 2021) and disentangles it from GW-induced TOA modulations. The method described in K24 complements standard PTA analyses by tracking the specific, measured, time-ordered realization of the TOA fluctuations instead of fitting for their ensemble-averaged statistics (such as their power spectral density), following the customary signal processing approach in many industrial and scientific electrical engineering applications. The method described in K24 also follows the example of many standard PTA analyses and drops the pulsar terms. In this paper we achieve two goals. First, we extend K24 to include the pulsar terms. We demonstrate how the static GW parameters and the GW phase at each pulsar can be estimated successfully by combining a Kalman filter (Kalman, 1960) with a Bayesian nested sampler (Skilling, 2006; Ashton et al., 2022), repeating the successful demonstration in K24 but with the pulsar terms now included. Second, we quantify how including the pulsar terms improves (i) the accuracy with which the static GW parameters are estimated, and (ii) the minimum detectable GW strain, compared to when the pulsar terms are omitted. We emphasize that the biases incurred by omitting the pulsar terms have been studied thoroughly in the context of standard PTA analyses (Zhu et al., 2016; Chen & Wang, 2022); they are not new effects discovered here for the first time. Rather, the goal of this paper is to study them again in the context of the promising state-space formulation of the PTA analysis problem introduced by K24, to check whether they raise any special extra impediments beyond those studied in standard PTA analyses. The state-space formulation complements standard analysis techniques, e.g. based on matched filters (Anholm et al., 2009; Chamberlin et al., 2015). It does not supplant them and is likely to be most informative when run in tandem.

The paper is organised as follows. In Section 2 we briefly review the state-space formulation of PTA data analysis introduced by K24. In Section 3 we show how to include the pulsar terms via a convenient reparametrisation. In Sections 4 and 5 we explain how to test the updated model, inclusive of the pulsar terms, by employing synthetic data for a single representative GW source across an astrophysically relevant domain of SMBHB source parameters. In Section 6 we quantify the parameter estimation accuracy and the minimum detectable GW strain, comparing the performance when the pulsar terms are included and excluded.. Conclusions are drawn in Section 7. Throughout the paper we adopt natural units, with c=G==1c=G=\hbar=1, and the metric signature (,+,+,+)(-,+,+,+).

2 State-Space Formulation

In this section we briefly review the state-space formulation of a PTA experiment, as presented in K24. There are NN pulsars in the array, labelled 1nN1\leq n\leq N. The intrinsic spin frequency of the nn-th pulsar, fp(n)(t)f_{\rm p}^{(n)}(t), measured in the local, freely-falling rest frame of the pulsar’s centre of mass, evolves according to a stochastic differential equation, which describes secular braking combined with intrinsic, achromatic spin wandering (“timing noise”). The radio pulse frequency measured by an observer at Earth, fm(n)(t)f_{\rm m}^{(n)}(t), is related to fp(n)(t)f_{\rm p}^{(n)}(t) via a measurement equation, which describes the TOA modulation induced by the GW. In Section 2.1 we define and justify the phenomenological equation of motion for fp(n)(t)f_{\rm p}^{(n)}(t). In Section 2.2 we outline the measurement equation relating fm(n)(t)f_{\rm m}^{(n)}(t) to fp(n)(t)f_{\rm p}^{(n)}(t). In Section 2.3 we summarise the static parameters of the model, including the GW source parameters, which are estimated ultimately from the data by nested sampling. The analysis method accepts as an input a sequence of pulse frequencies, instead of a sequence of pulse TOAs, in order to validate the approach in its simplest incarnation and to maintain consistency with previous work (Meyers et al., 2021a; Meyers et al., 2021b; Kimpson et al., 2024). It will be necessary to modify the method to accept pulse TOAs when analyzing real data, a subtle extension which we defer to a forthcoming paper.

2.1 Spin evolution

We assume that the rest frame spin frequency of the nn-th pulsar evolves according to a mean-reverting Ornstein-Uhlenbeck process, described by a Langevin equation with a time-dependent drift term (Vargas & Melatos, 2023),

dfp(n)dt=γ(n)[fp(n)fem(n)(t)]+f˙em(n)(t)+ξ(n)(t),\frac{df_{\rm p}^{(n)}}{dt}=-\gamma^{(n)}[f_{\rm p}^{(n)}-f_{\rm em}^{(n)}(t)]+\dot{f}_{\rm em}^{(n)}(t)+\xi^{(n)}(t)\ , (1)

where fem(n)f_{\rm em}^{(n)} is the deterministic component of the evolution, an overdot denotes a derivative with respect to tt, γ(n)\gamma^{(n)} is a damping constant whose reciprocal specifies the mean-reversion timescale, and ξ(n)(t)\xi^{(n)}(t) is a white noise stochastic process which satisfies

ξ(n)(t)\displaystyle\langle\xi^{(n)}(t)\rangle =0,\displaystyle=0\ , (2)
ξ(n)(t)ξ(n)(t)\displaystyle\langle\xi^{(n)}(t)\xi^{(n^{\prime})}(t^{\prime})\rangle =[σ(n)]2δn,nδ(tt).\displaystyle=[\sigma^{(n)}]^{2}\delta_{n,n^{\prime}}\delta(t-t^{\prime})\ . (3)

In Equation (3), [σ(n)]2[\sigma^{(n)}]^{2} parametrizes the noise amplitude and produces characteristic root mean square fluctuations σ(n)/[γ(n)]1/2\approx\sigma^{(n)}/[\gamma^{(n)}]^{1/2} in fp(n)(t)f_{\rm p}^{(n)}(t) (Gardiner, 2009). The deterministic evolution fem(n)(t)f_{\rm em}^{(n)}(t) is attributed to magnetic dipole braking for the sake of definiteness, with braking index nem=3n_{\rm em}=3 (Goldreich & Julian, 1969). PTAs are typically composed of millisecond pulsars (MSPs), for which the quadratic correction due to nemn_{\rm em} in fp(n)(t)f_{\rm p}^{(n)}(t) is negligible over the observation time Tobs10yrT_{\rm obs}\sim 10\,{\rm yr}. Consequently, fem(n)(t)f_{\rm em}^{(n)}(t) can be approximated accurately by

fem(n)(t)=fem(n)(t1)+f˙em(n)(t1)(tt1),f_{\rm em}^{(n)}(t)=f_{\rm em}^{(n)}(t_{1})+\dot{f}_{\rm em}^{(n)}(t_{1})(t-t_{1})\ , (4)

where t1t_{1} labels the first TOA.

Equations (1)–(4) are not unique. Rather, they offer one possible phenomenological description consistent with the main qualitative features of a typical PTA pulsar’s observed spin evolution, i.e. random, mean-reverting, small-amplitude excursions around a smooth, secular trend (Agazie et al., 2023b; Antoniadis et al., 2023b; Zic et al., 2023). A phenomenological approach is obligatory, because a predictive, first-principles theory of timing noise does not currently exist, c.f. the multiple theorized mechanisms referenced in Section 1 of K24. Langevin equations like (1)–(4) have been applied successfully to analyse anomalous braking indices (Vargas & Melatos, 2023) and in hidden Markov model glitch searches (Melatos et al., 2020; Lower et al., 2021; Dunn et al., 2022; Dunn et al., 2023). However, they are highly idealised (Meyers et al., 2021b; Meyers et al., 2021a; Antonelli et al., 2023; Vargas & Melatos, 2023). Idealisations include (i) the exclusion of physics, which is likely to be present in reality, e.g. the classic, two-component, crust-superfluid structure inferred from post-glitch recoveries (Baym et al., 1969; van Eysden & Melatos, 2010; Gügercinoǧlu & Alpar, 2017; Meyers et al., 2021a; Meyers et al., 2021b); (ii) the exclusion of non-Gaussian excursions such as Lévy flights (Sornette, 2004); (iii) the whiteness of ξ(n)(t)\xi^{(n)}(t) for MSPs in PTAs; and (iv) the formal interpretation of d2fp/dt2d^{2}f_{\rm p}/dt^{2}, noting that ξ(n)(t)\xi^{(n)}(t) in Equation (1) is not differentiable.

2.2 Modulation of pulsar frequency by a GW

In the presence of a GW, the rest-frame spin frequency of the nn-th pulsar is related to the radio pulse frequency measured by an observer on Earth via a measurement equation,

fm(n)(t)=fp(n)[td(n)]g(n)(t)+ε(n)(t),f_{\rm m}^{(n)}(t)=f_{\rm p}^{(n)}\left[t-d^{(n)}\right]g^{(n)}(t)+\varepsilon^{(n)}(t)\ , (5)

where d(n)d^{(n)} labels the distance to the nn-th pulsar, fp(n)f_{\rm p}^{(n)} is evaluated at the retarded time td(n)t-d^{(n)}, and ε(n)(t)\varepsilon^{(n)}(t) is a Gaussian measurement noise which satisfies

ε(n)(t)\displaystyle\langle\varepsilon^{(n)}(t)\rangle =0,\displaystyle=0\ , (6)
ε(n)(t)ε(n)(t)\displaystyle\langle\varepsilon^{(n)}(t)\varepsilon^{(n^{\prime})}(t^{\prime})\rangle =[σm(n)]2δn,nδ(tt).\displaystyle=\left[\sigma_{\rm m}^{(n)}\right]^{2}\delta_{n,n^{\prime}}\delta(t-t^{\prime})\ . (7)

In Equation (7), [σm(n)]2[\sigma_{\rm m}^{(n)}]^{2} is the variance of the measurement noise at the telescope. In Equation (5) the measurement function g(n)(t)g^{(n)}(t) is given by (e.g. Maggiore, 2018)

g(n)(t)=\displaystyle g^{(n)}(t)= 1Hij[q(n)]i[q(n)]j2[1+𝒏𝒒(n)]\displaystyle 1-\frac{H_{ij}[q^{(n)}]^{i}[q^{(n)}]^{j}}{2[1+\bm{n}\cdot\bm{q}^{(n)}]}
×[cos(Ωt+Φ0)\displaystyle\times\Big{[}\cos\left(-\Omega t+\Phi_{0}\right)
cos{Ωt+Φ0+Ω[1+𝒏𝒒(n)]d(n)}],\displaystyle-\cos\left\{-\Omega t+\Phi_{0}+\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)}\right\}\Big{]}\ , (8)

where [q(n)]i[q^{(n)}]^{i} labels the ii-th coordinate component of the nn-th pulsar’s position vector 𝒒(n)\bm{q}^{(n)}, Ω\Omega is the constant angular frequency of the GW, 𝒏\bm{n} is a unit vector specifying the direction of propagation of the GW, HijH_{ij} is the spatial part of the GW amplitude tensor, and Φ0\Phi_{0} is the phase offset of the GW with respect to some reference time.

Equation (8) assumes that (i) 𝒒(n)\bm{q}^{(n)} is constant; and (ii) the GW is a monochromatic plane wave.

Regarding point (i) a pulsar’s sky position varies due to its own orbital motion (if it is located in a binary) as well as the rotation and revolution of the Earth. However, pulsar TOAs are defined relative to the Solar System barycentre, after correcting for the pulsar’s binary motion (if any). The barycentering correction is typically applied when generating TOAs, e.g. with tempo2 (Hobbs et al., 2006; Edwards et al., 2006) and related timing software, and is inherited by fm(n)(t)f_{\rm m}^{(n)}(t). Some PTA pulsars do have non-negligible proper motions of order 10210^{2} km s-1 after barycentering (e.g. Jankowski et al., 2018), but we do not consider this effect in this introductory paper.

Regarding point (ii), there are two timescales to consider. The first is the timescale set by the observation period, TobsT_{\rm obs}. The second is the timescale set by the light travel time between the pulsar and the Earth, Tlight(n)=d(n)/cT_{\rm light}^{(n)}=d^{(n)}/c. Regarding the former, studies of SMBHB inspirals in the PTA context show that the GW frequency fgwf_{\rm gw} (=Ω/2π=\Omega/2\pi) evolves over a short time-scale by an amount (e.g. Sesana & Vecchio, 2010)

Δfgw=0.05nHz(Mc108.5M)5/3[fgw(t=t1)50nHz]11/3(Tobs10yr),\Delta f_{\rm gw}=0.05\,\mathrm{nHz}\left(\frac{M_{\rm c}}{10^{8.5}M_{\odot}}\right)^{5/3}\left[\frac{f_{\rm gw}(t=t_{1})}{50\mathrm{~{}nHz}}\right]^{11/3}\left(\frac{T_{\mathrm{obs}}}{10\mathrm{yr}}\right)\ , (9)

where McM_{\rm c} is the chirp mass of the SMBHB, fgw(t=t1)f_{\rm gw}(t=t_{1}) is the GW frequency at the time of the first observation, and one has typically Tobs10T_{\rm obs}\sim 10 years. A source can be considered monochromatic if Δfgw\Delta f_{\rm gw} is less than the PTA frequency resolution 1/Tobs1/T_{\rm obs} 111Strictly speaking, the frequency resolution is proportional to 1/Tobs1/T_{\rm obs} divided by the signal-to-noise ratio; brighter sources are resolved more accurately.. A vast majority of SMBHBs resolved by PTAs are expected to satisfy Δfgw<1/Tobs\Delta f_{\rm gw}<1/T_{\rm obs} and we are therefore justified in treating the GW source as monochromatic over the TobsT_{\rm obs} timescale as a first approximation (Sesana et al., 2008; Sesana & Vecchio, 2010; Ellis et al., 2012).

Regarding the light travel time, for SMBHBs which are sufficiently massive and have sufficiently high orbital frequencies, the source may undergo non-negligible evolution during Tlight(n)T_{\rm light}^{(n)}, such that the frequency of the GW which is incident upon the Earth does not equal the frequency of the GW which is incident upon the pulsar. Most SMBHBs detectable with PTAs are not expected to satisfy this condition, but some do; for a PTA composed of pulsars with a mean distance of 1.5 kpc, 78% of simulated SMBHBs detectable with the current IPTA undergo negligible evolution, whilst for the second phase of the Square Kilometre Array this fraction drops to 52%; see Figure 7 in Rosado et al. (2015). In this introductory paper, as a first pass, we focus exclusively on sources which undergo negligible evolution during Tlight(n)T_{\rm light}^{(n)}, following previous investigations of pulsar-term biases in the context of standard PTA analyses (e.g. Zhu et al., 2016; Chen & Wang, 2022). However, the issue is an important one, so we perform some preliminary tests regarding how sensitively the results depend on the monochromatic assumption in Appendix A. We find that the results do not depend sensitively on the assumption that the GW source has a constant angular frequency, especially for systems with low signal-to-noise ratio (SNR). More extensive testing is deferred to future work, after the state-space method matures sufficiently (e.g. by tracking pulsar phase) to be applied to real, astronomical data.

2.3 Static parameters

The model described in Sections 2.1 and 2.2 comprises 5N5N static parameters, that are specific to the pulsars in the array, viz.

𝜽psr={γ(n),σ(n),fem(n)(t1),f˙em(n)(t1),d(n)}1nN.\bm{\theta}_{\rm psr}=\left\{\gamma^{(n)},\sigma^{(n)},f_{\rm em}^{(n)}(t_{1}),\dot{f}_{\rm em}^{(n)}(t_{1}),d^{(n)}\right\}_{1\leq n\leq N}\ . (10)

It also comprises seven parameters, that are specific to the GW source, viz.

𝜽gw={h0,ι,ψ,δ,α,Ω,Φ0},\bm{\theta}_{\rm gw}=\left\{h_{0},\iota,\psi,\delta,\alpha,\Omega,\Phi_{0}\right\}\ , (11)

where h0h_{0} is the characteristic wave strain, ι\iota is the orbital inclination, ψ\psi is the polarisation angle, δ\delta is the declination and α\alpha is the right ascension. These parameters enter the model through Equation (8), with Hij=Hij(h0,ι,ψ,δ,α)H_{ij}=H_{ij}(h_{0},\iota,\psi,\delta,\alpha) and 𝒏=𝒏(δ,α)\bm{n}=\bm{n}(\delta,\alpha). The complete set of 7+5N7+5N static parameters is denoted by 𝜽=𝜽gw𝜽psr\bm{\theta}=\bm{\theta}_{\rm gw}\cup\bm{\theta}_{\rm psr}. While we assume no prior information about 𝜽gw\bm{\theta}_{\rm gw}, there are constraints on 𝜽psr\bm{\theta}_{\rm psr} from electromagnetic observations; for example estimates of d(n)d^{(n)} are accurate to \sim 10%\% typically (Cordes & Lazio, 2002; Verbiest et al., 2012; Desvignes et al., 2016; Yao et al., 2017).

3 Inference with the pulsar terms

K24 developed a likelihood-based Bayesian framework to infer the static parameters in Section 2.3 and select between models with and without a GW. Given a temporal sequence of noisy measurements 𝒀(t)\bm{Y}(t), the posterior distribution of 𝜽\bm{\theta} is calculated by Bayes’ Rule,

p(𝜽|𝒀)=(𝒀|𝜽)π(𝜽)𝒵,p(\bm{\theta}|\bm{Y})=\frac{\mathcal{L}(\bm{Y}|\bm{\theta})\pi(\bm{\theta})}{\mathcal{Z}}\ , (12)

where (𝒀|𝜽)\mathcal{L}(\bm{Y}|\bm{\theta}) is the likelihood calculated with a Kalman filter (Kalman, 1960), as discussed in Appendix B, π(𝜽)\pi(\bm{\theta}) is the prior distribution of 𝜽\bm{\theta}, and 𝒵\mathcal{Z} is the marginalised likelihood or evidence,

𝒵=𝑑𝜽(𝒀|𝜽)π(𝜽).\mathcal{Z}=\int d\bm{\theta}\mathcal{L}(\bm{Y}|\bm{\theta})\pi(\bm{\theta})\ . (13)

The measurements are 𝒀={fm(n)(t1),,fm(n)(Tobs)}1nN{\bm{Y}}=\{f^{(n)}_{\rm m}(t_{1}),\dots,f^{(n)}_{\rm m}(T_{\rm obs})\}_{1\leq n\leq N}. Given a specific realisation of 𝒀\bm{Y}, (𝒀|𝜽)\mathcal{L}(\bm{Y}|\bm{\theta}) is a function of 𝜽\bm{\theta}. Appendix B explains how to compute (𝒀|𝜽){\mathcal{L}}({\bm{Y}}|{\bm{\theta}}) from fm(n)(t)f_{\rm m}^{(n)}(t) by discretizing the dynamical equations (1)–(4) and the measurement equations (5)–(8) and solving them recursively using a Kalman filter. Nested sampling (Skilling, 2006) is used to estimate p(𝜽|𝒀)p(\bm{\theta}|\bm{Y}) and 𝒵\mathcal{Z} given (𝒀|𝜽){\mathcal{L}}({\bm{Y}}|{\bm{\theta}}). Appendix C reviews nested sampling briefly as applied in this paper. Appendix D summarizes the workflow of the combined Kalman filter and nested sampler. Appendices BD are designed to equip the interested reader to reproduce the key results in Sections 5 and 6.

The inference procedure in K24 drops the pulsar term on the last line of Equation (8), in keeping with several other PTA analyses (e.g. Sesana & Vecchio, 2010; Babak & Sesana, 2012a; Petiteau et al., 2013; Zhu et al., 2015; Taylor et al., 2016; Goldstein et al., 2018; Charisi et al., 2023). In this section we show how to remove this limitation, following previous authors (Zhu et al., 2016; Chen & Wang, 2022). In Section 3.1 we review briefly how the pulsar terms are dropped from typical PTA analyses as well as K24. In Section 3.2 we introduce a convenient reparametrisation of the pulsar terms for likelihood-based inference methods such as nested sampling. The new parametrisation is validated with synthetic data in Section 4.

3.1 Earth term

The GW modulates radio pulses according to Equation (8). The modulation is composed of an Earth term, proportional to cos(Ωt+Φ0)\cos(-\Omega t+\Phi_{0}), and a pulsar term, proportional to cos{Ωt+Φ0+Ω[1+𝒏𝒒(n)]d(n)}\cos\left\{-\Omega t+\Phi_{0}+\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)}\right\}. The Earth term describes the GW phase at the observer on Earth, while the pulsar term describes the GW phase at the nn-th pulsar. The Earth term depends only on the GW source parameters and is common across all pulsars. In contrast the pulsar term is a function of d(n)d^{(n)} and 𝒒(n)\bm{q}^{(n)} and varies between pulsars.

The phases of the pulsar terms are related unpredictably, which is why some published PTA analyses approximate the pulsar terms collectively as a source of self-noise. The pulsar term depends on d(n)d^{(n)} which is generally poorly constrained by electromagnetic observations, with uncertainties greater than the typical GW wavelength. Consequently the pulsar term is often — although not always — dropped in standard PTA analyses (e.g. Sesana & Vecchio, 2010; Babak & Sesana, 2012a; Petiteau et al., 2013; Zhu et al., 2015; Taylor et al., 2016; Goldstein et al., 2018; Charisi et al., 2023). Dropping the pulsar term is also convenient computationally because it reduces the dimensionality of the parameter space; the NN values of d(n)d^{(n)} are not inferred. Within the state-space model described in Section 2, dropping the pulsar terms equates to using a modified measurement equation,

fm(n)(t)=fp(n)[td(n)]gEarth(n)(t)+ε(n)(t),f_{\rm m}^{(n)}(t)=f_{\rm p}^{(n)}\left[t-d^{(n)}\right]g^{(n)}_{\rm Earth}(t)+\varepsilon^{(n)}(t)\ , (14)

with

gEarth(n)(t)=1Hij[q(n)]i[q(n)]j2[1+𝒏𝒒(n)]cos(Ωt+Φ0).g^{(n)}_{\rm Earth}(t)=1-\frac{H_{ij}[q^{(n)}]^{i}[q^{(n)}]^{j}}{2[1+\bm{n}\cdot\bm{q}^{(n)}]}\cos(-\Omega t+\Phi_{0})\ . (15)

However, dropping the pulsar term leads to well-known parameter estimation biases, especially in α\alpha and δ\delta, and reduces the detection probability by 5%\sim 5\% (Zhu et al., 2016; Chen & Wang, 2022; Kimpson et al., 2024).

3.2 Reparametrisation of the pulsar terms

In this paper, we generalize the Kalman filter analysis in K24 to include the pulsar terms. To do so, we define a new parameter

χ(n)={Ω[1+𝒏𝒒(n)]d(n)}mod2π,\chi^{(n)}=\left\{\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)}\right\}\mod 2\pi\ , (16)

such that Equation (8) becomes

g(n)(t)=\displaystyle g^{(n)}(t)= 1Hij[q(n)]i[q(n)]j2[1+𝒏𝒒(n)]\displaystyle 1-\frac{H_{ij}[q^{(n)}]^{i}[q^{(n)}]^{j}}{2[1+\bm{n}\cdot\bm{q}^{(n)}]}
×{cos(Ωt+Φ0)\displaystyle\times\Big{\{}\cos\left(-\Omega t+\Phi_{0}\right)
cos[Ωt+Φ0+χ(n)]}.\displaystyle-\cos\left[-\Omega t+\Phi_{0}+\chi^{(n)}\right]\Big{\}}\ . (17)

The reparametrisation in terms of χ(n)\chi^{(n)} treats the pulsar-dependent phase correction Ω[1+𝒏𝒒(n)]d(n)\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)} in the argument of the cosine of the pulsar term as a composite parameter to be inferred for each pulsar. In principle, it is possible to disentangle Ω\Omega, 𝒏\bm{n}, and d(n)d^{(n)} and infer them individually, if NN is large enough; for example, 𝒏\bm{n} appears independently in the phase of the pulsar term and in the denominator of the first line of Equation (8). In practice, however, the likelihood surface is highly corrugated along the d(n)d^{(n)}-axis, and the prior on d(n)d^{(n)} is broad compared to the wavelength 2π/Ω2\pi/\Omega, so d(n)d^{(n)} and hence Ω\Omega are not identifiable for reasonable N102N\lesssim 10^{2}. By replacing d(n)d^{(n)} with χ(n)\chi^{(n)}, we obtain a smooth likelihood function, whose maximum is located efficiently and accurately by the nested sampler. This point is discussed in more detail in Appendix E.

The new parametrisation does not increase the dimension of the parameter space, because we trade d(n)d^{(n)} for χ(n)\chi^{(n)}. Once Ω\Omega, 𝒏{\bm{n}}, and χ(n)\chi^{(n)} are inferred, it is possible to solve for d(n)d^{(n)}, albeit not uniquely; d(n)d^{(n)} can be inferred up to an integer multiple of the Doppler-shifted wavelength, due to the mod 2π{\rm mod}\,2\pi operation in Equation (16). However, this ambiguity is unavoidable, whether we replace d(n)d^{(n)} with χ(n)\chi^{(n)} or not, and occurs in every PTA analysis. The static parameters specific to the pulsars in the array (c.f. Equation (10)) are now

𝜽psr={γ(n),σ(n),fem(n)(t1),f˙em(n)(t1),χ(n)}1nN,\bm{\theta^{\prime}}_{\rm psr}=\left\{\gamma^{(n)},\sigma^{(n)},f_{\rm em}^{(n)}(t_{1}),\dot{f}_{\rm em}^{(n)}(t_{1}),\chi^{(n)}\right\}_{1\leq n\leq N}\ , (18)

whilst 𝜽gw\bm{\theta}_{\rm gw} remains unchanged. We define the complete set of 7+5N7+5N static parameters to be inferred as 𝜽=𝜽gw𝜽psr\bm{\theta^{\prime}}=\bm{\theta}_{\rm gw}\cup\bm{\theta^{\prime}}_{\rm psr}.

4 Validation with synthetic data

In the rest of this paper, we test the performance of the PTA analysis scheme in K24, once it is generalized to include the pulsar terms in the inference model as described in Section 3. We refer the reader to the appendices for detailed instructions about how to implement the Kalman filter (Appendix B) and nested sampler (Appendix C) and integrate them in a unified workflow (Appendix D). In this section, by way of preparation, we explain how to generate the synthetic data employed in the tests, namely noisy frequency time series fm(n)(t)f_{\rm m}^{(n)}(t) for 1nN1\leq n\leq N. Tests are performed for multiple random realizations of ξ(n)(t)\xi^{(n)}(t) in order to quantify the irreducible “cosmic” variance in the inference output (e.g. estimates of 𝜽gw{\bm{\theta}}_{\rm gw}). Every real PTA analysis witnesses a unique realization of ξ(n)(t)\xi^{(n)}(t) — the actual, astronomical one — but there is no way to determine where this realization lies within the admissible statistical ensemble.

Set Parameter Injected value Units Prior
𝜽gw\bm{\theta}_{\rm gw} Ω\Omega 5×1075\times 10^{-7} Hz LogUniform(10910^{-9}, 10510^{-5})
α\alpha 1.01.0 rad Uniform(0,2π0,2\pi)
δ\delta 1.01.0 rad Cosine(π/2,π/2-\pi/2,\pi/2)
ψ\psi 2.502.50 rad Uniform(0,2π0,2\pi)
Φ0\Phi_{0} 0.200.20 rad Uniform(0,2π0,2\pi)
h0h_{0} 5×10155\times 10^{-15} LogUniform(101510^{-15}, 10910^{-9})
ι\iota 1.01.0 rad Sin(0,π0,\pi)
fem(n)(t1)f_{\rm em}^{(n)}(t_{1}) fATNF(n)f_{\rm ATNF}^{(n)} Hz Uniform[fATNF(n)103ηf(n),fATNF(n)+103ηf(n)]\left[f_{\rm ATNF}^{(n)}-10^{3}\eta^{(n)}_{f},f_{\rm ATNF}^{(n)}+10^{3}\eta^{(n)}_{f}\right]
𝜽psr\bm{\theta^{\prime}}_{\rm psr} f˙em(n)(t1)\dot{f}_{\rm em}^{(n)}(t_{1}) f˙ATNF(n)\dot{f}_{\rm ATNF}^{(n)} s-2 Uniform[f˙ATNF(n)103ηf˙(n),f˙ATNF(n)+103ηf˙(n)]\left[\dot{f}_{\rm ATNF}^{(n)}-10^{3}\eta^{(n)}_{\dot{f}},\dot{f}_{\rm ATNF}^{(n)}+10^{3}\eta^{(n)}_{\dot{f}}\right]
σ(n)\sigma^{(n)} σSC(n)\sigma_{\rm SC}^{(n)} s3/2s^{-3/2} LogUniform[102σSC(n),102σSC(n)]\left[10^{-2}\sigma_{\rm SC}^{(n)},10^{2}\sigma_{\rm SC}^{(n)}\right]
γ(n)\gamma^{(n)} 101310^{-13} s-1
χ(n)\chi^{(n)} Ω[1+𝒏𝒒ATNF(n)]dATNF(n)\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}_{\rm ATNF}\right]d_{\rm ATNF}^{(n)} rad Uniform(0,2π0,2\pi)
Table 1: Injected static parameters used to generate synthetic data to validate the analysis scheme including the pulsar terms in Equation (17). The prior used for Bayesian inference is also displayed (rightmost column). The top and bottom sections of the table contain 𝜽gw\bm{\theta}_{\rm gw} and 𝜽psr\bm{\theta^{\prime}}_{\rm psr} respectively. The subscript “ATNF” denotes values obtained from the ATNF pulsar catalogue as described in Section 4. The subscript “SC” on σ(n)\sigma^{(n)} indicates that the injected value is calculated from Equation (19) and the empirical timing noise model for MSPs in Shannon & Cordes (2010a). The quantities ηf(n)\eta^{(n)}_{f} and ηf˙(n)\eta^{(n)}_{\dot{f}} are the uncertainties in fem(n)(t1)f^{(n)}_{\rm em}(t_{1}) and f˙em(n)(t1)\dot{f}^{(n)}_{\rm em}(t_{1}) respectively, as quoted in the ATNF catalogue. We do not infer γ(n)105Tobs\gamma^{(n)}\sim 10^{-5}T_{\rm obs} for simplicity, so no prior is set. The priors on 𝜽psr\bm{\theta^{\prime}}_{\rm psr} are justified in Appendix F.

In order to synthesize 𝒀={fm(1)(t),,fm(N)(t)}\bm{Y}=\{f_{\rm m}^{(1)}(t),\dots,f_{\rm m}^{(N)}(t)\}, we integrate Equations (1)–(4) numerically using a Runge-Kutta Ito^\hat{\text{o}} integrator implemented in the sdeint python package 222https://github.com/mattja/sdeint. This produces random realizations of fp(n)(t)f_{\rm p}^{(n)}(t) for 1nN1\leq n\leq N, which we convert to fm(n)(t)f_{\rm m}^{(n)}(t) via Equations (5)–(8). The numerical solutions depend on how we choose 𝜽psr\bm{\theta}_{\rm psr}, 𝒒(n){\bm{q}}^{(n)} and σm\sigma_{\rm m} or, equivalently, how we specify the configuration of a synthetic PTA. In Section 5 we describe how we choose the remaining elements of 𝜽\bm{\theta}, namely 𝜽gw\bm{\theta}_{\rm gw}. This latter step is equivalent to specifying the synthetic SMBHB source and differs from one test to the next according to the goal of the test.

In this paper we adopt for consistency the same 𝜽psr\bm{\theta}_{\rm psr} values as in K24, i.e. the N=47N=47 MSPs in the 12.5-year NANOGrav dataset (Arzoumanian et al., 2020). We assume all pulsars are observed with cadence Tcad=1weekT_{\rm cad}=1\,{\rm week} over a 10 year period. Fiducial values for 𝒒(n){\bm{q}}^{(n)}, d(n)d^{(n)}, fem(n)(t1)f_{\rm em}^{(n)}(t_{1}), and f˙em(n)(t1)\dot{f}^{(n)}_{\rm em}(t_{1}) are read from the Australia Telescope National Facility (ATNF) pulsar catalogue (Manchester et al., 2005) using the psrqpy package (Pitkin, 2018). No direct measurements exist for γ(n)\gamma^{(n)}. The mean reversion timescale typically satisfies [γ(n)]1Tobs[\gamma^{(n)}]^{-1}\gg T_{\rm obs} (Price et al., 2012; Meyers et al., 2021a; Meyers et al., 2021b; Vargas & Melatos, 2023); in this paper, for the sake of simplicity, we fix γ(n)=1013\gamma^{(n)}=10^{-13} s-1 for all nn. No direct measurements exist for σ(n)\sigma^{(n)} either. We relate σ(n)\sigma^{(n)} to the root mean square TOA noise σTOA(n)\sigma^{(n)}_{\rm TOA} accumulated over an interval of length TcadT_{\rm cad} by

σ(n)σTOA(n)fp(n)(t1)Tcad3/2.\displaystyle\sigma^{(n)}\approx\sigma_{\rm TOA}^{(n)}f_{\rm p}^{(n)}(t_{1}){T_{\rm cad}}^{-3/2}\ . (19)

As in K24, the empirical timing noise model for MSPs from Shannon & Cordes (2010b), applied to the 12.5-year NANOGrav dataset, implies median[σ(n)]=5.51×1024\text{median}[\sigma^{(n)}]=5.51\times 10^{-24} s-3/2, min[σ(n)]=1.67×1026\min[\sigma^{(n)}]=1.67\times 10^{-26}s-3/2 for PSR J0645+5158 and max[σ(n)]=2.56×1019\max[\sigma^{(n)}]=2.56\times 10^{-19} s-3/2 for PSR J1939+2134.

In a similar vein, σm(n)\sigma_{\rm m}^{(n)} can be related to, σTOA(n)\sigma_{\rm TOA}^{(n)}, by

σm(n)fp(n)(t1)σTOA(n)Tcad1.\sigma_{\rm m}^{(n)}\approx f_{\rm p}^{(n)}(t_{1})\sigma_{\rm TOA}^{(n)}\ {T_{\rm cad}}^{-1}\ . (20)

For an MSP with fp(n)0.1f_{\rm p}^{(n)}\sim 0.1 kHz, Tcad=1weekT_{\rm cad}=1\,{\rm week}, and σTOA(n)1μ\sigma_{\rm TOA}^{(n)}\sim 1\mus, Equation (20) implies σm(n)1010\sigma_{\rm m}^{(n)}\sim 10^{-10} Hz. The most accurately timed pulsars have σTOA(n)10\sigma_{\rm TOA}^{(n)}\sim 10 ns and σm(n)1012\sigma_{\rm m}^{(n)}\sim 10^{-12} Hz. In this paper, for simplicity and the sake of definiteness, we fix σm(n)=1011\sigma_{\rm m}^{(n)}=10^{-11} Hz for all nn, and take it as known a priori rather than a parameter to be inferred. When analysing real data this assumption is easily relaxed. Although σm(n)\sigma_{\rm m}^{(n)} is assumed to be the same for every pulsar, fm(n)f_{\rm m}^{(n)} is constructed from a different random realisation of ε(n)(t)\varepsilon^{(n)}(t) for each pulsar.

5 Parameter estimation

In this section, we apply the Kalman filter and nested sampler to calculate the joint posterior probability distribution p(𝜽|𝒀)p({\bm{\theta^{\prime}}}|{\bm{Y}}) and compare it to the known, injected values. The procedure is undertaken for multiple realisations of ξ(n)(t)\xi^{(n)}(t) and ε(n)(t)\varepsilon^{(n)}(t), and we estimate 𝜽\bm{\theta^{\prime}} independently for each realisation. The aims are (i) to demonstrate that the analysis scheme works, i.e. that it converges to a well-behaved, unimodal posterior for multiple noise realisations; (ii) to give a preliminary sense of its accuracy; and (iii) to quantify the natural random dispersion in the one-dimensional posterior medians. Quantifying the dispersion is important since it is a practical measure of the scheme’s accuracy when it is applied to real astronomical data, where the true parameter values and specific noise realisations are unknown (i.e. cosmic variance). The injected 𝜽gw\bm{\theta}_{\rm gw} values are selected to be astrophysically representative, as per the top section of Table 1. The static pulsar parameters 𝜽psr\bm{\theta^{\prime}}_{\rm psr} are specified in Section 4. All injected static parameters 𝜽\bm{\theta}^{\prime} are summarised in the second column of Table 1. The specification of the priors on 𝜽\bm{\theta^{\prime}} is described in Appendix F and summarised in the rightmost column of Table 1.

In Section 5.1 we apply the analysis scheme to synthetic data and start by estimating 𝜽gw\bm{\theta}_{\rm gw} for a particular, arbitrary, representative choice of 𝜽gw\bm{\theta}_{\rm gw}, i.e. the SMBHB system. The scheme is validated on multiple noise realisations of the pulsar process noise ξ(n)(t)\xi^{(n)}(t) and the detector measurement noise ε(n)(t)\varepsilon^{(n)}(t) in order to test the scheme multiple times and quantify the irreducible cosmic variance in the parameter estimates. In Section 5.2 we inspect and verify the estimates of χ(n)\chi^{(n)}. In Section 5.3 we briefly discuss the estimates of the remaining 4N4N parameters in 𝜽psr\bm{\theta^{\prime}}_{\rm psr}. In Section 5.4 we extend the tests across a broader parameter domain and consider a range of astrophysically relevant SMBHB source parameters 𝜽gw\bm{\theta}_{\rm gw}.

5.1 Representative SMBHB source

Figure 1 displays the posterior distribution of 𝜽gw\bm{\theta}_{\rm gw} for ten noise realisations in the form of a traditional corner plot for the representative SMBHB source parametrized in the top portion of Table 1. The histograms are one-dimensional posteriors, marginalized over the six other parameters. Each coloured curve corresponds to a different noise realisation. The solid orange line marks the known injected value. The two-dimensional contours mark the (0.5, 1, 1.5, 2)-sigma level surfaces. All histograms and contours are consistent with a unimodal joint posterior, which peaks near the known, injected values. There is scant evidence of railing against the prior bounds. There is no strong evidence for correlations between parameter pairs, e.g. banana-shaped contours, although weak correlations are evident between Ω\Omega and Φ0\Phi_{0}, ψ\psi and α\alpha, and ι\iota and h0h_{0}. The injected value falls within the 90% credible interval of the one-dimensional marginalized posteriors in 60 out of the 70 possible combinations of the seven parameters and 10 realizations. Indeed, the injected value falls within the 90% credible interval for Ω,Φ0,ψ\Omega,\Phi_{0},\psi, and ι\iota 40 out of 40 times.

There is appreciable natural dispersion in the one-dimensional posterior medians between realisations. We quantify the degree of dispersion using the coefficient of variation,

CV=μmed/σmed\displaystyle CV=\mu_{\rm med}/\sigma_{\rm med} (21)

where μmed\mu_{\rm med} is the mean of the 10 posterior medians and σmed\sigma_{\rm med} is their standard deviation. The minimum and maximum CVCV are 0.2% and 52% for Ω\Omega and Φ0\Phi_{0} respectively. The remaining parameters typically satisfy CVCV 10%\lesssim 10\%. Analogous results for synthetic data with higher strain (h0=1×1012h_{0}=1\times 10^{-12}) are presented in Appendix G (see Figure 12) for completeness. The dispersion between noise realisations decreases as h0h_{0} increases, and the one-dimensional posteriors for ι\iota and h0h_{0} are more symmetric about the injected value.

5.2 GW phase at each pulsar

It is important to check how well the pulsar-term phase χ(n)\chi^{(n)} is estimated, since the reparameterization involving χ(n)\chi^{(n)} is a key feature of our approach, as explained in Section 3.2. In short, it turns out that χ(n)\chi^{(n)} is estimated accurately for all nn, except under special circumstances. Figure 2 displays the results as a corner plot for the representative subset χ(1),,χ(5)\chi^{(1)},\dots,\chi^{(5)} over the ten noise realisations, in the same style as Figure 1. We do not display the corner plot for χ(6)\chi^{(6)}, \dots, χ(47)\chi^{(47)} because it is too big. With the exception of the results for χ(2)\chi^{(2)} (discussed below), all histograms and contours are consistent with a unimodal joint posterior, which peaks near the known, injected values. There is no evidence for correlations between parameter pairs. Most phases are recovered unambiguously across all noise realisations; for the 47 χ(n)\chi^{(n)} parameters (i.e. not just the five plotted in Figure 2), the injected value is contained within the 90% credible interval of the one-dimensional marginalized posteriors in 383 out of the 470 possible combinations of the 47 parameters and ten realizations. The next paragraph explains why 383 out of 470 is fewer than 90%.

Sometimes, albeit rarely, χ(n)\chi^{(n)} is not estimated consistently across multiple realizations for some nn, e.g χ(3)\chi^{(3)} (third row, third column of Figure 2) corresponding to PSR J0340+4130. Out of all the pulsars in the array, this object matches most closely the direction to the synthetic GW source, with 𝒏𝒒(2)=0.96\bm{n}\cdot\bm{q}^{(2)}=-0.96 and cosχ(2)0.07\cos\chi^{(2)}\approx 0.07. The ability to accurately infer χ(2)\chi^{(2)} improves with the SNR. For example, for h0=1×1012h_{0}=1\times 10^{-12}, χ(2)\chi^{(2)} is recovered unambiguously; see Figure 13 in Appendix G.

5.3 Timing parameters

Similar results are obtained for the 4NN parameters in 𝜽psr\bm{\theta}_{\rm psr}^{\prime} besides χ(n)\chi^{(n)}. The injected values are recovered unambiguously across the 10 noise realisations. For the sake of brevity we do not display the calculated posterior distributions, because inferring 𝜽gw\bm{\theta}_{\rm gw} is the main focus of this paper and most published PTA analyses. In short, the estimates of fem(t1)f_{\rm em}(t_{1}) and f˙em(t1)\dot{f}_{\rm em}(t_{1}) are guided into narrow ranges by the narrow priors. The one-dimensional posteriors inferred for σ(n)\sigma^{(n)} are generally broader due to the broader prior, but contain the injection within the 90% credible interval in a majority of cases. We remind the reader that γ(n)\gamma^{(n)} is not estimated in this paper; typically it satisfies γ(n)105Tobs\gamma^{(n)}\sim 10^{-5}T_{\rm obs} (Price et al., 2012; Meyers et al., 2021a; Meyers et al., 2021b; Vargas & Melatos, 2023), so its influence is muted.

Refer to caption
Figure 1: Posterior distribution of the GW source parameters 𝜽gw\bm{\theta}_{\rm gw} for the representative system in Table 1, for 10 realisations of the noise processes, with curves coloured uniquely per realisation. The horizontal and vertical orange lines indicate the true injected values. The contours in the two-dimensional histograms mark the (0.5, 1, 1.5, 2)-σ\sigma levels after marginalizing over all but two parameters. The one-dimensional histograms correspond to the joint posterior distribution marginalized over all but one parameter. The supertitles of the one-dimensional histograms record the median and the 0.16 and 0.84 quantiles of the median realisation. In this context the median realisation is defined as follows: a posterior is generated for all 10 realisations; the medians of the posteriors are ranked in ascending order; the median of the ranked list is associated with the median realisation. We plot the scaled variables 109Ω10^{9}\Omega (units: rads1{\rm rad\,s^{-1}}) and 1015h010^{15}h_{0}. The Kalman filter and nested sampler estimate accurately all seven parameters in 𝜽gw{\bm{\theta}}_{\rm gw}. The horizontal axes span a subset of the prior domain for all seven parameters. The known, injected value lies within the 90% credible interval for 60 out of the 70 combinations of seven parameters and ten noise realizations. There is appreciable dispersion among the peaks of the one-dimensional posteriors: the maximum coefficient of variation is CV=52%CV=52\% for Φ0\Phi_{0}, and the minimum is CV=0.2%CV=0.2\% for Ω\Omega.
Refer to caption
Figure 2: Same as Figure 1 but for the static parameters χ(1),,χ(5)\chi^{(1)},\dots,\chi^{(5)} (the remaining pulsar phases χ(6),,χ(N)\chi^{(6)},\dots,\chi^{(N)} are omitted for readability). Consistent unimodal posteriors are obtained across the ten noise realisations for four out of the five displayed parameters. No consistent posteriors are obtained for χ(3)\chi^{(3)} (third row, third column), because this pulsar’s sky position is close to the synthetic GW source. The known, injected value lies within the 90% credible interval for 38 out of the 50 combinations of five parameters and ten noise realizations.

5.4 Exploring the SMBHB parameter domain

Refer to caption
(a)
Refer to caption
(b)
Figure 3: Accuracy of SMBHB parameter estimation across an astrophysically plausible domain. (a) Fraction of injections included within a given credible interval of the estimated posterior, as a function of the credible interval itself (i.e. PP plot). The injections are 200 simulated GW sources generated by drawing randomly five parameters in 𝜽gw\bm{\theta}_{\rm gw} from the prior distributions in Table 1. Each coloured curve corresponds to a different parameter (see legend). The parameters h0h_{0} and ι\iota are fixed at 5×10155\times 10^{-15} and 1.0 rad respectively in order to maintain an approximately constant SNR. The grey shaded contours label the 1σ1\sigma, 2σ2\sigma and 3σ\sigma confidence intervals. For parameters with well estimated posteriors, the PP curve should fall along the diagonal of unit slope. Ω\Omega and α\alpha are generally well-estimated (i.e. the curves lie close to the unit diagonal). The remaining three parameters, ψ\psi, ϕ0\phi_{0} and δ\delta show modest evidence of being over-constrained and stray outside the shaded region. (b) Same as (a) but now assuming that χ(1),,χ(47)\chi^{(1)},\dots,\chi^{(47)} are known exactly a priori (i.e. with a delta-function prior). The excursions from the shaded region seen in Figure 3(a) are reduced; nearly all parameters, with the exception of δ\delta, lie wholly within the shaded region. pp-values are shown for each of the parameters.

Section 5.1 focuses on a single representative source, summarised in Table 1. In this section we test the method for a range of sources, varying 𝜽gw\bm{\theta}_{\rm gw}. The aim is to verify that the analysis scheme works across an astrophysically relevant domain and that the arbitrary choice of 𝜽gw\bm{\theta}_{\rm gw} in Table 1 is not advantageous by accident.

We analyse 200 injections constructed by fixing h0=5×1015h_{0}=5\times 10^{-15} and ι=1.0\iota=1.0 rad and drawing the remaining five elements of 𝜽gw\bm{\theta}_{\rm gw} randomly from the prior distributions defined in Table 1. We fix h0h_{0} and ι\iota in order to maintain an approximately constant SNR across the 200 injections. For each injection we compute the posterior distribution of 𝜽gw{\bm{\theta}}_{\rm gw}. The static pulsar parameters 𝜽psr\bm{\theta^{\prime}}_{\rm psr} are specified in Section 4 and Table 1. It is prohibitive to display corner plots analogous to Figure 1 for 200 injections and seven elements of 𝜽gw\bm{\theta}_{\rm gw}. Therefore, we summarise the results with the aid of a parameter-parameter (PP) plot (Cook et al., 2006). A PP plot displays the fraction of injections included within a given credible interval of the estimated posterior, as a function of the credible interval itself. In the ideal case of perfect recovery, the PP plot should be a diagonal line of unit slope.

The PP results are displayed in Figure 3(a). The shaded grey contours enclose the 1σ1\sigma, 2σ2\sigma, and 3σ3\sigma significance levels for 200 injections. The curves for the five parameters, colour-coded in the legend, are approximately linear. Some parameters are consistently well estimated across the parameter domain, e.g. the blue curve for Ω\Omega lies everywhere within the 2σ\sigma shaded region. Conversely some parameters, e.g. Φ0\Phi_{0}, stray outside the 1σ1\sigma shaded region and show evidence of being slightly over-constrained; there are more injections contained within lower-value credible intervals than would be expected statistically, and fewer injections contained within higher-value credible intervals. This occurs due to the low SNR of the injected GW signal and the varying sensitivity of the specific PTA configuration as a function of sky position. More quantitatively, for the best estimated parameter Ω\Omega the injection is contained within the 90% credible interval in 91% of cases. For the worst estimated parameter Φ0\Phi_{0}, the injection is contained within the 90% credible interval in 81% of cases. The next paragraph explains why 81% is less than the theoretical ideal 90%.

Manually inspecting the individual corner plots for each of the 200 injections, we see that the nested sampler does not return a unimodal posterior in the rare event that the source is located unfavourably, with 𝒏𝒒(n)1\bm{n}\cdot\bm{q}^{(n)}\approx-1, such that one cannot infer χ(n)\chi^{(n)} accurately (see Section 5.2). All injected sources are “observed” with the same settings such as TobsT_{\rm obs}, TcadT_{\rm cad} and nliven_{\rm live}. In Figure 3(b) we display a second PP plot, arranged identically to Figure 3(a), but assuming that the injected values of χ(n)\chi^{(n)} are known exactly for the sake of testing, i.e. setting a delta-function prior on χ(n)\chi^{(n)}. The excursions out of the shaded region diminish compared to Figure 3(a), confirming the importance of estimating χ(n)\chi^{(n)} accurately. For the best estimated parameter Ω\Omega the injection is contained within the 90% credible interval in 91% of cases, the same as in Figure 3(a). For the worst estimated parameters such as Φ0\Phi_{0}, the injection is contained within the 90% credible interval in 89% of cases, an improvement over the results of Figure 3(a).

6 Bias when neglecting the pulsar terms

In Section 4 we validate the state-space analysis scheme when the pulsar terms are included in the inference model, i.e. Equation (17). In this section, we compare what happens when we omit the pulsar terms intentionally by using Equation (15). We do this for the specific representative case of the individual quasi-monochromatic SMBHB source described in Table 1. We refer to the model where the pulsar terms are included alongside the Earth term as psr&Earth\mathcal{M}_{\rm psr\&Earth}. We refer to the model where the pulsar terms are omitted as Earth\mathcal{M}_{\rm Earth}. Model psr&Earth\mathcal{M}_{\rm psr\&Earth} is parameterised by 𝜽\bm{\theta}^{\prime}. Model Earth\mathcal{M}_{\rm Earth} is parameterised by 𝜽Earth=𝜽gw𝜽psr′′\bm{\theta}_{\rm Earth}=\bm{\theta}_{\rm gw}\cup\bm{\theta}^{\prime\prime}_{\rm psr} where 𝜽psr′′\bm{\theta}^{\prime\prime}_{\rm psr} equals 𝜽psr\bm{\theta}^{\prime}_{\rm psr} with χ(n)\chi^{(n)} removed. Both models are applied to identical realisations of the data, generated using the procedure in Section 4. The priors on the static parameters in each model are also identical, with the addition of a uniform prior on χ(n)\chi^{(n)} for psr&Earth\mathcal{M}_{\rm psr\&Earth}, as described in Appendix F.

In Section 6.1 we compare the accuracy of the estimates of 𝜽\bm{\theta^{\prime}} and 𝜽Earth\bm{\theta}_{\rm Earth} returned by the respective models. In Section 6.2 we compare the minimum detectable GW strain for the two models by calculating the model evidence and comparing it to the evidence for a null model that does not contain a GW.

6.1 Accuracy of parameter estimation

In this section we apply the Kalman filter in conjunction with nested sampling in order to infer the joint posterior distribution for the static parameters with and without the pulsar terms. For Earth\mathcal{M}_{\rm Earth} we apply the Kalman filter using Equation (15) to return (𝒀|𝜽Earth)\mathcal{L}(\bm{Y}|\bm{\theta}_{\rm Earth}) and the nested sampler to estimate p(𝜽Earth|𝒀)p(\bm{\theta}_{\rm Earth}|\bm{Y}). For psr&Earth\mathcal{M}_{\rm psr\&Earth} we apply the Kalman filter using Equation (17) to return (𝒀|𝜽)\mathcal{L}(\bm{Y}|\bm{\theta}^{\prime}) and the nested sampler to estimate p(𝜽|𝒀)p(\bm{\theta^{\prime}}|\bm{Y}). The settings of the nested sampler (for example the number of live points; see Appendix C) are identical for both models.

We consider two representative systems. The first is a “low-SNR” system with h0=5×1015h_{0}=5\times 10^{-15}, i.e. the system described in Table 1. The second is a “high-SNR” system which has the same static parameters as in Table 1 except with h0=1×1012h_{0}=1\times 10^{-12}. The “high-SNR” system is considered in order to quantify any systematic biases, as distinct from random errors caused by the measurement noise. In order to enable a clear comparison between the posterior probability distributions calculated using the two models, in this section we present a single noise realisation of the synthetic data 𝒀\bm{Y}. The results quoted are consistent across different noise realisations.

The results for the seven parameters in 𝜽gw\bm{\theta}_{\rm gw} are shown in Figure 4 for the high-SNR system and in Figure 5 for the low-SNR system. The corner plot is arranged identically to Figure 1, except that the different coloured curves now correspond to different inference models, rather than different realisations of 𝒀\bm{Y}. The blue curves are the results derived using Earth\mathcal{M}_{\rm Earth}. The green curves are the results derived using psr&Earth\mathcal{M}_{\rm psr\&Earth}.

For the high-SNR results in Figure 4, the one-dimensional posteriors for Earth\mathcal{M}_{\rm Earth} are biased, as was observed by K24, as well as Zhu et al. (2016) and Chen & Wang (2022). Particular biases are observed in ψ,ι\psi,\iota and α\alpha, with the blue posterior displaced with respect to the orange injection line. For example, for ι\iota, the injected value is contained within the 90% credible interval, but the median of the posterior of the Earth-term model is shifted from the injected value by 0.35 radians. The inclusion of the pulsar terms corrects for this bias. The green one-dimensional posteriors exhibit no bias and are generally symmetric about the injected value. Moreover, for every pair of parameters in 𝜽gw{\bm{\theta}}_{\rm gw}, the injected values are contained within the 2-sigma contours for psr&Earth\mathcal{M}_{\rm psr\&Earth}. This is not true for Earth\mathcal{M}_{\rm Earth}, where the injected values fall outside the 2-sigma contour for 15 out of the 21 parameter pairs.

We define a relative error to quantify the accuracy of the one-dimensional posteriors with respect to the injected value. The relative error is defined to be the relative unsigned displacement of the mode of pM(θ|𝒀)p_{\rm M}\left(\theta|\bm{Y}\right) from the injection, viz.

ΔM(θ)=|argmax 𝜃pM(θ|𝒀)θinj|θinj.\displaystyle\Delta_{\rm M}(\theta)=\frac{\left|\underset{\theta}{\text{argmax }}p_{\rm M}\left(\theta|\bm{Y}\right)-\theta_{\rm inj}\right|}{\theta_{\rm inj}}\ . (22)

In Equation (22), pM(θ|𝒀)p_{\rm M}\left(\theta|\bm{Y}\right) is the one-dimensional posterior for parameter θ\theta returned by nested sampling (c.f. Equation (12)). The subscript M{Earth, psr&Earth}\rm M\in\{\text{Earth, psr\&Earth}\} indicates whether the posterior is estimated using Equation (15) or Equation (8) respectively. θinj\theta_{\rm inj} denotes the true injection value. We note that ΔM(θ)\Delta_{\rm M}(\theta) quantifies the accuracy of the estimates returned by the two models, i.e. the closeness of the most probable estimate of θ\theta relative to θinj\theta_{\rm inj}. In contrast, ΔM(θ)\Delta_{\rm M}(\theta) does not quantify the uncertainty in the estimates. The error ΔM(θ)\Delta_{\rm M}(\theta) for each parameter in Figure 4 is summarised in Table 2. We find that the estimates are more accurate using the pulsar terms, i.e. ΔEarth(θ)>Δpsr&Earth(θ)\Delta_{\rm Earth}(\theta)>\Delta_{\rm psr\&Earth}(\theta) for θ𝜽gw\theta\in\bm{\theta}_{\rm gw}. In some cases the difference is modest; Ω\Omega is recovered with high accuracy by both models, with ΔEarth(Ω)Δpsr&Earth(Ω)=1.5×105\Delta_{\rm Earth}(\Omega)-\Delta_{\rm psr\&Earth}(\Omega)=1.5\times 10^{-5}. The improvement from including the pulsar terms is largest for ι\iota, with ΔEarth(ι)Δpsr&Earth(ι)=0.14\Delta_{\rm Earth}(\iota)-\Delta_{\rm psr\&Earth}(\iota)=0.14. Whilst we present only a single noise realisation in this section, the improvements from including the pulsar terms are found to be comparable across different realisations.

For the low-SNR results in Figure 5, there is less improvement in the parameter estimates. Qualitatively, the green and blue contours and histograms mostly overlap, although the green contours are centred slightly better on the injected values. The relative error ΔM(θ)\Delta_{\rm M}(\theta) for the low-SNR results is reported in the lower half of Table 2. We see that the error is larger for every element in 𝜽gw\bm{\theta}_{\rm gw} than in the high-SNR case, as expected. The inclusion of the pulsar terms improve the estimates for five out of the seven static parameters; we find ΔEarth(θ)>Δpsr&Earth(θ)\Delta_{\rm Earth}(\theta)>\Delta_{\rm psr\&Earth}(\theta) for all parameters except ψ\psi and h0h_{0}. However it is hard to draw strong conclusions; the improvements from including the pulsar terms vary randomly across different noise realisations. We show in Section 6.2 that including the pulsar terms increases the detection probability.

Refer to caption
Figure 4: Posterior distribution in the form of a standard corner plot of the GW source parameters 𝜽gw\bm{\theta}_{\rm gw} for the representative system described in Table 1, with h0=1×1012h_{0}=1\times 10^{-12}, for a single realisation of the system noise. The blue curves are the posteriors calculated using the Earth-term model, Equation (15). The green curves are calculated by including the Earth term and pulsar terms, Equation (8). The vertical and horizontal orange lines indicate the true injected values. The contours in the two-dimensional histograms denote the (0.5, 1, 1.5, 2)-σ\sigma levels. The supertitles of the one-dimensional histograms record the medians and the 0.16 and 0.84 quantiles of the green curves, i.e. the pulsar-term model. We plot the scaled variables 109Ω10^{9}\Omega (units: rad s-1) and 1012h010^{12}h_{0}. Some parameters (e.g. ψ,ι\psi,\iota) exhibit a bias when using the Earth-term model, which disappears when the pulsar terms are included.
Refer to caption
Figure 5: Same as Figure 4, but for a low-SNR system with h0=5×1015h_{0}=5\times 10^{-15}. The posterior distributions with (green curves) and without (blue curves) the pulsar terms overlap more closely than in Figure 4, but the green curves are still centered slightly more accurately on the injected values (horizontal and vertical orange lines).
θ\theta ΔEarth(θ)\Delta_{\rm Earth}(\theta) Δpsr&Earth(θ)\Delta_{\rm psr\&Earth}(\theta)
High SNR Ω\Omega 1.9×1051.9\times 10^{-5} 3.4×1063.4\times 10^{-6}
Φ0\Phi_{0} 1.6×1021.6\times 10^{-2} 2.2×1032.2\times 10^{-3}
ψ\psi 3.9×1023.9\times 10^{-2} 2.0×1042.0\times 10^{-4}
ι\iota 2.0×1012.0\times 10^{-1} 6.1×1026.1\times 10^{-2}
δ\delta 1.9×1031.9\times 10^{-3} 2.0×1042.0\times 10^{-4}
α\alpha 4.0×1024.0\times 10^{-2} 2.8×1042.8\times 10^{-4}
h0h_{0} 9.3×1029.3\times 10^{-2} 4.9×1024.9\times 10^{-2}
Low SNR Ω\Omega 4.8×1034.8\times 10^{-3} 1.5×1031.5\times 10^{-3}
Φ0\Phi_{0} 1.4×1001.4\times 10^{0} 3.3×1013.3\times 10^{-1}
ψ\psi 5.3×1035.3\times 10^{-3} 1.1×1021.1\times 10^{-2}
ι\iota 3.3×1013.3\times 10^{-1} 2.1×1012.1\times 10^{-1}
δ\delta 1.3×1011.3\times 10^{-1} 4.2×1024.2\times 10^{-2}
α\alpha 2.5×1022.5\times 10^{-2} 3.3×1033.3\times 10^{-3}
h0h_{0} 1.1×1011.1\times 10^{-1} 1.6×1011.6\times 10^{-1}
Table 2: Relative error ΔM(θ)\Delta_{\rm M}(\theta), Equation (22), in the mode of the one-dimensional posteriors calculated using the Earth-term model (M = Earth, Equation (15)) and the pulsar-term model (M = psr&Earth\rm psr\&Earth, Equation (8)) for θ𝜽gw\theta\in\bm{\theta}_{\rm gw}. The injected values are summarised in Table 1. The top and bottom halves of the table contain the high-SNR (h0=1×1012h_{0}=1\times 10^{-12}) and low-SNR (h0=5×1015h_{0}=5\times 10^{-15}) cases respectively. The psr&Earth model is more accurate than the Earth model for all θ𝜽gw\theta\in\bm{\theta}_{\rm gw} at high SNR, and five out of seven θ𝜽gw\theta\in\bm{\theta}_{\rm gw} at low SNR.

6.2 Detectability vs h0h_{0}

Refer to caption
Figure 6: Bayes factor (odds ratio) βM\beta_{\rm M} between the competing models M\mathcal{M}_{\rm M}, with M{psr&Earth,Earth}\rm M\in\left\{\rm psr\&Earth,Earth\right\} (GW present in data) and null\mathcal{M}_{\rm null} (GW not present in data) as a function of the signal amplitude, h0h_{0}, for the representative example in Table 1. The horizontal grey dashed line labels an arbitrary detection threshold, βM=10\beta_{\rm M}=10. The minimum detectable strain at βM=10\beta_{\rm M}=10 equals 3.2×10153.2\times 10^{-15} for Earth\mathcal{M}_{\rm Earth} (blue points) and 2.8×10152.8\times 10^{-15} for psr&Earth\mathcal{M}_{\rm psr\&Earth} (green points). The axes are plotted on logarithmic scales.

In this section we compute the minimum detectable strain for the representative source in Table 1, using Earth\mathcal{M}_{\rm Earth} and psr&Earth\mathcal{M}_{\rm psr\&Earth}. We frame the detection problem in terms of a Bayesian model selection procedure, following the lead of other PTA analyses (e.g. Agazie et al., 2023a; Antoniadis et al., 2023a; Reardon et al., 2023; Xu et al., 2023). We define null\mathcal{M}_{\rm null} as the null model that assumes no GW exists in the data. This is equivalent to setting g(n)(t)=1g^{(n)}(t)=1 in Equation (8). The evidence integral 𝒵\mathcal{Z} returned by nested sampling, Equation (13), is the probability of the data 𝒀\bm{Y} given a model M\mathcal{M}_{\rm M}. The support in the data for the presence of a GW signal, described by model M\mathcal{M}_{\rm M}, over the absence of a GW signal is quantified via the Bayes factor

βM=𝒵(𝒀|M)𝒵(𝒀|null).\beta_{\rm M}=\frac{\mathcal{Z}(\bm{Y}|\mathcal{M}_{\rm M})}{\mathcal{Z}(\bm{Y}|\mathcal{M}_{\rm null})}\ . (23)

In this paper we consider M{Earth,psr&Earth}\rm M\in\left\{\rm Earth,psr\&Earth\right\}.

The Bayes factors βEarth\beta_{\rm Earth} and βpsr&Earth\beta_{\rm psr{\&}Earth} are plotted as functions of h0h_{0} in Figure 6 for the representative source in Table 1. We vary the source amplitude from h0=1015h_{0}=10^{-15} (undetectable) to h0=1012h_{0}=10^{-12} (easily detectable). To sharpen the comparison between psr&Earth\mathcal{M}_{\rm psr\&Earth} and Earth\mathcal{M}_{\rm Earth} we present only a single noise realisation of the synthetic data 𝒀\bm{Y}, as in Section 6.1; the conclusions drawn below are consistent across different noise realisations. Moreover, the noise processes in the synthetic data are identical realisations for each value of h0h_{0}; the only change from one h0h_{0} value to the next is h0h_{0} itself, to smooth the curves in Figure 6.1.

Figure 6 reveals an approximate quadratic relationship βh02\beta\propto h_{0}^{2} for h01014h_{0}\gtrsim 10^{-14} for both Earth\mathcal{M}_{\rm Earth} and psr&Earth\mathcal{M}_{\rm psr\&Earth}. Moreover, we obtain βpsr&Earth>βEarth\beta_{\rm psr\&Earth}>\beta_{\rm Earth} for all h0h_{0}. That is, for a given h0h_{0}, psr&Earth\mathcal{M}_{\rm psr\&Earth} provides greater evidence for a GW signal in the noisy data than Earth\mathcal{M}_{\rm Earth}. The GW source is detectable with decisive evidence (β10\beta\geq 10) for h03.2×1015h_{0}\gtrsim 3.2\times 10^{-15} for Earth\mathcal{M}_{\rm Earth} and h02.8×1015h_{0}\gtrsim 2.8\times 10^{-15} for psr&Earth\mathcal{M}_{\rm psr\&Earth}, a relative improvement in the minimal detectable strain of 14%14\%. The minimum detectable strain and the relative improvement through using psr&Earth\mathcal{M}_{\rm psr\&Earth} are particular to the system in Table 1 and the realisation of 𝒀\bm{Y}. They are influenced in general by TobsT_{\rm obs}, TcadT_{\rm cad}, 𝜽gw{\bm{\theta}}_{\rm gw}, and 𝜽psr{\bm{\theta}}_{\rm psr}. A full parameter sweep is postponed to future work, after the analysis scheme in this paper is upgraded from ingesting pulse frequencies to pulse TOAs to enable a like-for-like comparison with standard PTA analyses. In the interim, we note that a 14%14\% improvement in sensitivity when including the pulsar terms is comparable to the improvements of 5%5\% achieved by Zhu et al. (2016).

7 Conclusion

In this paper we demonstrate how to extend state-space methods for PTA data analysis to include the pulsar terms as well as the Earth term. We emphasize that including the pulsar terms is not a new idea; the advantages of doing so are well known in standard PTA analyses (e.g. Zhu et al., 2016; Chen & Wang, 2022; Agazie et al., 2023a; Antoniadis et al., 2023a; Reardon et al., 2023; Xu et al., 2023; Arzoumanian et al., 2023; Antoniadis et al., 2023c). The goal of this paper is to verify whether the advantages apply equally to state-space methods, which complement standard analyses. In the state-space formulation, the rotational state of each pulsar evolves according to a mean-reverting Ornstein-Uhlenbeck process, Equations (1)–(4), and is tracked using a Kalman filter. The measurement equation in the Kalman filter is reparameterized in terms of a pulsar-specific static phase χ(n)\chi^{(n)} (1nN1\leq n\leq N) to be inferred for each pulsar. The Kalman filter is combined with a nested sampler to estimate the posterior distributions of each static parameter, as well as the associated Bayesian evidence of the signal models with and without the pulsar terms included.

The updated state-space model including the pulsar terms is tested on synthetic data. We start by considering 10 noise realisations for a single, astrophysically representative, SMBHB GW source with h0=5×1015h_{0}=5\times 10^{-15}, observed by the 12.5-year NANOGrav pulsars (N=47N=47) with Tobs=10yearsT_{\rm obs}=10\,{\rm years} and Tcad=1T_{\rm cad}=1 week. We find that the updated state-space model successfully detects injected signals and estimates their static parameters accurately for all noise realisations with relatively low computational cost. The irreducible random dispersion (cosmic variance) in the median of the marginalised one-dimensional posteriors is non-negligible, with a maximum CVCV of 52% for Φ0\Phi_{0}, a minimum of 0.2% for Ω\Omega and a median of 10%10\% for h0h_{0}.

The updated model is further tested across a broad and astrophysically plausible parameter domain of SMBHB sources, exploring 200 randomly sampled 𝜽gw\bm{\theta}_{\rm gw} at fixed ι\iota and h0h_{0}. Consistent parameter estimates are obtained, although the accuracy is lower, when one or more PTA pulsars coincide approximately on the sky with the SMBHB source. For the best estimated parameter Ω\Omega the injection is contained within the 90% credible interval in 91% of cases. For the worst estimated parameter Φ0\Phi_{0}, the injection is contained within the 90% credible interval in 81% of cases. Including the pulsar terms increases the accuracy (as quantified by the root-mean-square relative error, Equation (22)) for five out of the seven static parameters in 𝜽gw\bm{\theta}_{\rm gw}; we find ΔEarth(θ)>Δpsr&Earth(θ)\Delta_{\rm Earth}(\theta)>\Delta_{\rm psr\&Earth}(\theta) for all parameters except ψ\psi and h0h_{0}. In the high-SNR case (h0=1×1012h_{0}=1\times 10^{-12}) all of the seven static parameters are estimated more accurately; we find ΔEarth(θ)>Δpsr&Earth(θ)\Delta_{\rm Earth}(\theta)>\Delta_{\rm psr\&Earth}(\theta) for θ\theta in 𝜽gw\bm{\theta}_{\rm gw}. Including the pulsar terms lowers the minimum detectable strain by 14%14\%, comparable with standard PTA analyses (e.g. Zhu et al., 2016).

State-space methods for PTA data analysis are complementary to traditional approaches. They track the actual, astrophysical, time-ordered realization of the timing noise in every PTA pulsar instead of fitting for the noise power spectral density (PSD), which averages over the ensemble of possible timing noise realizations (not just the actual one) and discards time-ordered information when taking the modulus of the complex Fourier phase in each PSD frequency bin. That is, it is possible to disentangle statistically the specific time-ordered realisation of the timing noise from the GW-induced modulations, and thereby infer the GW source parameters. State-space models are conditional on a noise model that is related but different to the noise model in traditional approaches. The analysis in this paper assumes a mean-reverting Ornstein-Uhlenbeck process, whereas traditional analyses assume a stationary Gaussian process described by an ensemble-averaged, power-law PSD, whose amplitude and exponent are adjustable. The Ornstein-Uhlenbeck process also maps onto a stationary Gaussian process, whose PSD is a power law at high frequencies and rolls over at low frequencies, and whose amplitude and exponent can be adjusted by modifying the form of the damping term and Langevin driver in Equation (1). However, the Ornstein-Uhlenbeck process in the time domain in Equation (1) contains more information than its associated PSD in the frequency domain for the two reasons stated above: the Kalman filter “fits” the actual noise realization rather than an ensemble average, and it preserves time ordering by implicitly preserving the Fourier phases, which the PSD discards. Clarifying the similarities and differences between various approaches promises to be a fruitful avenue of future work. It is also a subject of attention in audio-band GW data analysis involving hidden Markov models applied to data from terrestrial long-baseline interferometers (Middleton et al., 2020; Abbott et al., 2022a, b, c).

There are at least four useful extensions to this work.

  1. 1.

    For the single representative SMBHB source in Section 6.1 there is no appreciable improvement in the parameter estimation accuracy from including the pulsar terms for h0=5×1015h_{0}=5\times 10^{-15} (i.e. low SNR). It would be of interest to explore a broader 𝜽\bm{\theta}^{\prime} domain (i.e. varying both the SMBHB source and the PTA configuration) to check whether including the pulsar terms improves the estimation accuracy at low SNR in some overlooked pockets of the astrophysically plausible 𝜽{\bm{\theta}}^{\prime} domain. A similar exercise should be undertaken with respect to the minimum detectable GW strain (see Section 6.2 ).

  2. 2.

    The assumption of a monochromatic source is well-justified astrophysically in various regimes (see Section 2.2). Nevertheless, SMBHBs are not strictly monochromatic. In principle it is straightforward to include the evolution of Ω(t)\Omega(t) in the differential state equations. However, issues related to identifiability (Bellman & Åström, 1970), the form of Equation (8), and the linear structure of the Kalman filter must be considered carefully and are postponed to a future paper. The Kalman framework can be applied to non-linear problems if needed using either an extended Kalman filter (Zarchan & Musoff, 2000), unscented Kalman filter (Wan & Van Der Merwe, 2000) or particle filter (Simon, 2006). Preliminary performance tests regarding the monochromatic assumption are presented in Appendix A.

  3. 3.

    The state-space analysis presented here ingests a frequency time series fm(n)(t)f_{\rm m}^{(n)}(t). It is necessary to generalize this approach to ingest TOAs directly, as happens in standard PTA analyses (e.g. Zhu et al., 2016; Chen & Wang, 2022; Agazie et al., 2023a; Antoniadis et al., 2023a; Reardon et al., 2023; Xu et al., 2023; Arzoumanian et al., 2023; Antoniadis et al., 2023c). Generalizing the algorithm is a surprisingly subtle task and will be presented in a forthcoming paper.

  4. 4.

    We assume in this paper that there is only one GW source. The Kalman framework extends naturally to multiple sources. It is straightforward to modify Equation (5) to describe a linear superposition of GWs. This is useful for two reasons. Firstly, it may be possible to resolve multiple continuous GW sources concurrently (Babak & Sesana, 2012b). Secondly, the stochastic background itself is an incoherent sum of many individual GW sources. It should be possible for a Kalman filter and nested sampler to operate together to detect the stochastic background. With respect to the first reason, it is straightforward to apply the Kalman filter and nested sampler with an extended range of static parameters associated with the GW sources. With respect to the second reason, it is necessary to summarize economically the additional static parameters, whilst respecting the mathematical structure of the Kalman filter. This is a subtle challenge which we postpone to a forthcoming paper. If successful, it will complement the traditional approach of cross-correlating TOA residuals to uncover the Hellings-Downs curve (Hellings & Downs, 1983; Agazie et al., 2023a).

Acknowledgements

This research was supported by the Australian Research Council Centre of Excellence for Gravitational Wave Discovery (OzGrav), grant number CE170100004. The numerical calculations were performed on the OzSTAR supercomputer facility at Swinburne University of Technology. The OzSTAR program receives funding in part from the Astronomy National Collaborative Research Infrastructure Strategy (NCRIS) allocation provided by the Australian Government.

Data Availability

No new data were generated or analysed in support of this research.

References

Appendix A Validity of the monochromatic assumption

In this paper, as discussed in Section 2.2, we treat the GW source as non-evolving, such that Ω\Omega is constant, and the GW that modulates the received pulsar signal is monochromatic. This is a reasonable approximation for the primary goal of this paper, namely studying the biases incurred by omitting the pulsar terms within a state-space formulation. Previous investigations of pulsar-term biases in the context of standard PTA analyses (e.g. Zhu et al., 2016; Chen & Wang, 2022) also treat the GW source as non-evolving.

The non-evolving approximation is almost exact for SMBHBs over the timescale set by Tobs10T_{\rm obs}\sim 10 years; see Equation (9) and the associated discussion in Section 2.2. However, over timescales that correspond to the light travel time between pulsar and Earth (i.e. d(n)/cTobsd^{(n)}/c\gg T_{\rm obs}) the inspiralling binary typically evolves appreciably. Consequently the value of Ω\Omega for the GW as it strikes the Earth (i.e. the Earth term) is distinct from the value of Ω\Omega as it strikes a pulsar (i.e. the pulsar term). In this appendix we repeat the Bayesian inference analysis of the main text (Section 5) but now consider synthetic data for evolving sources, whilst retaining the non-evolving inference model (Section 2.2). Our purpose is not to perform an in-depth performance analysis, but rather to confirm that the conclusions drawn in the main text are not compromised substantially by the treating the GW source as non-evolving. This appendix is organised as follows. In Appendix A.1 we review how the value of Ω\Omega at the pulsar is related to the value of Ω\Omega measured at Earth. We outline how to generate synthetic data for evolving sources and the procedure for testing the performance of the method on the new synthetic data. In Appendix A.2 we apply the state-space analysis scheme of the main text to synthetic data for evolving sources. We recover the system parameters 𝜽gw\bm{\theta}_{\rm gw}, and compare the accuracy of the results with those obtained for non-evolving sources. In Appendix A.3 we compare the minimum detectable GW strain for evolving sources with that of non-evolving sources.

A.1 Quasi-monochromatic synthetic data

The angular frequency of the GW at the Earth, ΩEarth\Omega_{\rm Earth}, is related to the angular frequency of the GW at the nn-th pulsar, Ωpsr(n)\Omega_{\rm psr}^{(n)}, by (Perrodin & Sesana, 2018; Agazie et al., 2023c; Arzoumanian et al., 2023)

Ωpsr(n)=ΩEarth{1+2565Mc5/3Ωearth8/3[tp(n)t]}3/8,\Omega_{\rm psr}^{(n)}=\Omega_{\rm Earth}\left\{1+\frac{256}{5}M_{\rm c}^{5/3}\Omega_{\rm earth}^{8/3}\left[t_{\rm p}^{(n)}-t\right]\right\}^{-3/8}\,, (24)

where tp(n)t_{\rm p}^{(n)} is the retarded time at which the GW is incident on the nn-th pulsar, tt is the time at which the GW is incident on the Earth, and we have

tp(n)t=d(n)[1+𝒏𝒒(n)].t_{\rm p}^{(n)}-t=-d^{(n)}\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]\,. (25)

Note that Ωpsr(n)\Omega_{\rm psr}^{(n)} depends on nn and is smaller than ΩEarth\Omega_{\rm Earth}. For example, for a GW with ΩEarth=5×107\Omega_{\rm Earth}=5\times 10^{-7} Hz, Equation (24) implies that a pulsar at a distance of 1 kpc experiences Ωpsr(n)3×107\Omega_{\rm psr}^{(n)}\approx 3\times 10^{-7} Hz, taking Mc=108MM_{\rm c}=10^{8}M_{\odot}.

In Sections 26 we assume Ωpsr(n)=ΩEarth\Omega_{\rm psr}^{(n)}=\Omega_{\rm Earth} for all nn. If we relax this assumption, the measurement equation, Equation (17), becomes (Perrodin & Sesana, 2018)

gevolving(n)(t)=\displaystyle g^{(n)}_{\rm evolving}(t)= 1Hij[q(n)]i[q(n)]j2[1+𝒏𝒒(n)]\displaystyle 1-\frac{H_{ij}[q^{(n)}]^{i}[q^{(n)}]^{j}}{2[1+\bm{n}\cdot\bm{q}^{(n)}]}
×{cos(Ωeartht+Φ0)\displaystyle\times\Big{\{}\cos\left(-\Omega_{\rm earth}t+\Phi_{0}\right)
cos[Ωpsr(n)t+Φ0+χ(n)]}.\displaystyle-\cos\left[-\Omega^{(n)}_{\rm psr}t+\Phi_{0}+\chi^{(n)}\right]\Big{\}}\ . (26)

In order to test how sensitive the results in Sections 26 are to the assumption Ωpsr(n)=ΩEarth\Omega_{\rm psr}^{(n)}=\Omega_{\rm Earth}, we take the following steps.

  1. 1.

    Generate synthetic data 𝒀evolving\bm{Y}_{\rm evolving} using Equation (26) via the procedure outlined in Section 4, using the parameters defined in Table 1.

  2. 2.

    Pass 𝒀evolving\bm{Y}_{\rm evolving} into the Kalman filter - and nested sampler described in Section 5. The analysis pipeline retains Equation (17) as the measurement equation for the purposes of inference.

  3. 3.

    Calculate p(𝜽|𝒀evolving)p({\bm{\theta}^{\prime}}|{\bm{Y}_{\rm evolving}}).

  4. 4.

    Calculate the evidences 𝒵(𝒀evolving|psr&Earth)\mathcal{Z}(\bm{Y}_{\rm evolving}|\mathcal{M}_{\rm\rm psr\&Earth}) and 𝒵(𝒀evolving|null)\mathcal{Z}(\bm{Y}_{\rm evolving}|\mathcal{M}_{\rm null}).

  5. 5.

    Take the ratio of the evidences in step (iv) to obtain a Bayes factor, βpsr&Earth,evolving\beta_{\rm psr\&Earth,evolving} cf. Equation (23).

In step (i), we assume that the SMBHB has chirp mass Mc=108MM_{\rm c}=10^{8}M_{\odot}. Step (iii) is the focus of Appendix A.2. We compare p(𝜽|𝒀evolving)p({\bm{\theta}^{\prime}}|{\bm{Y}_{\rm evolving}}) and p(𝜽|𝒀)p({\bm{\theta}^{\prime}}|{\bm{Y}}), the posterior obtained in the main text for monochromatic data. Steps (iv) and (v) are the focus of Appendix A.3. We compare βpsr&Earth,evolving\beta_{\rm psr\&Earth,evolving} with βpsr&Earth\beta_{\rm psr\&Earth}, the Bayes factors obtained in the main text for monochromatic data.

A.2 Parameter estimation

Refer to caption
Figure 7: Impact on parameter estimation of inspiral-driven SMBH evolution on the GW frequency at Earth and at the retarded time at every pulsar. Posterior distribution in the form of a standard corner plot of the GW source parameters 𝜽gw\bm{\theta}_{\rm gw} for the representative system described in Table 1, with h0=1×1012h_{0}=1\times 10^{-12}, for a single realisation of the system noise. The blue curves are the posteriors calculated using the Earth and pulsar terms inference model, Equation (8), computed for non-evolving data 𝒀\bm{Y}. The green curves are the posteriors calculated using the same inference model for evolving data 𝒀evolving\bm{Y}_{\rm evolving}. The vertical and horizontal orange lines indicate the true injected values. The contours in the two-dimensional histograms denote the (0.5, 1, 1.5, 2)-σ\sigma levels. The supertitles of the one-dimensional histograms record the medians and the 0.16 and 0.84 quantiles of the green curves. We plot the scaled variables 109Ω10^{9}\Omega (units: rad s-1) and 1012h010^{12}h_{0}. Qualitatively, the posteriors are broadly similar, with modest shifts in the modes of Φ0\Phi_{0} and h0h_{0}.
Refer to caption
Figure 8: Same as Figure 7, but for a low-SNR system with h0 5×1015h_{0}\ 5\times 10^{-15}. Qualitatively, the blue and green curves are similar and overlap for all θ𝜽gw\theta\in\bm{\theta}_{\rm gw}.
Refer to caption
Figure 9: Same as Figure 2, but for synthetic data 𝒀evolving\bm{Y}_{\rm evolving} generated for an inspiralling SMBH, whose frequency evolves. Consistent unimodal posteriors are obtained across all noise realisations and all five displayed parameters. The one-dimensional posteriors are biased with respect to the injection values due to the unmodelled component of Ωpsr(n)\Omega_{\rm psr}^{(n)}. Ten curves are plotted, corresponding to ten realisations of 𝒀evolving\bm{Y}_{\rm evolving}, but the curves overlap and are hard to distinguish by eye.

In this section we calculate the joint posterior probability distribution p(𝜽|𝒀evolving)p({\bm{\theta}^{\prime}}|{\bm{Y}_{\rm evolving}}). We consider two representative systems, analogously to Section 6.1. We again consider a “low-SNR” system with h0=5×1015h_{0}=5\times 10^{-15} and a “high-SNR” system with h0=1×1012h_{0}=1\times 10^{-12}. All other static parameters are as specified in Table 1.

The results for p(𝜽gw|𝒀evolving)p({\bm{\theta}_{\rm gw}}|{\bm{Y}_{\rm evolving}}) and p(𝜽gw|𝒀)p({\bm{\theta}_{\rm gw}}|{\bm{Y}}) are shown in Figure 7 for the high-SNR system and in Figure 8 for the low-SNR system. The corner plots are arranged identically to Figures 4 and 5, except that the different coloured curves correspond to posterior distributions inferred on different data. The blue curves are the results for p(𝜽gw|𝒀)p({\bm{\theta}_{\rm gw}}|{\bm{Y}}). The green curves are the results for p(𝜽gw|𝒀evolving)p({\bm{\theta}_{\rm gw}}|{\bm{Y}_{\rm evolving}}). The axes cover a subset of the prior domain and are identical to the scales in Figures 4 and 5, with the exception of the axis for h0h_{0} in Figure 8, which covers a slightly broader range.

For the high-SNR results in Figure 7, the one-dimensional posteriors for p(𝜽gw|𝒀evolving)p({\bm{\theta}_{\rm gw}}|{\bm{Y}_{\rm evolving}}) and p(𝜽gw|𝒀)p({\bm{\theta}_{\rm gw}}|{\bm{Y}}) are similar for five of the seven parameters in 𝜽gw\bm{\theta}_{\rm gw}. The two exceptions are Φ0\Phi_{0} and h0h_{0}. Regarding Φ0\Phi_{0}, the mode of the one-dimensional posterior p(Φ0|𝒀evolving)p(\Phi_{0}|{\bm{Y}_{\rm evolving}}) is offset from the mode of p(Φ0|𝒀)p(\Phi_{0}|{\bm{Y}}) by \approx 0.4 radians. The shift can be understood by inspecting Equation (26); the inference model based on Equation (17) attempts to compensate for the unmodelled phase induced by Ωpsr(n)\Omega_{\rm psr}^{(n)} by adjusting the value of Φ0\Phi_{0}. The estimates of χ(n)\chi^{(n)} are similarly biased. In Figure 9 we plot the results for a representative subset for five out of 47 pulsar phases χ(1)χ(5)\chi^{(1)}\dots\chi^{(5)}, for ten realisations of 𝒀evolving\bm{Y}_{\rm evolving}. The figure is exactly analogous to Figure 2. The one-dimensional histograms are evidently biased with respect to the injection value for the same reason as Φ0\Phi_{0}; the inference model must account for the unmodelled component of Ωpsr(n)\Omega_{\rm psr}^{(n)}. Regarding h0h_{0}, the mode of the one dimensional posterior p(h0|𝒀evolving)p(h_{0}|{\bm{Y}_{\rm evolving}}) is offset from the mode of p(h0|𝒀)p(h_{0}|{\bm{Y}}) by \approx 5×10135\times 10^{-13}. Similarly to Φ0\Phi_{0}, this shift occurs due to the unmodelled phase component of Ωpsr(n)\Omega_{\rm psr}^{(n)}, which manifests as a correction to the amplitude. The shift is comparable to the known dispersion (cosmic variance)in h0h_{0} (cf. Appendix G and Figure 12).

We define a relative error to quantify the accuracy of the one-dimensional posteriors with respect to the injected value, analogously to Section 6.1 and Equation (22), viz.

ΔI(θ)=|argmax 𝜃pM(θ|I)θinj|θinj,\displaystyle\Delta_{I}(\theta)=\frac{\left|\underset{\theta}{\text{argmax }}p_{\rm M}\left(\theta|I\right)-\theta_{\rm inj}\right|}{\theta_{\rm inj}}\,, (27)

where the subscript I{𝒀,𝒀evolving}I\in\{\bm{Y},\bm{Y}_{\rm evolving}\} indicates whether the posterior is estimated using monochromatic data (i.e. synthetic data generated using Equation (17)) or evolving data (i.e. synthetic data generated using Equation (26)), respectively. The error ΔI(θ)\Delta_{I}(\theta) for each parameter in Figure 7 is summarised in the upper half of Table 3.

θ\theta Δ𝒀(θ)\Delta_{\bm{Y}}(\theta) Δ𝒀evolving(θ)\Delta_{\bm{Y}_{\rm evolving}}(\theta)
High SNR Ω\Omega 1.4×1051.4\times 10^{-5} 9.0×1059.0\times 10^{-5}
Φ0\Phi_{0} 1.8×1021.8\times 10^{-2} 1.9×1001.9\times 10^{0}
ψ\psi 3.4×1043.4\times 10^{-4} 2.8×1032.8\times 10^{-3}
ι\iota 5.7×1025.7\times 10^{-2} 1.3×1011.3\times 10^{-1}
δ\delta 5.9×1045.9\times 10^{-4} 1.5×1021.5\times 10^{-2}
α\alpha 2.2×1042.2\times 10^{-4} 2.0×1032.0\times 10^{-3}
h0h_{0} 4.0×1024.0\times 10^{-2} 5.0×1015.0\times 10^{-1}
Low SNR Ω\Omega 7.8×1047.8\times 10^{-4} 4.3×1034.3\times 10^{-3}
Φ0\Phi_{0} 3.3×1013.3\times 10^{-1} 1.0×1001.0\times 10^{0}
ψ\psi 7.7×1027.7\times 10^{-2} 1.4×1011.4\times 10^{-1}
ι\iota 3.0×1013.0\times 10^{-1} 1.1×1011.1\times 10^{-1}
δ\delta 2.2×1012.2\times 10^{-1} 1.6×1011.6\times 10^{-1}
α\alpha 2.0×1012.0\times 10^{-1} 3.1×1013.1\times 10^{-1}
h0h_{0} 3.4×1013.4\times 10^{-1} 4.7×1014.7\times 10^{-1}
Table 3: Same as Table 2 but for relative error ΔI(θ)\Delta_{I}(\theta), Equation (27), where the subscript i{𝒀,𝒀evolving}i\in\{\bm{Y},\bm{Y}_{\rm evolving}\} indicates whether the posterior is estimated using monochromatic data (i.e. synthetic data generated using Equation (17)) or evolving data (i.e. synthetic data generated using Equation (26)), respectively. The top and bottom halves of the table record the high-SNR (h0=1×1012h_{0}=1\times 10^{-12}) and low-SNR (h0=5×1015h_{0}=5\times 10^{-15}) tests respectively. The injected values are summarised in Table 1. For high SNR, we obtain Δ𝒀(θ)<Δ𝒀evolving(θ)\Delta_{\bm{Y}}(\theta)<\Delta_{\bm{Y}_{\rm evolving}}(\theta) for θ𝜽gw\theta\in\bm{\theta}_{\rm gw}. For low-SNR, we obtain Δ𝒀(θ)<Δ𝒀evolving(θ)\Delta_{\bm{Y}}(\theta)<\Delta_{\bm{Y}_{\rm evolving}}(\theta) for five out of seven parameters, although the discrepancy is modest.

Table 3 confirms that we have Δ𝒀(θ)<Δ𝒀evolving(θ)\Delta_{\bm{Y}}(\theta)<\Delta_{\bm{Y}_{\rm evolving}}(\theta) for θ𝜽gw\theta\in\bm{\theta}_{\rm gw}, i.e. the estimates are always more accurate when the inference is run on 𝒀\bm{Y}. However, for five out of seven parameters the difference is modest; the parameters are recovered with high accuracy for 𝒀\bm{Y} and 𝒀evolving\bm{Y}_{\rm evolving}. For example, we find |Δ𝒀(Ω)Δ𝒀evolving(Ω)|=7.6×105|\Delta_{\bm{Y}}(\Omega)-\Delta_{\bm{Y}_{\rm evolving}}(\Omega)|=7.6\times 10^{-5}. For the remaining two parameters, Φ0\Phi_{0} and h0h_{0}, the difference is greater; we find Δ𝒀evolving(Φ0)/Δ𝒀(Φ0)102\Delta_{\bm{Y}_{\rm evolving}}(\Phi_{0})/\Delta_{\bm{Y}}(\Phi_{0})\approx 10^{2} and Δ𝒀evolving(h0)/Δ𝒀(h0)101\Delta_{\bm{Y}_{\rm evolving}}(h_{0})/\Delta_{\bm{Y}}(h_{0})\approx 10^{1}. We present only a single noise realisation in this section, but the improvements from including the pulsar terms are found to be comparable across different realisations.

For the low-SNR results in Figure 8, the one-dimensional posteriors for p(𝜽gw|𝒀evolving)p({\bm{\theta}_{\rm gw}}|{\bm{Y}_{\rm evolving}}) and p(𝜽gw|𝒀)p({\bm{\theta}_{\rm gw}}|{\bm{Y}}) resemble each other for all seven parameters 𝜽gw\bm{\theta}_{\rm gw}. Qualitatively, the green and blue contours and histograms mostly overlap, although the blue contours are centred slightly better on the injected values in some cases (e.g. Ω,Φ0\Omega,\Phi_{0}). Again, given the observed dispersion at low-SNR (e.g. Figure 1), it is difficult to drawn strong conclusions about whether p(𝜽gw|𝒀evolving)p({\bm{\theta}_{\rm gw}}|{\bm{Y}_{\rm evolving}}) is more accurate than p(𝜽gw|𝒀)p({\bm{\theta}_{\rm gw}}|{\bm{Y}}). The relative error ΔI(θ)\Delta_{I}(\theta) for the low-SNR results is reported in the lower half of Table 3. Due to the reduced GW signal strength and the correspondingly broader posteriors, the high-SNR hierarchy Δ𝒀(θ)<Δ𝒀evolving(θ)\Delta_{\bm{Y}}(\theta)<\Delta_{\bm{Y}_{\rm evolving}}(\theta) for θ𝜽gw\theta\in\bm{\theta}_{\rm gw} no longer holds. In some cases (e.g. ι\iota, δ\delta), we have Δ𝒀(θ)>Δ𝒀evolving(θ)\Delta_{\bm{Y}}(\theta)>\Delta_{\bm{Y}_{\rm evolving}}(\theta). However the difference is modest, and the posteriors inferred for 𝒀\bm{Y} and 𝒀evolving\bm{Y}_{\rm evolving} are comparable.

A.3 Detection probability

Refer to caption
Figure 10: Bayes factors calculated using the inference model psr&Earth\mathcal{M}_{\rm\rm psr\&Earth} acting on 𝒀\bm{Y} (βpsr&Earth\beta_{\rm psr{\&}Earth}, blue points) and on 𝒀evolving\bm{Y}_{\rm evolving} (βpsr&Earth,evolving\beta_{\rm psr{\&}Earth,evolving}, green points) as a function of the signal amplitude, h0h_{0}, for the representative example in Table 1, cf. Figure 6. The horizontal grey dashed line labels an arbitrary detection threshold, β=10\beta=10. The minimum detectable strain at β=10\beta=10 equals 2.9×10152.9\times 10^{-15} for 𝒀\bm{Y} and 3.7×10153.7\times 10^{-15} for 𝒀evolving\bm{Y}_{\rm evolving}. The axes are plotted on logarithmic scales.

In this section we calculate the minimum detectable strain for the data 𝒀evolving\bm{Y}_{\rm evolving} using the inference model psr&Earth\mathcal{M}_{\rm psr\&Earth}. We compare the minimum detectable strain to that calculated for the monochromatic data, 𝒀\bm{Y}, considered in the main text. We follow Section 6.2 and frame the detection problem in terms of a Bayesian model selection procedure. We emphasise that we compare the minimum detectable strain for the same inference model acting on different data. In contrast, in Section 6.2 we compare the minimum detectable strain for different inference models acting on the same data.

The Bayes factors βpsr&Earth,evolving\beta_{\rm psr\&Earth,evolving} and βpsr&Earth\beta_{\rm psr\&Earth} are plotted as functions of h0h_{0} in Figure 10. We vary the source amplitude from h0=1015h_{0}=10^{-15} (undetectable) to h0=1013h_{0}=10^{-13} (easily detectable). As in Section 6.2 we present only a single noise realisation of the synthetic data pair 𝒀\bm{Y} and 𝒀evolving\bm{Y}_{\rm evolving}. The conclusions drawn below are consistent across different noise realisations.

Figure 10 reveals βpsr&Earth>βpsr&Earth,evolving\beta_{\rm psr\&Earth}>\beta_{\rm psr\&Earth,evolving} for all h0h_{0}. We follow Section 6.2 and take β=10\beta=10 as an arbitrary detection threshold. For monochromatic data 𝒀\bm{Y}, the GW source is detectable for h02.9×1015h_{0}\gtrsim 2.9\times 10^{-15}. For data from evolving sources, 𝒀evolving\bm{Y}_{\rm evolving}, the GW source is detectable for h03.7×1015h_{0}\gtrsim 3.7\times 10^{-15}. That is, the minimal detectable strain deteriorates by 26%. As in Section 6.2, the minimum detectable strain is particular to the system in Table 1 and the realisations of 𝒀\bm{Y} and 𝒀evolving\bm{Y}_{\rm evolving}.

Appendix B Kalman filter

The Kalman filter (Kalman, 1960) is an optimal solver for Gauss-Markov processes. Given a temporal sequence of noisy measurements, 𝒀(t)\bm{Y}(t), the Kalman filter recovers a temporal sequence of stochastically evolving system state variables, 𝑿(t)\bm{X}(t), which are hidden from the observer. It is a standard method in control theory (Zarchan & Musoff, 2000; Challa et al., 2011; Chui & Chen, 2017) and finds common use in engineering applications (e.g. Zhu et al., 2022; Won et al., 2010; Zhang et al., 2019; Wiltshire et al., 2007) as well as successful application to neutron star astrophysics (e.g. Meyers et al., 2021a; Meyers et al., 2021b; Melatos et al., 2023). In this appendix, we outline the Kalman filter used in this paper. The general recursion relations for the discrete-time Kalman filter are presented for an arbitrary linear dynamical system in Section B.1, along with the formula for the Bayesian likelihood. The application to the specific continuous-time state-space model in Section 2 is outlined in Section B.2.

B.1 Recursion equations and likelihood

In this work we use the linear Kalman filter, which assumes linear relations between d𝑿/dtd{\bm{X}}/dt and 𝑿(t){\bm{X}}(t) (dynamics) and between 𝒀(t){\bm{Y}}(t) and 𝑿(t){\bm{X}}(t) (measurement). The linear Kalman filter operates on temporally discrete, noisy measurements 𝒀k=𝒀(tk)\bm{Y}_{k}=\bm{Y}(t_{k}), which are related via a linear transformation to a set of unobservable discrete system states 𝑿k=𝑿(tk)\bm{X}_{k}=\bm{X}(t_{k}). Each discrete timestep is indexed by 1kK1\leq k\leq K. The measurements are related to the states via

𝒀k=𝑯k𝑿k+𝒗k,\bm{Y}_{k}=\bm{H}_{k}\bm{X}_{k}+\bm{v}_{k}\ , (28)

where 𝑯k\bm{H}_{k} is the measurement matrix or observation model, 𝒗k\bm{v}_{k} is a zero-mean Gaussian measurement noise, one has 𝒩(0,𝑹k)\mathcal{N}\sim(0,\bm{R}_{k}) with covariance 𝑹k\bm{R}_{k}, and the subscript kk labels the time-step. The Kalman filter evolves the underlying states according to

𝑿k=𝑭k𝑿k1+𝑮k𝒖k+𝒘k,\bm{X}_{k}=\bm{F}_{k}\bm{X}_{k-1}+\bm{G}_{k}\bm{u}_{k}+\bm{w}_{k}\ , (29)

where 𝑭k\bm{F}_{k} is the system dynamics matrix, 𝑮k\bm{G}_{k} is the control matrix. 𝒖k\bm{u}_{k} is the control vector, and 𝒘k\bm{w}_{k} is a zero-mean Gaussian process noise, with 𝒘k𝒩(0,𝑸k)\bm{w}_{k}\sim{\cal N}(0,\bm{Q}_{k}) and covariance 𝑸k\bm{Q}_{k}.

The Kalman filter is a recursive estimator with two distinct stages: a “predict” stage and an “update” stage. The predict stage predicts 𝑿^k|k1\hat{\bm{X}}_{k|k-1}, the estimate of the state at discrete step kk, given the state estimate from step k1k-1. Specifically, the predict step proceeds as

𝑿^k|k1\displaystyle\hat{\bm{X}}_{k|k-1} =𝑭k𝑿^k1|k1+𝑮k𝒖k,\displaystyle=\bm{F}_{k}\hat{\bm{X}}_{k-1|k-1}+\bm{G}_{k}\bm{u}_{k}\ , (30)
𝑷^k|k1\displaystyle\hat{\bm{P}}_{k|k-1} =𝑭k𝑷^k1|k1𝑭k+𝑸k,\displaystyle=\bm{F}_{k}\hat{\bm{P}}_{k-1|k-1}\bm{F}_{k}^{\intercal}+\bm{Q}_{k}\ , (31)

where 𝑷^k|k1\hat{\bm{P}}_{k|k-1} is the covariance of the prediction. Note that the predict stage is independent of the measurements. The measurement 𝒀k\bm{Y}_{k} updates the prediction during the update stage as follows:

ϵk\displaystyle\bm{\epsilon}_{k} =𝒀k𝑯k𝑿^k|k1,\displaystyle=\bm{Y}_{k}-\bm{H}_{k}\hat{\bm{X}}_{k|k-1}\ , (32)
𝑺k\displaystyle\bm{S}_{k} =𝑯k𝑷^k|k1𝑯k+𝑹k,\displaystyle=\bm{H}_{k}\hat{\bm{P}}_{k|k-1}\bm{H}_{k}^{\intercal}+\bm{R}_{k}\ , (33)
𝑲k\displaystyle\bm{K}_{k} =𝑷^k|k1𝑯k𝑺k1,\displaystyle=\hat{\bm{P}}_{k|k-1}\bm{H}_{k}^{\intercal}\bm{S}_{k}^{-1}\ , (34)
𝑿^k|k\displaystyle\hat{\bm{X}}_{k|k} =𝑿^k|k1+𝑲kϵk,\displaystyle=\hat{\bm{X}}_{k|k-1}+\bm{K}_{k}\bm{\epsilon}_{k}\ , (35)
𝑷^k|k\displaystyle\hat{\bm{P}}_{k|k} =(𝑰𝑲k𝑯k)𝑷^k|k1.\displaystyle=\left(\bm{I}-\bm{K}_{k}\bm{H}_{k}\right)\hat{\bm{P}}_{k|k-1}\ . (36)

Equation (32) defines a residual ϵk=𝒀k𝒀^k\bm{\epsilon}_{k}=\bm{Y}_{k}-\hat{\bm{Y}}_{k}, which is sometimes termed the innovation. The uncertainty in ϵk\bm{\epsilon}_{k} is quantified via the innovation covariance 𝑺k=ϵkϵkT\bm{S}_{k}=\langle\bm{\epsilon}_{k}\bm{\epsilon}_{k}^{T}\rangle, viz. Equation (33) (noting that the Einstein summation convention is suppressed temporarily in the latter definition). Equation (34) defines the Kalman gain 𝑲k\bm{K}_{k} For a full review of the Kalman filter, including its derivation, we refer the reader to Gelb et al. (1974) and Zarchan & Musoff (2000).

The Gaussian log-likelihood of obtaining 𝒀k{\bm{Y}}_{k} given 𝑿^k{\bm{\hat{X}}}_{k} is calculated at each timestep from the Kalman filter output according to (Zarchan & Musoff, 2000)

logk=12(Dklog2π+log|𝑺k|+ϵk𝑺k1ϵk),\displaystyle\log\mathcal{L}_{k}=-\frac{1}{2}\left(D_{k}\log 2\pi+\log\left|\bm{S}_{k}\right|+\bm{\epsilon}_{k}^{\intercal}\bm{S}_{k}^{-1}\bm{\epsilon}_{k}\right)\ , (37)

where DkD_{k} is the dimension of ϵk\bm{\epsilon}_{k} at timestep kk. The total log-likelihood for the entire sequence is

log=k=1Klogk.\displaystyle\log\mathcal{L}=\sum_{k=1}^{K}\log\mathcal{L}_{k}\ . (38)

Given 𝒀k{\bm{Y}}_{k}, \mathcal{L} is a function of the estimates 𝜽^{\bm{\hat{\theta}}} of the static parameters passed to the Kalman filter, i.e. \mathcal{L} = (𝒀|𝜽^)\mathcal{L}(\bm{Y}|\bm{\hat{\theta}}). Similarly the estimates of the state and measurement variables, 𝑿^\hat{\bm{X}} and 𝒀^\hat{\bm{Y}}, are functions of 𝜽^\bm{\hat{\theta}}. In Appendix C, we explain how to combine the Kalman filter with a nested sampler to iteratively guide 𝜽^{\bm{\hat{\theta}}} towards the true value of 𝜽{\bm{\theta}}.

To apply the Kalman filter in practice means specifying the eight component matrices that make up the “machinery” of the filter: 𝑿k\bm{X}_{k}, 𝒀k\bm{Y}_{k}, 𝑭k\bm{F}_{k}, 𝑮k\bm{G}_{k}, 𝒖k\bm{u}_{k}, 𝑯k\bm{H}_{k}, 𝑸k\bm{Q}_{k} and 𝑹k\bm{R}_{k}. In Appendix B.2 we define the machinery for the state-space model in Section 2.

B.2 State space representation of a PTA analysis

We apply the Kalman recursion relations in Section B.1 to the state-space model of a PTA with NN pulsars described in Section 2 as follows.

We identify 𝑿(t)\bm{X}(t) with a vector of length NN composed of the intrinsic pulsar frequency states, i.e.

𝑿(t)=(fp(1)(t),fp(2)(t),,fp(N)(t)).\bm{X}(t)=\left(f_{\rm p}^{(1)}(t),f_{\rm p}^{(2)}(t),...,f_{\rm p}^{(N)}(t)\right)\ . (39)

Analogously, we package the measured pulsar frequencies as

𝒀(t)=(fm(1)(t),fm(2)(t),,fm(N)(t)).\bm{Y}(t)=\left(f_{\rm m}^{(1)}(t),f_{\rm m}^{(2)}(t),...,f_{\rm m}^{(N)}(t)\right)\ . (40)

The states evolve according to the continuous stochastic differential equation (c.f. Equation (1))

d𝑿=𝑨𝑿dt+𝑪(t)dt+𝚺d𝑩(t),d\bm{X}=\bm{A}\bm{X}dt+\bm{C}(t)dt+\bm{\Sigma}d\bm{B}(t)\ , (41)

where 𝑨\bm{A} is a diagonal N×NN\times N matrix,

𝑨=diag(γ(1),γ(2),,γ(N)),\bm{A}=\text{diag}\left(-\gamma^{(1)},-\gamma^{(2)},...,-\gamma^{(N)}\right)\ , (42)

and 𝑪(t)\bm{C}(t) is a time-dependent N×1N\times 1 vector with nn-th component

C(n)(t)=γ(n)[fem(n)(t1)+f˙em(n)(t1)(tt1)]+f˙em(n)(t1).C^{(n)}(t)=\gamma^{(n)}\left[f^{(n)}_{\rm em}(t_{1})+\dot{f}^{(n)}_{\rm em}(t_{1})\,(t-t_{1})\right]+\dot{f}^{(n)}_{\rm em}(t_{1})\ . (43)

The N×NN\times N matrix 𝚺\bm{\Sigma} governs the magnitude of the increments of Brownian motion (Wiener process) d𝑩(t)d\bm{B}(t), with

𝚺=diag(σ(1),σ(2),,σ(N)).\bm{\Sigma}=\text{diag}\left(\sigma^{(1)},\sigma^{(2)},...,\sigma^{(N)}\right)\ . (44)

In the idealized model in this paper, each pulsar’s rotational state evolves phenomenologically according to a mean-reverting Ornstein-Uhlenbeck process, described by a Langevin equation, Equation (41), whose general solution is given by (Gardiner, 2009)

𝑿(t)=e𝑨t𝑿(0)+0t𝑑te𝑨(tt)𝑪(t)+0t𝑑𝑩(t)e𝑨(tt)𝚺.\bm{X}(t)=e^{\bm{A}t}\bm{X}(0)+\int_{0}^{t}dt^{\prime}e^{\bm{A}(t-t^{\prime})}\bm{C}(t^{\prime})+\int_{0}^{t}d\bm{B}(t^{\prime})e^{\bm{A}(t-t^{\prime})}\bm{\Sigma}\ . (45)

From Equation (45) we construct the discrete, recursive solution for 𝑿(tk)=𝑿k\bm{X}(t_{k})=\bm{X}_{k} in the form of Equation (29), with

𝑭k\displaystyle\bm{F}_{k} =e𝑨Δt\displaystyle=e^{\bm{A}\Delta t}\ (46)
=diag(eγ(1)Δt,eγ(2)Δt,,eγ(N)Δt),\displaystyle=\text{diag}\left(e^{-\gamma^{(1)}\Delta t},e^{-\gamma^{(2)}\Delta t},...,e^{-\gamma^{(N)}\Delta t}\right)\ , (47)
𝑮k𝒖k\displaystyle\bm{G}_{k}\bm{u}_{k} =tktk+1𝑑te𝑨(tk+1t)𝑪(t),\displaystyle=\int_{t_{k}}^{t_{k+1}}dt^{\prime}e^{\bm{A}\left(t_{k+1}-t^{\prime}\right)}\bm{C}(t^{\prime})\ , (48)
=(Gk(1),Gk(2),,Gk(N)),\displaystyle=\left(G^{(1)}_{k},G^{(2)}_{k},...,G^{(N)}_{k}\right), (49)
𝒘k=tktk+1𝑑𝑩(t)e𝑨(tk+1t)𝚺,\bm{w}_{k}=\int_{t_{k}}^{t_{k+1}}d\bm{B}(t^{\prime})e^{\bm{A}\left(t_{k+1}-t^{\prime}\right)}\bm{\Sigma}\ , (50)
Gk(n)=\displaystyle G_{k}^{(n)}= fem(n)(t1)+f˙em(n)(t1)(Δt+tk)\displaystyle f^{(n)}_{\rm em}(t_{1})+\dot{f}^{(n)}_{\rm em}(t_{1})\left(\Delta t+t_{k}\right)
eγΔt[fem(n)(t1)+f˙em(n)(t1)tk],\displaystyle-e^{-\gamma\Delta t}\left[f^{(n)}_{\rm em}(t_{1})+\dot{f}^{(n)}_{\rm em}(t_{1})t_{k}\right]\ , (51)

and Δt=tk+1tk\Delta t=t_{k+1}-t_{k}. From Equation (50) the process noise covariance matrix is

𝑸k𝜹kj=𝜼k𝜼j=diag(Q(1),Q(2),,Q(N)),\bm{Q}_{k}\bm{\delta}_{kj}=\langle\bm{\eta}_{k}\bm{\eta}_{j}^{\intercal}\rangle=\text{diag}\left(Q^{(1)},Q^{(2)},...,Q^{(N)}\right)\ , (52)

with

Q(n)=[σn]22γ(n)[1e2γ(n)Δt].Q^{(n)}=\frac{[\sigma^{n}]^{2}}{2\gamma^{(n)}}\left[1-e^{-2\gamma^{(n)}\Delta t}\right]\ . (53)

The two remaining unspecified component matrices of the Kalman filter are the measurement matrix 𝑯k\bm{H}_{k} and the measurement covariance matrix 𝑹k\bm{R}_{k}. These are defined straightforwardly from Equations (5)–(8). Specifically, 𝑯k\bm{H}_{k} is a diagonal matrix where the nn-th component of the diagonal is given by g(n)(tk)g^{(n)}(t_{k}) from Equation (8). The measurement covariance satisfies 𝑹k=E[𝒗𝒗]=σm2\bm{R}_{k}=E\left[\bm{v}\bm{v}^{\intercal}\right]=\sigma^{2}_{\rm m} for all kk.

Appendix C Nested sampling

We can use the likelihood returned by the Kalman filter, Equation (38), in conjunction with likelihood-based inference methods to estimate the posterior distribution of 𝜽\bm{\theta} by Bayes’ Rule,

p(𝜽|𝒀)=(𝒀|𝜽)π(𝜽)𝒵,p(\bm{\theta}|\bm{Y})=\frac{\mathcal{L}(\bm{Y}|\bm{\theta})\pi(\bm{\theta})}{\mathcal{Z}}\ , (54)

where π(𝜽)\pi(\bm{\theta}) is the prior distribution on 𝜽\bm{\theta} and 𝒵\mathcal{Z} is the marginalised likelihood, or evidence,

𝒵=𝑑𝜽(𝒀|𝜽)π(𝜽).\mathcal{Z}=\int d\bm{\theta}\mathcal{L}(\bm{Y}|\bm{\theta})\pi(\bm{\theta})\ . (55)

We estimate the posterior distribution and the model evidence through nested sampling (Skilling, 2006) in this paper. Nested sampling evaluates marginalised likelihood integrals, of the form given by Equation (55). It also approximates the posterior by returning samples from p(𝜽|𝒀)p(\bm{\theta}|\bm{Y}). It does so by drawing a set of nliven_{\rm live} live points from π(𝜽)\pi(\bm{\theta}) and iteratively replacing the live point with the lowest likelihood with a new live point drawn from π(𝜽)\pi(\bm{\theta}), where the new live point is required to have a higher likelihood than the discarded point. The primary advantage of nested sampling is its ability to compute 𝒵\mathcal{Z}, on which model selection relies, as in Sections 3 and 6.2. Nested sampling is also computationally efficient and can handle multi-modal problems (Ashton et al., 2022). For these reasons, it has enjoyed widespread adoption in the physical sciences, particularly within the cosmological community (Mukherjee et al., 2006; Feroz & Hobson, 2008; Handley et al., 2015), neutron star astrophysics (Meyers et al., 2021a; Meyers et al., 2021b; Melatos et al., 2023), particle physics (Trassinelli, 2019) and materials science (Pártay et al., 2009). For reviews of nested sampling we refer the reader to Buchner (2021a) and Ashton et al. (2022). Multiple nested sampling algorithms and computational libraries exist (e.g. Feroz & Hobson, 2008; Feroz et al., 2009; Handley et al., 2015; Speagle, 2020; Buchner, 2021b). In gravitational wave research it is common to use the dynesty sampler (Speagle, 2020) via the Bilby (Ashton & Talbot, 2021) front-end library. We follow this precedent and use Bilby for all nested sampling Bayesian inference in this work.

The primary tunable parameter in nested sampling is nliven_{\rm live}. More live points address larger parameter spaces and multi-modal problems, whilst the uncertainties in the evidence and the posterior scale as 𝒪(nlive1/2)\mathcal{O}\left(n_{\rm live}^{-1/2}\right). However the computational runtime scales as 𝒪(nlive)\mathcal{O}(n_{\rm live}). Ashton et al. (2022) offered a rule-of-thumb trade-off, where the minimum number of live points should be greater than the number of static parameters. Informal empirical tests conducted as part of this paper support the trade-off suggested by Ashton et al. (2022); we find typically that the true 𝜽{\bm{\theta}} is contained within the 90% credible interval of the one-dimensional marginalised posteriors of 𝜽^{\bm{\hat{\theta}}} for nlive>7+5Nn_{\rm live}>7+5N with N50N\leq 50. Unless stated otherwise we take nlive=2000n_{\rm live}=2000 conservatively for all results presented in this work.

Appendix D Workflow summary

For the reader’s convenience we now summarise the workflow for a representative PTA analysis using the Kalman filter and nested sampler for parameter estimation and model selection:

  1. 1.

    Specify a PTA composed of NN pulsars

  2. 2.

    Obtain NN data inputs fm(n)(t)f_{\rm m}^{(n)}(t), collectively labelled 𝒀\bm{Y}

  3. 3.

    Specify a state-space model \mathcal{M}, with static parameters 𝜽\bm{\theta}

  4. 4.

    Specify prior distribution π(𝜽)\pi(\bm{\theta})

  5. 5.

    Sample nliven_{\rm live} points from π(𝜽)\pi(\bm{\theta})

  6. 6.

    For each live point:

    1. (a)

      Pass the sample 𝜽sample\bm{\theta}_{\rm sample} to the Kalman filter

    2. (b)

      Iterate over the input data using the Kalman filter and obtain a single log\log\mathcal{L} value through Equation (38)

  7. 7.

    Remove the live point with the lowest likelihood value, loglowest\log\mathcal{L}_{\rm lowest}

  8. 8.

    Sample a new live point from π(𝜽)\pi(\bm{\theta}), subject to the requirement that the new likelihood obeys new\mathcal{L}_{\rm new} > lowest\mathcal{L}_{\rm lowest}, where lognew\log\mathcal{L}_{\rm new} is calculated via steps (vi)(a)–(vi)(b).

  9. 9.

    Update p(𝜽|𝒀)p\left(\bm{\theta}|\bm{Y}\right) and 𝒵\mathcal{Z} with nested sampler

  10. 10.

    Repeat steps (vii)–(ix) until convergence criteria are satisfied.

In order to compute the odds ratio β\beta the above workflow is repeated for a different \mathcal{M}. The resulting 𝒵\mathcal{Z} values can then be divided. We remind the reader that the above workflow differs from a realistic PTA analysis in one important respect, namely that the data are input as frequency time series fm(n)(t)f_{\rm m}^{(n)}(t) instead of pulse TOAs. The generalization to TOAs is subtle and will be tackled in a forthcoming paper.

Appendix E Challenges of the pulsar term

Refer to caption
Figure 11: Logarithm of the cross-section along the α\alpha-axis of the likelihood function slice(α)\mathcal{L}_{\rm slice}(\alpha) returned by the Kalman filter acting on synthetic data 𝒀\bm{Y}. All static elements of 𝜽\bm{\theta} other than α\alpha are held fixed at their true injected values for the purpose of the tests in Appendix E. In the left panel the Kalman filter uses Equation (8) to calculate slice(α)\mathcal{L}_{\rm slice}(\alpha), i.e. inclusive of the pulsar terms. In the right-hand panel the Kalman filter uses Equation (15), i.e. just the Earth terms. The vertical dashed lines indicate the true injected value α=1.0\alpha=1.0 rad used to generate 𝒀\bm{Y}. The inset in the left panel covers the region 1.0104α1.0+1041.0-10^{-4}\leq\alpha\leq 1.0+10^{-4} rad. In the left panel slice(α)\mathcal{L}_{\rm slice}(\alpha) is jagged, although a unique peak exists in principle at α=1.0rad\alpha=1.0\,{\rm rad}, and slice(α)\mathcal{L}_{\rm slice}(\alpha) is smooth within the neighbourhood plotted in the inset. The jagged form of slice(α)\mathcal{L}_{\rm slice}(\alpha) hampers the convergence of the nested sampler. In the right panel slice(α)\mathcal{L}_{\rm slice}(\alpha) is smooth for 0απ0\leq\alpha\leq\pi. The green (blue) curve is the likelihood returned by psr&Earth\mathcal{M}_{\rm psr\&Earth} (Earth\mathcal{M}_{\rm Earth}).

In this appendix we elucidate the complications introduced by the pulsar terms into the state-space analysis and justify the reparametrisation in terms of χ(n)\chi^{(n)} introduced in Section 3.2.

As a first step, we demonstrate empirically that sticking with the original parameterization of the pulsar terms in Equation (8), i.e. writing them in terms of d(n)d^{(n)} instead of χ(n)\chi^{(n)}, produces a highly oscillatory likelihood function, which the nested sampler fails to navigate successfully. Physically, this happens because the Doppler factor depends on the GW wavelength, and there are 103\gtrsim 10^{3} GW wavelengths across the Galactic volume spanned by the PTA. To enact the demonstration, we generate noisy synthetic data 𝒀\bm{Y} as outlined in Section 4. The SMBHB source parameters 𝜽gw\bm{\theta}_{\rm gw} are chosen arbitrarily, with α=1.0\alpha=1.0 rad. We run the Kalman filter using the measurement equation inclusive of the pulsar terms, Equation (8), for 0απ0\leq\alpha\leq\pi. All other elements of 𝜽gw\bm{\theta}_{\rm gw} are fixed at their true injected values. At this point we do not use the χ(n)\chi^{(n)} parametrisation. Therefore changing α\alpha also changes 𝒏\bm{n} everywhere it appears in Equation (8). For each value of α\alpha, the Kalman filter returns a likelihood (𝒀|α)=slice(α)\mathcal{L}(\bm{Y}|\alpha)=\mathcal{L}_{\rm slice}(\alpha), i.e. a cross-section of (𝜽gw)\mathcal{L}(\bm{\theta}_{\rm gw}) along the α\alpha-axis for the purpose of testing.

The function slice(α)\mathcal{L}_{\rm slice}(\alpha) is displayed in the left panel of Figure 11 (orange curve). The dashed grey line indicates the injected value α=1.0\alpha=1.0 rad. The inset shows a zoomed-in section of (α)\mathcal{L}(\alpha) within the region 1.0104α1.0+1041.0-10^{-4}\leq\alpha\leq 1.0+10^{-4} rad. There are two important features in the left panel. First, on scales of the order the width of the prior π(α)\pi(\alpha) (i.e radians), the curve slice(α)\mathcal{L}_{\rm slice}(\alpha) is jagged. Similar jagged curves are obtained for slice(Ω)\mathcal{L}_{\rm slice}(\Omega) and slice(δ)\mathcal{L}_{\rm slice}(\delta) (not plotted for brevity). For the other static parameters, the curves are smooth and globally concave. Second, despite being jagged, a true likelihood maximum coincident with the injected value does exist and is locally concave in a neighbourhood spanning 104rad\sim 10^{-4}\,{\rm rad}. Hence it is theoretically possible to use likelihood-based methods for Bayesian estimation of α\alpha without reparameterizing in terms of χ(n)\chi^{(n)}. In practice, however, the computation is intractable and does not converge for reasonable choices of nliven_{\rm live}. The challenge for sampling algorithms is exacerbated when we extend the above test from slice(α)\mathcal{L}_{\rm slice}(\alpha) to the α\alpha-δ\delta-Ω\Omega subspace or the full 7+5N7+5N domain in Section 2.3.

Why do α\alpha, δ\delta and Ω\Omega exhibit jagged likelihood curves whereas other static parameters do not? The culprit is the Doppler factor flagged above: α\alpha, δ\delta, and Ω\Omega appear in the phase term Ω[1+𝒏𝒒(n)]d(n)\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)}, with 𝒏=𝒏(δ,α)\bm{n}=\bm{n}(\delta,\alpha). For d(n)1d^{(n)}\sim 1 kpc, a 10 nHz GW accumulates 103\gtrsim 10^{3} cycles. The inference problem becomes multiply degenerate, once the cosine of the phase is calculated (modulo 2π2\pi).

In order to smooth the likelihood function, one can try two things. The first is to drop the pulsar term completely, i.e. using Earth\mathcal{M}_{\rm Earth}. This is the approach taken in K24 and some (not all) other standard PTA analyses (e.g. Sesana & Vecchio, 2010; Babak & Sesana, 2012a; Petiteau et al., 2013; Zhu et al., 2015; Taylor et al., 2016; Goldstein et al., 2018; Charisi et al., 2023). The second is to reparameterize the pulsar terms such that the phase term Ω[1+𝒏𝒒(n)]d(n)\Omega\left[1+\bm{n}\cdot\bm{q}^{(n)}\right]d^{(n)} is no longer a function of α\alpha (or Ω\Omega or δ\delta), but an independent parameter χ(n)\chi^{(n)} to be estimated. This is the approach presented in this paper, i.e. using psr&Earth\mathcal{M}_{\rm psr\&Earth}, as in Section 3.2. The right panel of Figure 11 displays slice(α)\mathcal{L}_{\rm slice}(\alpha) calculated using Earth\mathcal{M}_{\rm Earth} (blue curve) and psr&Earth\mathcal{M}_{\rm psr\&Earth} (green curve). In both instances there is no ambiguity about the number of cycles the wave has gone through over a distance d(n)d^{(n)}, and the likelihood function is smooth.

Appendix F Priors

To deploy the nested sampling outlined in Appendix C and the workflow in Appendix D it is necessary to specify a Bayesian prior π(𝜽)\pi(\bm{\theta}) on the static parameters. In this section we outline how the priors are chosen. The priors on each static parameter are summarised in Table 1.

For π(𝜽gw)\pi(\bm{\theta}_{\rm gw}) we assume no a priori information about the parameters. We therefore choose standard non-informative priors (e.g. Bhagwat et al., 2021). For π(𝜽psr)\pi(\bm{\theta}_{\rm psr}) a priori information from electromagnetic observations does exist. We adopt constrained uniform priors on fem(n)(t1)f_{\rm em}^{(n)}(t_{1}) and f˙em(n)(t1)\dot{f}_{\rm em}^{(n)}(t_{1}), which extend ±103ηf(n)\pm 10^{3}\eta_{f}^{(n)} and ±103ηf˙(n)\pm 10^{3}\eta_{\dot{f}}^{(n)} respectively about the central, injected values, where ηf(n)\eta_{f}^{(n)} and ηf˙(n)\eta_{\dot{f}}^{(n)} denote the errors quoted in the ATNF Pulsar Database. By using wider-than-necessary priors we expose the analysis scheme to a more stringent test. We set an uninformative broad prior π[σ(n)/(1s3/2)]\pi[\sigma^{(n)}/(1\,{\rm s^{-3/2}})]\sim LogUniform[102σSC(n),102σSC(n)]\left[10^{-2}\sigma_{\rm SC}^{(n)},10^{2}\sigma_{\rm SC}^{(n)}\right] where σSC(n)\sigma_{\rm SC}^{(n)} is the noise amplitude for pulsar nn inferred from Equation (19). We set a non-informative prior on the new static parameter χ(n)Uniform(0,2π)\chi^{(n)}\sim\text{Uniform}\left(0,2\pi\right). We do not set a prior on γ(n)\gamma^{(n)}, because one typically has γ(n)Tobs105\gamma^{(n)}T_{\rm obs}\sim 10^{-5} astrophysically, and γ(n)\gamma^{(n)} is effectively “unobservable” for Tobs10yearsT_{\rm obs}\sim 10\,{\rm years}. For validation purposes it is sufficient to carry γ(n)\gamma^{(n)} through the analysis at its injected value. This reduces the total dimension of the parameter space to 7+4N7+4N.

Appendix G High-SNR example

In this appendix we repeat the analysis of Section 5 in the high-SNR regime. We do so as a sanity check to confirm that the state-space analysis scheme with the pulsar terms included returns reasonable results in the “easy” case with a strong GW signal. The high-SNR limit allows us to check for estimation biases unobscured by measurement noise.

We apply the parameter estimation framework as in Section 5. All injected static parameters 𝜽\bm{\theta}^{\prime}, and the associated priors are specified in Table 1, with the exception of h0h_{0} which now takes the value h0=1×1012h_{0}=1\times 10^{-12}. The procedure is undertaken for ten realisations of the noise processes ξ(n)(t)\xi^{(n)}(t) and ε(n)(t)\varepsilon^{(n)}(t).

Figure 12 displays results for the seven parameters in 𝜽gw\bm{\theta}_{\rm gw} for ten arbitrary noise realisations in the form of a traditional corner plot. The figure is exactly analogous to Figure 1. All histograms and contours are consistent with a unimodal joint posterior, which peaks near the known, injected values. There is no evidence of railing against the prior bounds. Correlations between ι\iota and h0h_{0} are evident over multiple noise realisations due to the weak identifiability 333By weak identifiability in this context we mean that the ι\iota-h0h_{0} likelihood contours plateau to a flat-topped ridge with a small, non-zero gradient close to the maximum likelihood solution. of these two parameters (Bellman & Åström, 1970). For five of the seven parameters, there is effectively zero dispersion in the one-dimensional posterior medians between noise realisations. For the remaining two parameters, ι\iota and h0h_{0}, appreciable dispersion is observed, again due to weak-identifiability.

Figure 13 displays results for a representative subset of five out of 47 pulsar phases χ(n)\chi^{(n)}, {χ(1)χ(5)}\{\chi^{(1)}\dots\chi^{(5)}\} for the ten noise realisations. The figure is exactly analogous to Figure 2. All histograms and contours are consistent with a unimodal joint posterior, which peaks near the known, injected values, with no evidence for correlations between parameter pairs. Unlike in the low-SNR regime, it is now possible to infer χ(2)\chi^{(2)} (the phase correction for PSR J0340+4130) consistently (third row, third column of Figure 13). Even though PSR J0340+4130 nearly coincides on the sky with the synthetic GW source, the GW signal is loud enough for χ(2)\chi^{(2)} to be estimated.

Refer to caption
Figure 12: Same as Figure 1, but for a loud SMBHB with h0=1×1012h_{0}=1\times 10^{-12}. The Kalman filter and nested sampler estimate all seven parameters in 𝜽gw\bm{\theta}_{\rm gw} accurately. There is less dispersion in the one-dimensional posterior medians among noise realisations compared to the low-SNR case (c.f. Figure 1). An appreciable dispersion remains for ι\iota and h0h_{0}, which are correlated and weakly identifiable.
Refer to caption
Figure 13: Same as Figure 2, but for a loud SMBHB with 1×h0=10121\times h_{0}=10^{-12}. Consistent unimodal posteriors are obtained across all noise realisations and all five displayed parameters. It is now possible to infer χ(2)\chi^{(2)} due to the loundness of the GW signal, unlike in the low-SNR case in Figure 2. The colour scheme is the same as in Figure 2, but all the curves overlap and so are obscured by the orange curves.