Residual dependence structures¶
brms adds residual dependence to a formula with autocorrelation terms (Bürkner, 2017). QVR parses the same terms, fits them with the existing SVI, HMC, and NUTS drivers, and replays them through the shared prediction, likelihood, refit, and artifact paths. The term changes the response law. It does not add a predictor column.
| Term | Structure | Parameters (prior class) | Default prior |
|---|---|---|---|
ar(time, gr, p), ma(time, gr, q), arma(time, gr, p, q) |
conditional ARMA of order (p, q) | ar_1, ..., ma_1, ... (ar, ma) |
Normal(0, 1) |
the same with cov = TRUE |
stationary ARMA covariance, orders at most one | ar_1, ma_1 (ar, ma) |
Uniform(-1, 1) |
cosy(time, gr) |
compound symmetry | cosy |
Uniform(0, 1) |
unstr(time, gr) |
unstructured correlation over time levels | Lcortime (Lcortime) |
LKJCholesky(1) |
sar(M, type = "lag"), sar(M, type = "error") |
simultaneous autoregression | lagsar, errorsar |
uniform between the extreme eigenvalues of M |
fcor(M) |
fixed covariance M scaled by sigma |
none | none |
car(M, gr, type) with type one of escar, esicar, icar, bym2 |
conditional autoregressive location effect | sdcar, car (escar), rhocar (bym2) |
random_scale_prior, Uniform(0, 1), Beta(1, 1) |
For family="gaussian" and family="student_t", the residual terms change
the response law, as brms's natural-residual formulation does. For every
other family with one location predictor, brms adds latent residuals to the
linear predictor instead, and QVR does the same (see below). A car term
adds a spatial effect to the linear predictor of any such family. Matrices
named in sar, fcor, and car come from the data2 argument, as in brms.
As in brms, a formula may combine one time-series term, one spatial term,
and at most one covariance of natural residuals. A conditional ARMA term
holds no covariance, so it combines with sar or with fcor, and a car
term combines with any residual term. Every other pair of residual terms is
refused with brms's message.
A first-order autoregression¶
import numpy as np
import pandas as pd
from quivers.formulas import PriorSpec, fit, get_prior
rows = []
for series in ("a", "b"):
noise = 0.0
for time in range(8):
noise = 0.6 * noise + 0.3 * np.sin(3.0 * time + len(series))
rows.append({"g": series, "t": time, "x": time / 8, "y": 0.5 * time / 8 + noise})
data = pd.DataFrame(rows)
assert "ar_1" in get_prior("y ~ x + ar(time = t, gr = g)", data=data)
model = fit(
"y ~ x + ar(time = t, gr = g)",
data=data,
method="svi",
num_steps=50,
seed=1,
priors=PriorSpec(distribution="Normal(0.0, 0.5)", class_="ar"),
)
forecast = pd.DataFrame(
{"g": "a", "t": [8, 9, 10], "x": [1.0, 1.125, 1.25], "y": [0.4, np.nan, np.nan]}
)
draws = model.posterior_predict(new_data=forecast, num_samples=20, seed=2)
assert draws.shape == (1, 20, 3)
Time ordering and groups¶
As in brms, rows are ordered by the grouping factor gr and then by time
within each group. Without time, the data order defines time. Without gr,
all rows form one series. Time points must be unique within a group. Groups
may have different lengths. The fitted rows keep their original order in
every prediction and likelihood result.
The conditional ARMA formulation adds lagged residuals to the linear
predictor, so its rows need no positions beyond their order. The covariance
formulations (cov = TRUE, cosy) assign positions one to n within each
group, and every group uses the leading block of one correlation matrix.
unstr instead indexes its correlation matrix by the observed time values,
so a group missing a time level uses the matching sub-block.
Student responses¶
A conditional ARMA model with family="student_t" gives every row an
independent Student innovation. A covariance formulation or a SAR term uses
one multivariate Student law
per correlated series, so all rows of a series share one mixing scale. This
follows brms's student_t_time and student_t_lagsar likelihoods.
Prediction¶
posterior_epred, posterior_predict, posterior_linpred, and
log_likelihood follow brms.
- For the conditional ARMA formulation, the expected response and
muinclude the ARMA terms computed from response history. Predictions on the fitted rows condition on the observed responses, one step ahead. - New rows form their own series, as brms's
newdatadoes. Rows whose response is present serve as history. A row with a missing or NaN response is predicted, and a draw from its predictive law becomes the history of the following rows. brms'soosargument corresponds to setting those rows' responses to NaN. Each returned draw is a separate draw from the row's predictive law, as in brms. - Covariance formulations predict each series jointly. Their expected response is the linear predictor.
- A lag SAR expectation is \((I-\rho W)^{-1}\mu\). SAR and
fcorpredictions on new rows read a matrix over those rows from thedata2argument of the prediction method, brms'snewdata2. incl_autocor=Falsealso removescareffects and latent residuals.incl_autocor=Falseremoves the structure: rows become independent with the fitted residual scale.unstrrefuses time values that were not fitted.
log_likelihood returns brms's pointwise values. A conditional ARMA row
scores its one-step conditional density. A covariance or SAR row scores its
leave-one-out predictive density given all other rows of its series
(Bürkner et al., 2021), the value that PSIS-LOO requires for these
non-factorized models. Those values do not sum to the joint density. The
fitted target always uses the exact joint density.
Combining terms¶
A conditional ARMA term with sar or fcor¶
brms adds the conditional ARMA terms to the linear predictor and then scores
the whole response vector under the SAR or fixed-covariance law. With
\(\mu_n=\eta_n+\sum_k\theta_k e_{n-k}+\sum_k\phi_k e_{n-k}\) and
\(e_n=y_n-\eta_n-\sum_k\theta_k e_{n-k}\) computed along each series, a lag SAR
scores \((I-\rho W)y-\mu\), an error SAR scores \((I-\rho W)(y-\mu)\), and fcor
scores \(y-\mu\) under covariance \(\sigma^2 M\). The lag SAR expectation is
\((I-\rho W)^{-1}\mu\), and log_likelihood returns the leave-one-out
densities of the SAR or fixed-covariance law around \(\mu\).
import numpy as np
import pandas as pd
from quivers.formulas import fit
rows = [
{"g": series, "t": time, "x": np.cos(1.7 * time + shift)}
for shift, series in enumerate(("a", "b"))
for time in range(5)
]
data = pd.DataFrame(rows)
data["y"] = 0.4 * data.x + 0.3 * np.sin(2.0 * np.arange(10))
W = np.zeros((10, 10))
for i in range(9):
W[i, i + 1] = W[i + 1, i] = 1.0
W = W / W.sum(axis=1, keepdims=True)
model = fit(
"y ~ x + ar(time = t, gr = g) + sar(W)",
data=data,
data2={"W": W},
method="svi",
num_steps=20,
seed=1,
)
assert {"ar_1", "lagsar"} <= set(model.draws(num_samples=1))
fresh = data.iloc[:5]
expected = model.posterior_epred(
new_data=fresh, data2={"W": W[:5, :5] / W[:5, :5].sum(axis=1, keepdims=True)},
num_samples=4,
)
assert expected.shape == (1, 4, 5)
Every row's response must be present when these terms are combined. brms draws a missing response from the joint law of all rows, which defines no draw for one row, so QVR refuses missing history here.
Known standard errors with a time-series covariance¶
y | se(s, sigma = TRUE) ~ ... + cosy(time, gr), and the same with
unstr or an ARMA term with cov = TRUE, adds the squared standard errors
to the diagonal of each series covariance,
\(\Sigma_g=\operatorname{diag}(\sigma)\,R\,\operatorname{diag}(\sigma)+\operatorname{diag}(s^2)\),
brms's normal_time_hom_se and student_t_time_hom_se likelihoods. The
measurement errors thus stay independent across the rows of a series. brms
requires sigma = TRUE here, and so does QVR. Each series then has its own
covariance factor, O(n_g^3) for a series of n_g rows. New rows read their
standard errors from the prediction frame, and incl_autocor=False keeps
them in the scale \(\sqrt{\sigma^2+s^2}\) of independent rows.
data["s"] = 0.1 + 0.05 * data.t
known = fit(
"y | se(s, sigma = TRUE) ~ x + cosy(time = t, gr = g)",
data=data,
method="svi",
num_steps=20,
seed=1,
)
assert known.log_likelihood(num_samples=2)["y"].shape == (1, 2, 10)
Censoring, truncation, and weights with a conditional ARMA term¶
The rows of the conditional ARMA formulation are conditionally independent given the earlier responses, so brms adjusts each row. The residuals \(e_n\) use the recorded response of every row, which for a censored row is its recorded endpoint (the lower endpoint of an interval). A censored row then contributes a tail or interval mass of its conditional law \(F(\cdot\mid\mu_n,\sigma)\), a truncated row divides by the mass of its truncation interval, and a weight multiplies the row's log density.
data["status"] = np.where(data.y > 0.3, "right", "none")
censored = fit(
"y | cens(status) + trunc(lb = -5, ub = 5) ~ x + ar(time = t, gr = g)",
data=data,
method="svi",
num_steps=20,
seed=1,
)
draws = censored.posterior_predict(num_samples=3, seed=2)
assert bool(((draws >= -5) & (draws <= 5)).all())
Prediction draws each row from its truncated conditional law. A missing response among new rows is drawn from that truncated law before it enters the history of later rows, as in brms.
Refits and cross-validation¶
update and refit_callback reuse the retained recipe, including data2.
Fold refits recompute the series from the training rows of each fold, and
the held-out rows form their own series when scored, as brms's kfold does
through its newdata path. brms places no restriction on the folds of these
models. A fold that removes rows from inside a series joins the remaining
rows as if they were consecutive, because the ARMA terms use row order
rather than time values. Folds that hold out whole series by gr avoid
this, and they are the folds QVR's tests use for time-series terms. A SAR or
fcor matrix covers all fitted rows, so partitioned refits of those terms
fail with the dimension error that brms also raises. save_fit stores the data2 matrices with the other fitting
options, and load_fit rebuilds the same response law.
Cost¶
Scoring the conditional ARMA formulation costs O(n (p + q)) for n rows. The
stationary ARMA and compound-symmetry formulations factor one correlation of
the longest series, O(m^3) for m positions, and then solve each series. A
SAR term computes the eigenvalues of M once when the model is prepared.
Each evaluation then costs O(nnz) sparse products plus an O(n) log
determinant \(\sum_i \log(1-\rho\lambda_i)\) (Ord, 1975), and never
forms the dense covariance. Lag SAR prediction solves
\((I-\rho W)x=b\) by sparse LU.
Conditional autoregressive effects¶
car(M, gr = loc, type = ...) adds rcar[loc] to the linear predictor
(Morris et al., 2019). M is an adjacency matrix over locations, passed as a
pandas DataFrame whose index and columns are the location labels, which must
cover the levels of gr. Without gr, every row is a location and M is
indexed by row. Every nonzero entry marks two neighbors. M must be square,
symmetric, and zero on the diagonal, and escar and esicar require every
location to have a neighbor, as brms requires.
escardrawsrcarfrom the proper CAR law with precision \(\tau(D-\alpha W)\), where \(D\) holds neighbor counts, \(\tau=1/\mathrm{sdcar}^2\), and \(\alpha\) iscar.esicarandicaruse the intrinsic law with precision \(\tau(D-W)\).bym2sets \(\mathrm{rcar}=\mathrm{sdcar}\,(\sqrt{1-\rho}\,\theta+\sqrt{\rho/s}\,\phi)\) with \(\theta\sim\mathcal N(0,I)\), an intrinsic \(\phi\), \(\rho\) =rhocar, and the scaling factor \(s\) (Riebler et al., 2016).
The intrinsic effects sum to zero within every connected component of the
graph, exactly: the sampler moves the free coordinates and the last location
of each component is minus the sum of the others. The scaling factor \(s\) is
the geometric mean of the diagonal of the generalized inverse of the graph
Laplacian, which is the
marginal variance of the intrinsic law under that constraint. Scoring uses
the sparse edge list, O(n + E) for n locations and E edges. The eigenvalues
needed by escar and the scaling factors are computed once, when the model
is prepared.
import numpy as np
import pandas as pd
from quivers.formulas import fit
labels = [f"L{i}" for i in range(5)]
adjacency = np.zeros((5, 5))
for i in range(4):
adjacency[i, i + 1] = adjacency[i + 1, i] = 1.0
M = pd.DataFrame(adjacency, index=labels, columns=labels)
data = pd.DataFrame({"loc": labels * 2, "x": np.linspace(-1, 1, 10)})
data["y"] = [0, 1, 2, 1, 3, 1, 0, 2, 4, 2]
model = fit(
"y ~ x + car(M, gr = loc, type = 'bym2')",
data=data,
data2={"M": M},
family="poisson",
method="svi",
num_steps=20,
)
rates = model.posterior_epred(
new_data=pd.DataFrame({"loc": ["L4"], "x": [0.5]}), num_samples=5
)
assert rates.shape == (1, 5, 1)
Prediction maps rows to fitted locations. New locations, and new rows of a
car term without gr, are refused with brms's messages.
Latent residuals for other families¶
For a family without natural residuals, such as poisson or bernoulli,
brms adds a latent vector err with scale sderr to the linear predictor.
A conditional ar term draws err independently and adds
\(\sum_k \phi_k\,\mathrm{err}_{t-k}\), so ma terms there need cov = TRUE. A
covariance term (cov = TRUE, cosy, unstr) correlates err within each
series. The ARMA coefficients are bounded by \([-1, 1]\) and cosy by
\([0, 1]\), as brms bounds them, and sderr takes random_scale_prior. sar
and fcor need natural residuals and are refused, as in brms. The fitted
err values are replayed on the training rows. New rows of a covariance
term draw fresh residuals from the structure at each posterior draw, as
brms does, and new rows of the conditional formulation are refused with
brms's message.
Where QVR differs from brms¶
- brms gives the conditional ARMA coefficients an improper flat prior, and
so does
default_priors="brms". The default quivers policy uses the properNormal(0, 1), which keeps the same unbounded support and leaves prior draws,sample_prior, and SMC evidence available; these refuse a fit with flat priors. - brms's SAR likelihoods omit the constants \(-\tfrac{n}{2}\log(2\pi)\) and \(-\tfrac{n}{2}\log\pi\). QVR scores the normalized densities, so their values differ by a constant.
- A
sarorfcormatrix indexes the rows ofdataas given. brms sorts the rows of a model with a time-series term by group and time and leaves the matrix as given, so in brms the matrix rows of a conditional ARMA term besidesarorfcorare the sorted rows. The two agree for data sorted bygrand thentime. QVR keeps the row identities of the data, because a matrix row then names the same observation whatever the row order. - brms's intrinsic CAR (
icar,bym2) uses a soft sum-to-zero over all locations, and itsesicarand BYM2 scaling assume one connected graph. QVR constrains each connected component exactly and scales each component by its own factor, following @riebler2016intuitive. On a connected graph the two agree, up to brms's diagonal jitter of order 1e-8. An isolated location then has no intrinsic effect. - Every refusal of
se(), likelihood weights, censoring, or truncation beside a residual term follows brms. Its time-covariance and FCOR likelihoods reject weights, censoring, and truncation (R/stan-likelihood.R,stan_log_lik_gaussian_timeandstan_log_lik_gaussian_fcor), FCOR rejectsse(), the conditional ARMA formulation rejectsse()(R/stan-predictor.R,stan_ac), and a time-series covariance requiresse(..., sigma = TRUE). For asarterm brms writes each of these response terms into a Stan program that does not compile, because its SAR densities take the whole response vector and one scalarsigma. No brms model exists to match, and QVR refuses those combinations with a message that says so. - brms predicts new rows of a
sarorfcorterm with the fitted matrix whennewdata2is absent and the row counts agree. QVR requires a matrix over the new rows indata2, because an equal row count does not make the new rows the fitted observations. - With a conditional ARMA term beside
sarorfcor, a missing response among the prediction rows is refused, for the reason given above. - A row that is both censored and truncated scores the mass of its censored event inside the truncation interval, as every censored and truncated response does in QVR. brms divides the whole tail mass by the mass of the truncation interval. The two agree for rows that are only censored or only truncated.
Cited sources¶
Paul-Christian Bürkner. brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software, 2017. doi:10.18637/jss.v080.i01.
Paul-Christian Bürkner, Jonah Gabry, and Aki Vehtari. Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models. Computational Statistics, 36(2):1243–1261, 2021. doi:10.1007/s00180-020-01045-4.
Keith Ord. Estimation Methods for Models of Spatial Interaction. Journal of the American Statistical Association, 70(349):120–126, 1975. doi:10.1080/01621459.1975.10480272.
Mitzi Morris, Katherine Wheeler-Martin, Dan Simpson, Stephen J. Mooney, Andrew Gelman, and Charles DiMaggio. Bayesian hierarchical spatial models: Implementing the Besag York Mollié model in stan. Spatial and Spatio-temporal Epidemiology, 31:100301, 2019. URL: https://www.sciencedirect.com/science/article/pii/S1877584518301175, doi:10.1016/j.sste.2019.100301.
Andrea Riebler, Sigrunn H. Sørbye, Daniel Simpson, and Håvard Rue. An intuitive Bayesian spatial model for disease mapping that accounts for scaling. Statistical Methods in Medical Research, 25(4):1145–1165, 2016. doi:10.1177/0962280216660421.