Grouping terms¶
A group-level term (lhs | group) gives each level of a grouping factor its
own coefficients for the columns of lhs. The coefficients of one block share
a multivariate normal distribution
with a standard deviation per coefficient and, for more than one coefficient,
an LKJ correlation matrix (Bürkner, 2017). This guide covers brms's syntax
for the right side of the bar: gr(), interactions and nesting of grouping
factors, and multi-membership terms. The
formula guide
covers the left side, double bars, and shared identifiers.
Options of gr()¶
gr(g, cor, id, cov, by, dist, pw) names the grouping factor g together
with options of its block. (lhs | gr(g)) is (lhs | g).
| Option | Meaning | Bar notation |
|---|---|---|
cor = FALSE |
independent coefficients | (lhs \|\| g) |
id = "i" |
one correlation block for every term with this id and factor |
(lhs \|i\| g) |
cov = M |
known covariance between the levels of g, read from data2 |
group_covariances |
by = f |
separate standard deviations and correlations per level of f |
none |
dist = "student" |
multivariate Student-t coefficients | none |
pw = w |
per-level weight on the prior of the coefficients | none |
gr(g, cov = M) takes M from the data2 argument of fit, get_prior,
and formula_to_qvr. M must be a pandas DataFrame whose row and column
labels name the levels of g, and it may include levels absent from the
training rows. As in brms, the coefficients of level \(j\) become
\(\sigma \, (L z)_j\) with \(L\) the Cholesky factor of M and standard-normal
\(z\), the model a
GroupCovariance for g defines. The fit retains
the covariance, so new levels covered by M are predicted from their
conditional distribution given the fitted levels.
Correlated coefficients of a factor with a known covariance have the
Kronecker product
covariance of M and the coefficient covariance, for every positive
definite M. brms computes them with a loop that skips the entries of \(L\)
that are not above \(10^{-10}\), so its coefficients have that covariance only
when \(L\) has no negative entry. The two models agree for such an M and
for a single or independent coefficients, and they differ deliberately
otherwise.
gr(g, by = f) gives every level of the factor f its own standard
deviation for each coefficient and, in a correlated block, its own LKJ
correlation matrix. Each level of g must lie within one level of f,
and otherwise the formula is refused with brms's message. A new level of g
at prediction takes the level of f named in its fresh rows, which must be a
fitted level.
gr(g, dist = "student") follows brms. It draws degrees of freedom
\(\nu \sim \operatorname{Gamma}(2, 0.1)\) restricted to \(\nu > 1\), of
class df, and for each level \(j\) a mixing variable
\(u_j \sim \text{inverse-}\chi^2(\nu)\), and multiplies the level's Gaussian
coefficients by \(\sqrt{\nu u_j}\). The coefficients of a level then follow a
multivariate Student-t distribution
with \(\nu\) degrees of freedom, which is less sensitive to outlying levels
than the normal (Bürkner, 2017). The mixing variables are level
coordinates, so new levels draw them with the coefficients under every
new-level policy.
gr(g, pw = w) weights the standard-normal prior of level \(j\)'s
innovations by \(w_j\), as brms adds \(w_j \log \phi(z_j)\) to the log
density. w must be finite, nonnegative, and constant within each level.
Weights below one let a level's coefficients move farther from zero, and a
weight of zero removes its prior. With weights the model density is no
longer a normalized prior times a likelihood, so independent prior draws
refuse it, as for any model with a log-density factor.
These options use brms's noncentered innovations and are refused with
reparameterize="centered" and with a known level covariance. In a
categorical or other per-category response model each free category takes
its own group coefficients with these options (see
category-specific formulas).
Every group-level term of one factor must use the same options.
import numpy as np
import pandas as pd
from quivers.formulas import fit, get_prior
rows = np.arange(48)
data = pd.DataFrame(
{
"x": np.sin(rows),
"clinic": np.array(list("abcdef"))[rows % 6],
}
)
data["region"] = np.where(data["clinic"].isin(list("abc")), "north", "south")
data["visits"] = np.where(data["clinic"].isin(list("ab")), 2.0, 1.0)
data["y"] = 0.4 * data["x"] + 0.1 * (rows % 5)
formula = "y ~ x + (1 + x | gr(clinic, by = region, dist = 'student', pw = visits))"
priors = get_prior(formula, data=data)
assert {"df_clinic", "cor_clinic_by1", "cor_clinic_by2"} <= priors.keys()
model = fit(formula, data=data, method="svi", num_steps=20, seed=1)
fresh = pd.DataFrame({"x": [0.0, 1.0], "clinic": ["a", "g"], "region": ["north", "south"]})
draws = model.posterior_epred(new_data=fresh, num_samples=10, seed=2, allow_new_levels=True)
assert draws.shape == (1, 10, 2)
Interactions and nesting¶
(1 | g1:g2) uses the combinations of g1 and g2 as levels. As in brms,
a combination is labeled by pasting the levels with an underscore, so the
levels of a and u form a_u, and only combinations that occur in the
training rows are levels. (1 | g1/g2) nests g2 within g1 and expands to
(1 | g1) + (1 | g1:g2), one block per level of the hierarchy. Prediction
builds the combinations from the source columns of the fresh rows, so an
unseen combination is a new level.
import numpy as np
import pandas as pd
from quivers.formulas import fit
rows = np.arange(36)
data = pd.DataFrame(
{
"x": np.sin(rows),
"school": np.array(["a", "b", "c"])[rows % 3],
"class": np.array(["u", "v", "w"])[(rows // 3) % 3],
}
)
data["y"] = 0.4 * data["x"] + 0.1 * (rows % 5)
labels = ["u", "v", "w"]
similarity = pd.DataFrame(
[[1.0, 0.5, 0.2], [0.5, 1.0, 0.3], [0.2, 0.3, 1.0]], index=labels, columns=labels
)
model = fit(
"y ~ x + (1 | school/class) + (1 + x | gr(class, cov = K, cor = FALSE))",
data=data,
data2={"K": similarity},
method="svi",
num_steps=20,
seed=1,
)
assert set(model.formula.group_levels) == {"school", "school:class", "class"}
fresh = pd.DataFrame({"x": [0.0, 1.0], "school": ["a", "d"], "class": ["u", "w"]})
draws = model.posterior_epred(
new_data=fresh, num_samples=10, seed=2, allow_new_levels=True
)
assert draws.shape == (1, 10, 2)
A coefficient appears once per grouping factor. (1 | g) + (x | g) repeats
the intercept of g in two blocks and is refused, as brms refuses it with
"Duplicated group-level effects are not allowed".
Multi-membership terms¶
A row can belong to several levels of one grouping factor at once, for
instance a pupil who attended two schools. mm(g1, ..., gK) declares such a
multi-membership factor (Browne et al., 2001). Its levels are the union of
the levels of the member columns g1 to gK, and a group-level coefficient
\(r\) enters row \(n\) as
where \(J_{kn}\) is the level in column gk, \(W_{kn}\) its weight, and
\(Z_{kn}\) the row's covariate for that member. By default every member has
weight \(1/K\). weights = cbind(w1, ..., wK) takes the weights from columns,
normalized to sum to one in each row unless scale = FALSE, which also
permits negative weights. A covariate x enters every member with the same
value. mmc(x1, ..., xK) gives each member its own covariate, as in
(1 + mmc(x1, x2) | mm(g1, g2)). mm() also takes cor, id, and cov
with their gr() meaning.
import numpy as np
import pandas as pd
from quivers.formulas import fit
rows = np.arange(40)
data = pd.DataFrame(
{
"first": np.array(["a", "b", "c", "d"])[rows % 4],
"second": np.array(["b", "c", "d", "e"])[(rows // 4) % 4],
"years1": 1.0 + rows % 3,
"years2": 1.0 + rows % 2,
"x1": np.cos(rows),
"x2": np.sin(rows),
}
)
data["y"] = 0.3 * data["x1"] + 0.05 * (rows % 7)
model = fit(
"y ~ x1 + (1 + mmc(x1, x2) | mm(first, second, weights = cbind(years1, years2)))",
data=data,
method="svi",
num_steps=20,
seed=1,
)
assert model.formula.group_levels["mmfirstsecond"] == ("a", "b", "c", "d", "e")
fresh = data.iloc[:3].assign(second=["e", "z", "a"])
draws = model.posterior_epred(
new_data=fresh, num_samples=10, seed=2, allow_new_levels=True
)
assert draws.shape == (1, 10, 3)
The factor is named by brms's label, mm followed by the member names
(mmfirstsecond), which PriorSpec(class_="sd", group=...) selects. The
coefficients use the ordinary Gaussian group layout, correlation blocks,
centered and noncentered coordinates, and new-level policies. Each member
gathers the shared coefficient vector at its own level, so scoring and
prediction cost O(n K) per coefficient for n rows, without a rows-by-levels
matrix. Fresh rows are coded on one prediction axis that covers every
member, and a level unseen in training is a new level for every member that
names it. Member columns may not be missing. mm(g1, g2, cov = M) gives
the shared coefficient vector the known covariance M over the union of
the member levels, as gr(g, cov = M) does for one factor, and a member
level that is absent from the training rows and covered by M is predicted
from its conditional distribution. Multi-membership factors take scalar
group coefficients only: ordinal category-specific coefficients, such as
(cs(1) | mm(g1, g2)), which brms admits, and grouped thresholds are
refused. A
categorical or other per-category response gives each free category its own
scalar multi-membership coefficients (see
category-specific formulas).
Cited sources¶
Paul-Christian Bürkner. brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software, 2017. doi:10.18637/jss.v080.i01.
William J. Browne, Harvey Goldstein, and Jon Rasbash. Multiple Membership Multiple Classification (MMMC) Models. Statistical Modelling, 1(2):103–124, 2001. doi:10.1177/1471082X0100100202.