quivers.formulas.compile

The FormulaToQVRModule lens: Formula → quivers.dsl.ast_nodes.Module.

compile

Bidirectional didactic.api.Lens from Formula to a QVR quivers.dsl.ast_nodes.Module AST.

The compilation from a formula to a QVR program is a panproto-style lens whose complement is the strict subset of Formula fields that are not recoverable from the emitted Module, packaged as FormulaData. The forward direction produces (module, formula_data) where the structural fields of the formula (which columns exist, intercept flag, random effect group / slope structure, response identifier) are encoded as QVR latent / let / observe shape and the un-encodable fields (per-row data, original identifier names lost to _qvr_name's non-alphanumeric → underscore substitution, the term / name presentation labels, the original formula string) ride in FormulaData. The backward direction recovers the structure by calling _decode_module on the target and fuses it with the complement.

The lens never touches strings: the target is a Module that the existing quivers.dsl.compiler.Compiler consumes directly, identical in shape to one produced by quivers.dsl.parser.parse.

Emitted structure (one named scalar coefficient per design-matrix column, matching the brms / lme4 canonical layout in which poly(x, 2) produces two coefficients poly(x, 2)_1 and poly(x, 2)_2):

• One object Resp : N declaration per response plate.
• One object G : K declaration per random-effect grouping factor (with K levels).
• For each fixed-design column c: one scalar latent draw inside the program body, beta_c <- Normal(0, fixed_prior), whose scale is autoscaled by 1 / rms(c) so the nominal value reads in contribution space and means the same thing on a raw predictor and on an orthonormal poly column. An explicit per-name prior is emitted as written. The per-row covariate values for c flow in as a free variable via the host-data channel (observations[c]).
• For each random-effect group (slope | g): a HalfNormal scale latent plus a per-level plate draw, with the per-row contribution as a plate-gather alpha_g[g_idx] (or beta_g_slope[g_idx] * slope for a random slope). Multiple random slopes per group are emitted as independent random-effect terms (the uncorrelated / (... || g) semantics in lme4).
• One observe step closes the program with the family's observation kernel applied to the inverse-link of the linear predictor.
• Categorical responses use category zero as an identified reference and lift every fixed and random term over the remaining K - 1 logits before applying softmax.
• Gaussian mixtures use a configurable component axis, shared softmax weights, centered component offsets around the formula predictor, and one positive scale per component.

GetPut holds for every Formula f: backward(*forward(f)) == f.

FormulaToQVRModule

FormulaToQVRModule(family: Family, *, fixed_prior: str = 'Normal(0.0, 5.0)', random_scale_prior: str = 'HalfNormal(1.0)', user_priors: Mapping[str, str] | None = None, reparameterize: Literal['centered', 'noncentered'] = 'noncentered', binomial_trials: int | str = 1, thresholds_by: str | None = None, mixture_components: int = 2, predictor_name: str | None = None)

Bases: Lens[Formula, Module, FormulaData]

Translate a Formula to a QVR Module AST.

A typed didactic.api.Lens whose complement is a FormulaData carrier containing the fields of the source Formula that are not recoverable from the emitted Module. The per-row data arrays, the original (pre- _qvr_name) identifiers, the per-column term / name presentation labels, and the original formula string travel in the complement; everything else (which columns there are, the intercept / random-term structure, the family choice) is decoded back out of the Module by _decode_module.

PARAMETER	DESCRIPTION
family	Response family from quivers.formulas.families. TYPE: Family
fixed_prior	Default prior for fixed-effect coefficients, in the surface form "Family(arg, arg, ...)"; numeric args become floats, identifier args stay as variable references in the emitted program. TYPE: str DEFAULT: 'Normal(0.0, 5.0)'
random_scale_prior	Default prior for random-effect scale parameters. TYPE: str DEFAULT: 'HalfNormal(1.0)'
user_priors	Per-name prior overrides keyed by the latent's variable name in the emitted module. TYPE: Mapping[str, str] DEFAULT: None
reparameterize	Random-effect parameterization. Categorical random effects use the non-centered form. TYPE: ('centered', 'noncentered') DEFAULT: "centered"
binomial_trials	Common trial count or the name of a per-row trial-count column. TYPE: int or str DEFAULT: 1
thresholds_by	Grouping factor whose levels receive partially pooled ordinal cutpoint spacings. TYPE: str or None DEFAULT: None
mixture_components	Number of components for family="mixture"; must be at least two. TYPE: int DEFAULT: 2
predictor_name	Host-data name for an external differentiable predictor. TYPE: str or None DEFAULT: None

RAISES	DESCRIPTION
ValueError	If an option is out of range or does not apply to family: thresholds_by outside the cumulative family, binomial_trials outside the binomial family or below one, mixture_components below two or outside the mixture family, or an unknown reparameterize value.

Notes

GetPut: backward (forward(f)) = f for every Formula f. PutGet holds on pairs (t, c) for which t is in the image of forward and c is the corresponding FormulaData.

Source code in src/quivers/formulas/compile.py

def __init__(
    self,
    family: Family,
    *,
    fixed_prior: str = "Normal(0.0, 5.0)",
    random_scale_prior: str = "HalfNormal(1.0)",
    user_priors: Mapping[str, str] | None = None,
    reparameterize: Literal["centered", "noncentered"] = "noncentered",
    binomial_trials: int | str = 1,
    thresholds_by: str | None = None,
    mixture_components: int = 2,
    predictor_name: str | None = None,
) -> None:
    self._family = family
    self._fixed_prior = fixed_prior
    self._random_scale_prior = random_scale_prior
    self._user_priors: Mapping[str, str] = dict(user_priors or {})
    if thresholds_by is not None and family.name != "cumulative":
        raise ValueError(
            "FormulaToQVRModule: thresholds_by is only valid for the "
            "cumulative family"
        )
    if family.name != "binomial" and binomial_trials != 1:
        raise ValueError(
            "FormulaToQVRModule: binomial_trials is only valid for the "
            "binomial family"
        )
    if not isinstance(binomial_trials, (int, str)) or isinstance(
        binomial_trials, bool
    ):
        raise ValueError(
            "FormulaToQVRModule: binomial_trials must be a positive integer "
            "or a data-column name"
        )
    if isinstance(binomial_trials, int) and binomial_trials < 1:
        raise ValueError("FormulaToQVRModule: binomial_trials must be at least one")
    if (
        not isinstance(mixture_components, int)
        or isinstance(mixture_components, bool)
        or mixture_components < 2
    ):
        raise ValueError(
            "FormulaToQVRModule: mixture_components must be an integer "
            "of at least two"
        )
    if family.name != "mixture" and mixture_components != 2:
        raise ValueError(
            "FormulaToQVRModule: mixture_components is only valid for the "
            "mixture family"
        )
    self._binomial_trials = binomial_trials
    self._thresholds_by = thresholds_by
    self._mixture_components = mixture_components
    self._predictor_name = (
        _qvr_name(predictor_name) if predictor_name is not None else None
    )
    if reparameterize not in ("centered", "noncentered"):
        raise ValueError(
            f"FormulaToQVRModule: reparameterize must be 'centered' "
            f"or 'noncentered', got {reparameterize!r}"
        )
    self._reparameterize = reparameterize

forward

forward(formula: Formula) -> tuple[Module, FormulaData]

Compile a formula to a QVR module and its complement.

PARAMETER	DESCRIPTION
formula	The parsed formula with its design data. TYPE: Formula

RETURNS	DESCRIPTION
tuple[Module, FormulaData]	The emitted QVR module AST, and the fields of formula the module does not encode.

RAISES	DESCRIPTION
ValueError	If the response is incompatible with the family (for instance non-integer or non-contiguous category labels for a categorical or cumulative family), or a prior does not parse as "Family(arg, ...)".

Source code in src/quivers/formulas/compile.py

def forward(self, formula: Formula, /) -> tuple[Module, FormulaData]:
    """Compile a formula to a QVR module and its complement.

    Parameters
    ----------
    formula : Formula
        The parsed formula with its design data.

    Returns
    -------
    tuple[Module, FormulaData]
        The emitted QVR module AST, and the fields of ``formula``
        the module does not encode.

    Raises
    ------
    ValueError
        If the response is incompatible with the family (for
        instance non-integer or non-contiguous category labels
        for a categorical or cumulative family), or a prior does
        not parse as ``"Family(arg, ...)"``.
    """
    module = self._build_module(formula)
    complement = FormulaData(
        formula=formula.formula,
        response_name=formula.response_name,
        response_values=formula.response_values,
        fixed_column_names={
            col.qvr_name: (col.term, col.name) for col in formula.fixed_columns
        },
        fixed_column_data={
            col.qvr_name: col.data
            for col in formula.fixed_columns
            if not col.is_intercept
        },
        group_original_names={_qvr_name(g): g for g in formula.group_levels.keys()},
        group_levels=dict(formula.group_levels),
        group_indices=dict(formula.group_indices),
    )
    return module, complement

backward

backward(target: Module, complement: FormulaData) -> Formula

Recover a formula from a QVR module and its complement.

The column, intercept, and random-term structure is decoded from target; the data arrays and presentation labels come from complement.

PARAMETER	DESCRIPTION
target	A QVR module in the image of forward. TYPE: Module
complement	The complement forward returned with that module. TYPE: FormulaData

RETURNS	DESCRIPTION
Formula	The reconstructed formula.

Source code in src/quivers/formulas/compile.py

def backward(self, target: Module, complement: FormulaData, /) -> Formula:
    """Recover a formula from a QVR module and its complement.

    The column, intercept, and random-term structure is decoded
    from ``target``; the data arrays and presentation labels come
    from ``complement``.

    Parameters
    ----------
    target : Module
        A QVR module in the image of `forward`.
    complement : FormulaData
        The complement `forward` returned with that module.

    Returns
    -------
    Formula
        The reconstructed formula.
    """
    decoded = _decode_module(target)
    fixed_column_names = complement.fixed_column_names
    fixed_column_data = complement.fixed_column_data

    fixed_columns: list[FixedColumn] = []
    n_obs = decoded["n_obs"]
    ones = np.ones(n_obs, dtype=float)
    for qvr_name, is_intercept in decoded["fixed_qvr_names"]:
        if is_intercept:
            term, name = fixed_column_names.get(
                "intercept", ("Intercept", "Intercept")
            )
            fixed_columns.append(
                FixedColumn(
                    term=term,
                    name=name,
                    qvr_name="intercept",
                    is_intercept=True,
                    data=ones,
                )
            )
        else:
            term, name = fixed_column_names.get(qvr_name, (qvr_name, qvr_name))
            fixed_columns.append(
                FixedColumn(
                    term=term,
                    name=name,
                    qvr_name=qvr_name,
                    is_intercept=False,
                    data=fixed_column_data[qvr_name],
                )
            )

    random_terms: list[RandomTerm] = []
    for qgroup, slope_qvr in decoded["random_terms_qvr"]:
        group = complement.group_original_names.get(qgroup, qgroup)
        if slope_qvr == "Intercept":
            random_terms.append(RandomTerm(slope="Intercept", group=group))
        else:
            # Recover the original slope name by inverting via the
            # presentation map when available; otherwise pass the
            # qvr-name through.
            term_name_pairs = {
                qvr: (term, name)
                for qvr, (term, name) in fixed_column_names.items()
            }
            _, original_slope = term_name_pairs.get(
                slope_qvr, (slope_qvr, slope_qvr)
            )
            random_terms.append(RandomTerm(slope=original_slope, group=group))

    return Formula(
        formula=complement.formula,
        response_name=complement.response_name,
        fixed_columns=tuple(fixed_columns),
        random_terms=tuple(random_terms),
        response_values=complement.response_values,
        group_levels=complement.group_levels,
        group_indices=complement.group_indices,
    )

fixed_column_observations

fixed_column_observations(formula: Formula) -> dict[str, Tensor]

Per-column free-variable bindings for the host-data channel. One entry per non-intercept fixed column, shape (N,).

PARAMETER	DESCRIPTION
formula	The formula whose fixed-effect columns are bound. TYPE: Formula

RETURNS	DESCRIPTION
dict[str, Tensor]	Each non-intercept column's float32 data, keyed by its QVR name.

Source code in src/quivers/formulas/compile.py

def fixed_column_observations(self, formula: Formula) -> dict[str, torch.Tensor]:
    """Per-column free-variable bindings for the host-data
    channel.  One entry per non-intercept fixed column, shape
    ``(N,)``.

    Parameters
    ----------
    formula : Formula
        The formula whose fixed-effect columns are bound.

    Returns
    -------
    dict[str, torch.Tensor]
        Each non-intercept column's ``float32`` data, keyed by its
        QVR name.
    """
    obs: dict[str, torch.Tensor] = {}
    for col in formula.fixed_columns:
        if col.is_intercept:
            continue
        obs[col.qvr_name] = torch.as_tensor(col.data.copy(), dtype=torch.float32)
    return obs