Skip to content

Custom response families

A custom family binds a checked distribution contract to a native distribution factory. The contract states parameter types, intrinsic dimensions, sample support, and sampling capabilities; the factory supplies the corresponding PyTorch distribution. The same law then supplies fitting densities, raw held-out likelihoods, response expectations, and predictive draws through the existing formula workflow.

FamilyCatalog is immutable and local to a compilation. Extending one catalog does not change another model's laws or the built-in registry. Thus two models may use the same custom source name with different implementations, while their checked caches and fitted recipes retain the appropriate catalog identity.

Constructor parameters and predictor parameters

custom_family maps native constructor parameters onto existing formula roles: (i) location receives the linked main predictor mu; (ii) auxiliary supplies ordinary scalar priors or distributional regressions; and (iii) data supplies deterministic parameters from typed numeric columns. An auxiliary mapping can separate the constructor name from the regression name. For instance, native scale can receive formula parameter sigma.

The following factory returns a symmetric two-component Normal mixture. Its component locations differ by two, and a known per-row shift enters both components. The example uses a small optimization budget to exercise the API.

import pandas as pd
import torch
from torch.distributions import Categorical, MixtureSameFamily, Normal

from quivers.family_catalog import FamilyCatalog
from quivers.formulas import AuxParam, custom_family, fit, links
from quivers.qiec.families import DistributionFamily, FamilyParameter
from quivers.qiec.types import REAL

contract = DistributionFamily(
    name="TwoMode",
    parameters=(
        FamilyParameter(name="center", constraint="real", rank=0),
        FamilyParameter(name="scale", constraint="positive", rank=0),
        FamilyParameter(name="bias", constraint="real", rank=0),
    ),
    support="real", element=REAL, event_rank=0, event_sources=(),
    discrete=False, reparameterizable=False, compositional=False,
)

def two_mode(parameters):
    center = parameters["center"] + parameters["bias"]
    locations = torch.stack((center - 1, center + 1), dim=-1)
    scales = parameters["scale"].unsqueeze(-1).expand_as(locations)
    weights = Categorical(probs=torch.full_like(locations, 0.5))
    return MixtureSameFamily(weights, Normal(locations, scales))

catalog = FamilyCatalog().with_family(contract, two_mode)
response_family = custom_family(
    "TwoMode", catalog=catalog, location="center",
    auxiliary={
        "scale": AuxParam(
            name="sigma", prior="LogNormal(0.0, 0.3)", link=links["log"],
        ),
    },
    data={"bias": "known_shift"},
)
frame = pd.DataFrame({
    "y": [-0.7, 0.8, -0.4, 1.2],
    "x": [-1.0, 0.0, 0.5, 1.0],
    "known_shift": [0.1, -0.2, 0.3, 0.0],
})
result = fit(
    "y ~ x", data=frame, family=response_family,
    distributional={"sigma": "~x"},
    method="svi", num_steps=2, seed=7,
)
fresh = frame.iloc[[3, 1]].copy()
expected = result.posterior_epred(
    new_data=fresh.drop(columns="y"), num_samples=5, seed=11,
)
scores = result.log_likelihood(new_data=fresh, num_samples=5, seed=11)["y"]
assert expected.shape == scores.shape == (1, 5, 2)
assert result.posterior_predict(
    new_data=fresh.drop(columns="y"), num_samples=5, seed=11,
).shape == expected.shape

Deterministic data parameters introduce no latent coefficient or prior. They enter the existing checked predictor graph, preserving their fitted numeric carriers and row identities during new-data prediction, subsets, and refits. The native and formula parameter names need not coincide. Location and auxiliary predictors retain ordinary group effects, nonlinear expressions, priors, and likelihood weights; raw posterior likelihoods remain unweighted.

Capabilities and ownership

The factory must return the normalized law described by its contract. QVR checks constructor argument names, intrinsic dimensions, batch broadcasting, and declared reparameterization at the native boundary. These checks cannot prove that a user-written log_prob integrates to one or that its sampler has the claimed law; those mathematical properties require independent tests.

A custom torch.distributions.Distribution subclass can supply an arbitrary log_prob, sample or rsample, and mean. Existing response helpers use those operations directly. Censoring and truncation additionally require the existing interval-mass or CDF capabilities and a truthful atomicity contract. The catalog API describes typed native inputs and execution capacity. Host implementations do not acquire external target implementations through a name alone; unsupported source targets report their capabilities.

Catalogs retain live native factories by identity. Updates and cross-validation reuse that binding, while durable fitted artifacts require an explicit factory key and configuration when restoring it. The stored semantic contract is checked against the supplied implementation catalog.

custom_family

custom_family(name: str, *, catalog: FamilyCatalog, location: str, link: str | Link = 'identity', auxiliary: tuple[AuxParam, ...] | Mapping[str, AuxParam] = (), data: Mapping[str, str] | None = None) -> Family

Bind an arbitrary checked native law to ordinary regression predictors.

PARAMETER DESCRIPTION
name

Custom distribution name already defined in the supplied catalog.

TYPE: str

catalog

Immutable semantic contracts and native distribution factories.

TYPE: FamilyCatalog

location

Native parameter receiving the linked main predictor named mu.

TYPE: str

link

Existing inverse link for that predictor.

TYPE: str or Link DEFAULT: 'identity'

auxiliary

Other native parameters, with ordinary priors and distributional links. A mapping separates native names from formula names, for instance native scale mapped to an auxiliary named sigma.

TYPE: tuple[AuxParam, ...] or Mapping[str, AuxParam] DEFAULT: ()

data

Native parameters supplied by typed numeric data columns. These enter the existing predictor graph as deterministic identity-linked roots.

TYPE: Mapping[str, str] or None DEFAULT: None

RETURNS DESCRIPTION
Family

Response law using the same compiler, inference and replay operations.

RAISES DESCRIPTION
(ValueError, TypeError)

If parameter roles overlap, omit a constructor parameter, or use an unknown link, noncustom name, or invalid deterministic input name.

Source code in src/quivers/formulas/_custom_family.py
def custom_family(
    name: str,
    *,
    catalog: FamilyCatalog,
    location: str,
    link: str | Link = "identity",
    auxiliary: tuple[AuxParam, ...] | Mapping[str, AuxParam] = (),
    data: Mapping[str, str] | None = None,
) -> Family:
    """Bind an arbitrary checked native law to ordinary regression predictors.

    Parameters
    ----------
    name : str
        Custom distribution name already defined in the supplied catalog.
    catalog : FamilyCatalog
        Immutable semantic contracts and native distribution factories.
    location : str
        Native parameter receiving the linked main predictor named ``mu``.
    link : str or Link
        Existing inverse link for that predictor.
    auxiliary : tuple[AuxParam, ...] or Mapping[str, AuxParam]
        Other native parameters, with ordinary priors and distributional links.
        A mapping separates native names from formula names, for instance
        native ``scale`` mapped to an auxiliary named ``sigma``.
    data : Mapping[str, str] or None
        Native parameters supplied by typed numeric data columns. These enter
        the existing predictor graph as deterministic identity-linked roots.

    Returns
    -------
    Family
        Response law using the same compiler, inference and replay operations.

    Raises
    ------
    ValueError, TypeError
        If parameter roles overlap, omit a constructor parameter, or use an
        unknown link, noncustom name, or invalid deterministic input name.
    """
    if not isinstance(catalog, FamilyCatalog):
        raise TypeError("catalog must be an immutable FamilyCatalog")
    if name not in {record.name for record in catalog.definitions}:
        raise ValueError(f"{name!r} is not a custom law in this catalog")
    record = catalog[name]
    if isinstance(link, str):
        if link not in links:
            raise ValueError(f"unknown inverse link {link!r}")
        link = links[link]
    if not isinstance(link, Link):
        raise TypeError("link must be a registered inverse link or Link")
    if link.name not in links or link != links[link.name]:
        raise ValueError("custom families use a registered inverse link")
    extra = dict(data or {})
    if not isinstance(auxiliary, (tuple, Mapping)):
        raise TypeError("auxiliary must be a tuple or mapping of AuxParam values")
    auxiliary_values = (
        tuple(auxiliary.values()) if isinstance(auxiliary, Mapping) else auxiliary
    )
    if any(not isinstance(item, AuxParam) for item in auxiliary_values):
        raise TypeError("auxiliary parameters must be AuxParam values")
    auxiliary_map = (
        dict(auxiliary)
        if isinstance(auxiliary, Mapping)
        else {item.name: item for item in auxiliary_values}
    )
    names = [item.name for item in auxiliary_values]
    if len(auxiliary_map) != len(auxiliary) or len(set(names)) != len(names):
        raise ValueError("custom auxiliary formula names must be distinct")
    roles = (location, *auxiliary_map, *extra)
    if len(set(roles)) != len(roles) or set(roles) != set(record.parameter_names):
        raise ValueError(
            "location, auxiliary and data roles must partition all native parameters"
        )
    if any(
        not isinstance(value, str) or not re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", value)
        for value in (*names, *extra, *extra.values())
    ):
        raise ValueError(
            "custom parameter and data column names require QVR identifiers"
        )
    if "mu" in names:
        raise ValueError("mu is reserved for the main formula predictor")
    # Deterministic inputs use private graph roots, avoiding collisions with
    # authored constructor names and ordinary population coefficient sites.
    data_roots = {parameter: f"custom_data_{parameter}" for parameter in extra}
    if set(names) & set(data_roots.values()):
        raise ValueError("custom_data_ names are reserved for declared data parameters")
    deterministic = tuple(
        AuxParam(name=root, prior="Normal(0.0, 1.0)", link=links["identity"])
        for root in data_roots.values()
    )
    binding_names = {
        location: "mu",
        **{native: value.name for native, value in auxiliary_map.items()},
        **data_roots,
    }
    bindings = tuple(
        (parameter, binding_names[parameter]) for parameter in record.parameter_names
    )
    return Family(
        name=name,
        observe_family=name,
        location_link=link,
        aux_params=(*auxiliary_values, *deterministic),
        custom=CustomFamilySpec(
            catalog=catalog,
            bindings=bindings,
            data=tuple((data_roots[name], column) for name, column in extra.items()),
        ),
    )

CustomFamilySpec didactic-model

CustomFamilySpec(*, catalog: FamilyCatalog = field(opaque=True, extras=OPAQUE_BY_IDENTITY), bindings: tuple[tuple[str, str], ...] = field(opaque=True, extras=OPAQUE_BY_VALUE), data: tuple[tuple[str, str], ...] = field(default=(), opaque=True, extras=OPAQUE_BY_VALUE))

Bases: ValueModel

One immutable custom-law binding, with a compilation-local implementation.

PARAMETER DESCRIPTION
catalog

Checked semantic contracts and native implementations, held by identity.

TYPE: FamilyCatalog DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

bindings

Native constructor parameters paired with formula predictor bindings.

TYPE: tuple[tuple[str, str], ...] DEFAULT: field(opaque=True, extras=OPAQUE_BY_VALUE)

data

Deterministic distributional parameters paired with source data columns.

TYPE: tuple[tuple[str, str], ...] DEFAULT: field(default=(), opaque=True, extras=OPAQUE_BY_VALUE)