Skip to content

Shared posterior analysis

ModelFit.analysis and BayesianFit.analysis produce the same PosteriorAnalysis: constrained posterior draws, raw observation likelihoods, observed responses, and recorded sampler statistics. A request samples a variational guide once; later methods reuse the retained tensors. Tensor mappings are held by identity, so callers own their storage.

summary returns means, sample standard deviations, and requested quantiles on each latent site's event coordinates. Recorded MCMC chains also supply a ConvergenceSummary; variational draws supply None. Their one pseudo-chain records array shape, without asserting MCMC convergence. Original integer capacity is checked before Float64 conversion. A coordinate containing a nonfinite draw has NaN mean, SD, and quantiles; draws are not deleted, and valid coordinates remain independent of that failure. The same scaled arithmetic and type-7 quantile kernel serve recorded-chain and retained uncertainty summaries.

A source program with no latent sites can still supply observation likelihoods. Its recorded posterior count determines the chain/draw layout; the checked registry must establish that no unobserved variable is missing. Such an analysis has an empty parameter summary, while raw likelihoods, WAIC, and LOO remain available. No placeholder parameter or convergence statistic is introduced. Covariance and contrast requests still require their selected parameter values.

Joint covariance and correlation

analysis.covariance(var_names=None, ddof=1) returns a PosteriorCovariance containing both empirical covariance and Pearson correlation. It pools chain and draw axes by default; chainwise=True returns one matrix per retained chain axis. The divisor is the reduced sample count minus ddof, which must be a nonnegative integer smaller than that count. No convergence diagnostic is implied by either matrix.

Selected variables retain their requested order. Each event flattens in row-major order, while coordinates retains each exact (site, index_tuple); variables, event_shapes, and sample_shape retain the original layout. Pass dims and coords as for to_datatree: the same validator checks named axis extents and label lengths. dimension_names and coordinate_labels identify each matrix column without parsing a rendered name. Chain and draw labels remain available even when those axes were reduced.

These summaries use retained tensors and one joint Float64 matrix, without resampling or likelihood evaluation. Boolean and integer prior draws are numeric inputs; original integers outside the exact Float64 range are refused before conversion. Coordinate labels retain their original values. Missing and nonfinite draws are refused, without pairwise deletion. A constant coordinate has zero covariance and NaN correlations with every coordinate, including itself. Mathematical covariances below Float64 resolution may underflow; correlations are computed from scaled coordinates independently of that underflow. Unrepresentable covariance magnitudes cause an explicit error. Tensor mappings and returned tensor storage remain caller-owned.

Observation units and predictive scores

Log-likelihood tensors have shape (chain, draw, *observation_shape), after reducing event coordinates. They contain raw conditional densities, excluding priors and likelihood weights. Multiple responses require an explicit var_name; combining their scores requires a justified common predictive unit. An observation hidden inside exact marginalization requires a corresponding likelihood observer.

waic needs only PyTorch and at least two pooled posterior draws with finite scores. It uses unbiased sample log-score variances, with divisor S - 1, matching the loo implementation. WAICResult exposes elpd_waic, effective parameter estimate p_waic, and waic = -2 * elpd_waic. se is the standard error on the ELPD scale; se_waic is twice that value, and se_p_waic describes the effective parameter estimate. These standard errors are undefined for one observation unit. elpd_i, variance_i, and waic_i retain the observation axes. A pointwise log-score variance above 0.4 sets warning, following the loo diagnostic.

to_datatree and loo require the optional diagnostics extra. The DataTree retains chain and draw axes and records quivers_posterior_kind in its attributes. Pass coords={"row": [...]} and dims={"y": ["row"]} to label observation axes, or assign event-axis labels to latent variables. The shared converter uses the same sampler-statistic names and integer/Boolean dtypes for direct MCMC results and retained analysis snapshots; conversion detaches tensors without changing the retained draws. PSIS-LOO exposes ArviZ 1 results, including elpd, elpd_i, and pareto_k. Variational draws use relative efficiency one; this does not correct approximation error. If every selected likelihood column is finite and constant across draws, LOO uses exact uniform importance weights through ArviZ's precomputed-weight interface. The pointwise score is unchanged; Pareto shapes are NaN because no tail was fitted. This also applies when retained parameters vary but do not affect the selected response likelihood. Varying-score analyses retain ordinary PSIS. Unstable importance weights may require exact refitting or a different predictive unit (Vehtari et al., 2017).

The formula workflow gives an executable fit and analysis. Low-level samplers can construct PosteriorAnalysis directly with explicit tensor mappings and provenance.

Retained analysis storage

A posterior archive retains the analysis tensors and their coordinate contract. to_netcdf writes the existing ArviZ DataTree groups, and from_netcdf restores owned CPU tensors without replaying the model. from_datatree accepts the same groups directly. The optional diagnostics extra supplies this storage path.

Archives retain (i) original tensor dtypes and draw provenance, (ii) chain, draw, and event labels, and (iii) group-specific event dimensions. The last distinction matters for vector observations: their recorded response can retain a category axis even when each raw likelihood has already reduced that event to one score. The same converter serves ordinary ArviZ analysis and storage.

coords and dims on PosteriorAnalysis supply defaults for later conversion and covariance requests; explicit method arguments override them. group_dims preserves dimensions that differ between observed data and reduced likelihoods. Readers validate sample-axis labels across groups, rather than matching draws by array extent. External trees need explicit kind; chain count alone cannot establish MCMC provenance. Unknown format versions, lossy dtype metadata, and conflicting sampler-statistic aliases are refused.

Float16 and BFloat16 values use exact Float32 storage where the NumPy or NetCDF backend lacks their dtype, then restore their recorded dtype. Integer draws keep their original capacity; reading them does not bypass summary checks before a Float64 conversion. Writes use an atomic sibling file, shared with source migration, so an interrupted write preserves the previous destination.

from pathlib import Path
from tempfile import TemporaryDirectory
import torch
from quivers.diagnostics import PosteriorAnalysis

analysis = PosteriorAnalysis(
    samples={"theta": torch.tensor([[[0.2, 0.7], [0.4, 0.6], [0.3, 0.8]]])},
    log_likelihood={}, kind="variational",
    coords={"coefficient": ["intercept", "slope"]},
    dims={"theta": ["coefficient"]},
)
with TemporaryDirectory() as directory:
    path = Path(directory) / "posterior.nc"
    analysis.to_netcdf(path)
    restored = PosteriorAnalysis.from_netcdf(path)
assert restored.kind == "variational"
torch.testing.assert_close(restored.samples["theta"], analysis.samples["theta"])
assert restored.covariance().coordinate_labels == (("intercept",), ("slope",))

This archive stores retained analysis. Executable prediction also requires the fitted program, its deterministic parameters, and preprocessing state; those objects are not reconstructed from posterior draws alone. The file layout uses xarray DataTree NetCDF storage.

posterior

Reusable posterior analysis over explicit chain, draw, and observation axes.

PosteriorSummary didactic-model

PosteriorSummary(*, mean: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), sd: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), probs: tuple[float, ...], quantiles: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), convergence: ConvergenceSummary | None = field(default=None, opaque=True, extras=OPAQUE_BY_VALUE))

Bases: ValueModel

Pooled uncertainty summaries, with MCMC diagnostics when applicable.

PARAMETER DESCRIPTION
mean

Mean and sample standard deviation on the site's event coordinates.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

sd

Mean and sample standard deviation on the site's event coordinates.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

probs

Requested quantile probabilities.

TYPE: tuple[float, ...]

quantiles

Quantiles with shape (len(probs), *event_shape).

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

convergence

Recorded-chain diagnostics, or None for variational draws.

TYPE: ConvergenceSummary or None DEFAULT: field(default=None, opaque=True, extras=OPAQUE_BY_VALUE)

PosteriorCovariance didactic-model

PosteriorCovariance(*, covariance: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), correlation: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), coordinates: tuple[tuple[str, tuple[int, ...]], ...] = field(opaque=True, extras=OPAQUE_BY_VALUE), dimension_names: tuple[tuple[str, ...], ...], coordinate_labels: tuple[tuple[object, ...], ...] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), variables: tuple[str, ...], event_shapes: tuple[tuple[int, ...], ...], sample_shape: tuple[int, int] = field(opaque=True, extras=OPAQUE_BY_VALUE), chain_labels: tuple[object, ...] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), draw_labels: tuple[object, ...] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), ddof: int, chainwise: bool, kind: Literal['mcmc', 'variational', 'prior'])

Bases: ValueModel

Joint empirical covariance and correlation of retained coordinates.

PARAMETER DESCRIPTION
covariance

Float64 square matrices, or one matrix per chain when chainwise. A constant coordinate has zero covariance and NaN correlations.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

correlation

Float64 square matrices, or one matrix per chain when chainwise. A constant coordinate has zero covariance and NaN correlations.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

coordinates

Matrix columns as an exact variable name and row-major event index. The same coordinates label matrix rows; no string parsing is needed.

TYPE: tuple[tuple[str, tuple[int, ...]], ...] DEFAULT: field(opaque=True, extras=OPAQUE_BY_VALUE)

dimension_names

Event dimension names and their original labels for each matrix column.

TYPE: tuple

coordinate_labels

Event dimension names and their original labels for each matrix column.

TYPE: tuple

variables

Selected variable order and corresponding original event shapes.

TYPE: tuple

event_shapes

Selected variable order and corresponding original event shapes.

TYPE: tuple

sample_shape

Original chain and draw counts, before the empirical reduction.

TYPE: tuple[int, int] DEFAULT: field(opaque=True, extras=OPAQUE_BY_VALUE)

chain_labels

Original sample-axis labels, or their positional defaults.

TYPE: tuple DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

draw_labels

Original sample-axis labels, or their positional defaults.

TYPE: tuple DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

ddof

Degrees of freedom subtracted from the pooled or per-chain draw count.

TYPE: int

chainwise

Whether each chain was summarized separately.

TYPE: bool

kind

Original draw provenance; this record adds no convergence diagnostics.

TYPE: str

WAICResult didactic-model

WAICResult(*, elpd_waic: float, p_waic: float, waic: float, se: float, se_p_waic: float, elpd_i: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), variance_i: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), warning: bool, num_samples: int, num_observations: int)

Bases: ValueModel

Pointwise WAIC estimates using sample log-likelihood variances.

PARAMETER DESCRIPTION
elpd_waic

Expected log predictive density, effective parameter count, and deviance-scale criterion -2 * elpd_waic.

TYPE: float

p_waic

Expected log predictive density, effective parameter count, and deviance-scale criterion -2 * elpd_waic.

TYPE: float

waic

Expected log predictive density, effective parameter count, and deviance-scale criterion -2 * elpd_waic.

TYPE: float

se

Standard errors of elpd_waic and p_waic across observation units; NaN for one unit.

TYPE: float

se_p_waic

Standard errors of elpd_waic and p_waic across observation units; NaN for one unit.

TYPE: float

elpd_i

Pointwise ELPD and unbiased sample variance over pooled posterior draws.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

variance_i

Pointwise ELPD and unbiased sample variance over pooled posterior draws.

TYPE: Tensor DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

warning

Whether any pointwise log-likelihood variance exceeds 0.4.

TYPE: bool

num_samples

Pooled posterior draw count and number of observation units.

TYPE: int

num_observations

Pooled posterior draw count and number of observation units.

TYPE: int

se_waic property

se_waic: float

Standard error of the deviance-scale WAIC estimate.

RETURNS DESCRIPTION
float

Twice the ELPD-scale standard error.

waic_i property

waic_i: Tensor

Deviance-scale contributions on the original observation axes.

RETURNS DESCRIPTION
Tensor

Negative twice each retained pointwise ELPD estimate.

PosteriorAnalysis didactic-model

PosteriorAnalysis(*, samples: Mapping[str, Tensor] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_likelihood: Mapping[str, Tensor] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), observed_data: Mapping[str, Tensor] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY), sample_stats: Mapping[str, Tensor] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY), kind: Literal['mcmc', 'variational', 'prior'] = 'mcmc', coords: Mapping[str, Sequence[object]] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY), dims: Mapping[str, Sequence[str]] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY), group_dims: Mapping[str, Mapping[str, Sequence[str]]] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY))

Bases: ValueModel

A reusable tensor posterior and its raw observation likelihoods.

PARAMETER DESCRIPTION
samples

Constrained draws with chain and draw axes.

TYPE: Mapping[str, Tensor] DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

log_likelihood

Raw pointwise scores retaining observation axes.

TYPE: Mapping[str, Tensor] DEFAULT: field(opaque=True, extras=OPAQUE_BY_IDENTITY)

observed_data

Responses corresponding to the score units.

TYPE: Mapping[str, Tensor] DEFAULT: field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY)

sample_stats

Recorded per-draw sampler statistics, empty for variational draws.

TYPE: Mapping[str, Tensor] DEFAULT: field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY)

kind

Provenance of the draws. Variational and independent prior layout axes never supply MCMC convergence diagnostics.

TYPE: 'mcmc' or 'variational' or 'prior' DEFAULT: 'mcmc'

group_dims

Optional group-specific event dimensions, preserving observed vector events alongside their reduced pointwise likelihood axes.

TYPE: Mapping DEFAULT: field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY)

coords

Retained coordinate labels and event dimension names, used by default for DataTree conversion and covariance. Explicit method arguments override them.

TYPE: Mapping DEFAULT: field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY)

dims

Retained coordinate labels and event dimension names, used by default for DataTree conversion and covariance. Explicit method arguments override them.

TYPE: Mapping DEFAULT: field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY)

Notes

Methods consume these retained tensors without replaying the model. Tensor mappings are held by identity; callers own their storage.

summary

summary(*, probs: Sequence[float] = (0.05, 0.5, 0.95)) -> dict[str, PosteriorSummary]

Summarize uncertainty, attaching diagnostics only to recorded chains.

PARAMETER DESCRIPTION
probs

Quantile probabilities strictly between zero and one.

TYPE: Sequence[float] DEFAULT: (0.05, 0.5, 0.95)

RETURNS DESCRIPTION
dict[str, PosteriorSummary]

Float64 summaries on each site's event axes. Any coordinate with a nonfinite draw has NaN location, dispersion, and quantiles; no draws are deleted. Original integer capacity is checked before conversion, including independent prior draws.

RAISES DESCRIPTION
ValueError

If sample axes or quantile probabilities are invalid, or an original integer exceeds exact Float64 capacity.

Source code in src/quivers/diagnostics/posterior.py
def summary(
    self, *, probs: Sequence[float] = (0.05, 0.5, 0.95)
) -> dict[str, PosteriorSummary]:
    """Summarize uncertainty, attaching diagnostics only to recorded chains.

    Parameters
    ----------
    probs : Sequence[float]
        Quantile probabilities strictly between zero and one.

    Returns
    -------
    dict[str, PosteriorSummary]
        Float64 summaries on each site's event axes. Any coordinate with
        a nonfinite draw has NaN location, dispersion, and quantiles;
        no draws are deleted. Original integer capacity is checked before
        conversion, including independent prior draws.

    Raises
    ------
    ValueError
        If sample axes or quantile probabilities are invalid, or an original
        integer exceeds exact Float64 capacity.
    """
    self._validate_axes()
    levels = probabilities(probs)
    diagnostics = (
        convergence_summary(self.samples, probs=levels)
        if self.kind == "mcmc"
        else {}
    )
    result = {}
    for name, values in self.samples.items():
        if (
            values.ndim < 2
            or (not values.is_floating_point() and self.kind != "prior")
            or not all(values.shape)
        ):
            raise ValueError(
                "posterior samples require nonempty chain and draw axes"
            )
        if name in diagnostics:
            d = diagnostics[name]
            mean, sd, quantiles = d.mean, d.sd, d.quantiles
        else:
            pooled = real_samples(
                values, label=f"summary {name!r}", finite=False
            ).flatten(0, 1)
            mean, sd, quantiles = empirical_summary(pooled, levels)
        result[name] = PosteriorSummary(
            mean=mean,
            sd=sd,
            probs=levels,
            quantiles=quantiles,
            convergence=diagnostics.get(name),
        )
    return result

covariance

covariance(var_names: Sequence[str] | None = None, *, ddof: int = 1, chainwise: bool = False, coords: Mapping[str, Sequence[object]] | None = None, dims: Mapping[str, Sequence[str]] | None = None) -> PosteriorCovariance

Summarize joint empirical dependence without replaying a posterior.

PARAMETER DESCRIPTION
var_names

Distinct selected sites, in matrix order; None selects insertion order. Each site's event coordinates flatten in row-major order.

TYPE: Sequence[str] or None DEFAULT: None

ddof

Nonnegative degrees of freedom, strictly below the reduced count.

TYPE: int DEFAULT: 1

chainwise

Retain a separate matrix per chain instead of pooling chain and draw.

TYPE: bool DEFAULT: False

coords

Labels for named event axes and optional chain/draw axes.

TYPE: Mapping[str, Sequence[object]] or None DEFAULT: None

dims

Event-axis names per site, following the DataTree convention.

TYPE: Mapping[str, Sequence[str]] or None DEFAULT: None

RETURNS DESCRIPTION
PosteriorCovariance

Float64 covariance and Pearson correlation, structural coordinate labels, sample-axis labels, and provenance. Constants have NaN correlations. No convergence statistics are manufactured.

RAISES DESCRIPTION
(ValueError, TypeError)

If axes, labels, selection, ddof, devices, or numeric carriers are invalid; missing/nonfinite draws are refused, without deletion. Integer values outside the exact Float64 range are refused before casting, as are unrepresentable covariance results.

Source code in src/quivers/diagnostics/posterior.py
def covariance(
    self,
    var_names: Sequence[str] | None = None,
    *,
    ddof: int = 1,
    chainwise: bool = False,
    coords: Mapping[str, Sequence[object]] | None = None,
    dims: Mapping[str, Sequence[str]] | None = None,
) -> PosteriorCovariance:
    """Summarize joint empirical dependence without replaying a posterior.

    Parameters
    ----------
    var_names : Sequence[str] or None
        Distinct selected sites, in matrix order; None selects insertion order.
        Each site's event coordinates flatten in row-major order.
    ddof : int
        Nonnegative degrees of freedom, strictly below the reduced count.
    chainwise : bool
        Retain a separate matrix per chain instead of pooling chain and draw.
    coords : Mapping[str, Sequence[object]] or None
        Labels for named event axes and optional chain/draw axes.
    dims : Mapping[str, Sequence[str]] or None
        Event-axis names per site, following the DataTree convention.

    Returns
    -------
    PosteriorCovariance
        Float64 covariance and Pearson correlation, structural coordinate
        labels, sample-axis labels, and provenance. Constants have NaN
        correlations. No convergence statistics are manufactured.

    Raises
    ------
    ValueError, TypeError
        If axes, labels, selection, ddof, devices, or numeric carriers are
        invalid; missing/nonfinite draws are refused, without deletion.
        Integer values outside the exact Float64 range are refused before
        casting, as are unrepresentable covariance results.
    """
    from quivers.diagnostics._coordinates import axis_labels, event_dimensions
    from quivers.diagnostics._numeric import real_samples

    self._validate_axes()
    coords = self.coords if coords is None else coords
    dims = self.dims if dims is None else dims
    if isinstance(var_names, str | bytes):
        raise TypeError("var_names must be a sequence of site names")
    names = tuple(self.samples) if var_names is None else tuple(var_names)
    if any(not isinstance(name, str) for name in names):
        raise TypeError("posterior variable names must be strings")
    if not names or len(set(names)) != len(names):
        raise ValueError("covariance requires a nonempty distinct site selection")
    unknown = set(names) - self.samples.keys()
    if unknown:
        raise ValueError(f"unknown posterior variables: {sorted(unknown)}")
    if isinstance(ddof, bool) or not isinstance(ddof, int) or ddof < 0:
        raise ValueError("ddof must be a nonnegative integer")
    if not isinstance(chainwise, bool):
        raise TypeError("chainwise must be Boolean")
    first = self.samples[names[0]]
    sample_shape = (first.shape[0], first.shape[1])
    chains, draws = sample_shape
    count = draws if chainwise else chains * draws
    if ddof >= count:
        raise ValueError("ddof must be smaller than the reduced sample count")
    shapes = {name: tuple(self.samples[name].shape[2:]) for name in names}
    dimensions = event_dimensions(shapes, coords=coords, dims=dims)
    chain_labels = axis_labels("chain", chains, coords)
    draw_labels = axis_labels("draw", draws, coords)
    columns, structural, dimension_names, labels = [], [], [], []
    device = self.samples[names[0]].device
    for name in names:
        values = self.samples[name]
        if values.device != device:
            raise ValueError("covariance requires samples on one device")
        columns.append(
            real_samples(values, label="covariance").reshape(chains, draws, -1)
        )
        dim_names = dimensions[name]
        dim_labels = [
            axis_labels(dim, size, coords)
            for dim, size in zip(dim_names, shapes[name], strict=True)
        ]
        for index in product(*(range(size) for size in shapes[name])):
            structural.append((name, index))
            dimension_names.append(dim_names)
            labels.append(
                tuple(
                    label[position]
                    for label, position in zip(dim_labels, index, strict=True)
                )
            )
    values = torch.cat(columns, dim=-1)
    if not chainwise:
        values = values.flatten(0, 1)
    # Anchor subtraction retains small differences around a large common
    # offset. Scaling also handles opposite finite endpoints whose
    # subtraction would overflow, without summing their raw magnitudes.
    delta = values - values[..., :1, :]
    working = torch.where(
        torch.isfinite(delta).all(dim=-2, keepdim=True), delta, values
    )
    scale = working.abs().amax(dim=-2, keepdim=True)
    normalized = working / torch.where(scale == 0, 1.0, scale)
    centered = normalized - normalized.mean(dim=-2, keepdim=True)
    gram = centered.transpose(-1, -2) @ centered / (count - ddof)
    mantissa, exponent = torch.frexp(scale.squeeze(-2))
    base_mantissa, base_exponent = torch.frexp(gram)
    covariance = torch.ldexp(
        base_mantissa * mantissa.unsqueeze(-1) * mantissa.unsqueeze(-2),
        base_exponent + exponent.unsqueeze(-1) + exponent.unsqueeze(-2),
    )
    if not bool(torch.isfinite(covariance).all()):
        raise ValueError("covariance exceeds Float64 capacity")
    sd = gram.diagonal(dim1=-2, dim2=-1).sqrt()
    correlation = (gram / sd.unsqueeze(-1) / sd.unsqueeze(-2)).clamp(-1.0, 1.0)
    return PosteriorCovariance(
        covariance=covariance,
        correlation=correlation,
        coordinates=tuple(structural),
        dimension_names=tuple(dimension_names),
        coordinate_labels=tuple(labels),
        variables=names,
        event_shapes=tuple(shapes.values()),
        sample_shape=sample_shape,
        chain_labels=chain_labels,
        draw_labels=draw_labels,
        ddof=ddof,
        chainwise=chainwise,
        kind=self.kind,
    )

hypothesis

hypothesis(expression: str | ContrastExpression, *, prior: PosteriorAnalysis | None = None, null: float | int | Tensor = 0.0, probs: Sequence[float] = (0.025, 0.975), robust: bool = False, point: PointNullSpec | None = None, name: str = 'contrast', dims: Mapping[str, Sequence[str]] | None = None, coords: Mapping[str, Sequence[object]] | None = None) -> HypothesisSummary

Evaluate one checked QVR contrast over retained joint draws.

PARAMETER DESCRIPTION
expression

QVR value expression and optional aliases to original site names.

TYPE: str or ContrastExpression

prior

Joint draws from the corresponding proper encompassing prior. Draw counts may differ; event shapes and devices must agree.

TYPE: PosteriorAnalysis or None DEFAULT: None

null

Finite comparison value on event axes only.

TYPE: float, int, or torch.Tensor DEFAULT: 0.0

probs

Requested quantile probabilities in output order.

TYPE: Sequence[float] DEFAULT: (0.025, 0.975)

robust

Use median and scaled MAD instead of mean and sample SD.

TYPE: bool DEFAULT: False

point

Optional marginal density comparison on the same joint prior and posterior contrast; requires a checked Real result and prior draws.

TYPE: PointNullSpec or None DEFAULT: None

name

Original label of the derived contrast.

TYPE: str DEFAULT: 'contrast'

dims

Input event-axis identities and labels, with optional sample labels. Distinct axes remain distinct even when their extents coincide.

TYPE: Mapping or None DEFAULT: None

coords

Input event-axis identities and labels, with optional sample labels. Distinct axes remain distinct even when their extents coincide.

TYPE: Mapping or None DEFAULT: None

RETURNS DESCRIPTION
HypothesisSummary

Existing uncertainty, strict-region posterior odds, and optional prior-corrected region evidence. Equality frequencies are descriptive; requested point_evidence retains a separate marginal density ratio.

Source code in src/quivers/diagnostics/posterior.py
def hypothesis(
    self,
    expression: str | ContrastExpression,
    *,
    prior: PosteriorAnalysis | None = None,
    null: float | int | torch.Tensor = 0.0,
    probs: Sequence[float] = (0.025, 0.975),
    robust: bool = False,
    point: PointNullSpec | None = None,
    name: str = "contrast",
    dims: Mapping[str, Sequence[str]] | None = None,
    coords: Mapping[str, Sequence[object]] | None = None,
) -> HypothesisSummary:
    """Evaluate one checked QVR contrast over retained joint draws.

    Parameters
    ----------
    expression : str or ContrastExpression
        QVR value expression and optional aliases to original site names.
    prior : PosteriorAnalysis or None
        Joint draws from the corresponding proper encompassing prior.
        Draw counts may differ; event shapes and devices must agree.
    null : float, int, or torch.Tensor
        Finite comparison value on event axes only.
    probs : Sequence[float]
        Requested quantile probabilities in output order.
    robust : bool
        Use median and scaled MAD instead of mean and sample SD.
    point : PointNullSpec or None
        Optional marginal density comparison on the same joint prior and
        posterior contrast; requires a checked Real result and prior draws.
    name : str
        Original label of the derived contrast.
    dims, coords : Mapping or None
        Input event-axis identities and labels, with optional sample labels.
        Distinct axes remain distinct even when their extents coincide.

    Returns
    -------
    HypothesisSummary
        Existing uncertainty, strict-region posterior odds, and optional
        prior-corrected region evidence. Equality frequencies are descriptive;
        requested point_evidence retains a separate marginal density ratio.
    """
    from quivers.diagnostics.contrast import _contrast_hypothesis

    self._validate_axes()
    if prior is not None:
        prior._validate_axes()
    return _contrast_hypothesis(
        self.samples,
        expression,
        prior=prior.samples if prior is not None else None,
        null=null,
        probs=probs,
        robust=robust,
        point=point,
        name=name,
        kind=self.kind,
        dims=dims,
        coords=coords,
    )

waic

waic(*, var_name: str | None = None) -> WAICResult

Compute WAIC for one response's retained raw observation scores.

PARAMETER DESCRIPTION
var_name

Observation variable; required for multiple response variables.

TYPE: str or None DEFAULT: None

RETURNS DESCRIPTION
WAICResult

Pointwise and aggregate WAIC estimates.

Source code in src/quivers/diagnostics/posterior.py
def waic(self, *, var_name: str | None = None) -> WAICResult:
    """Compute WAIC for one response's retained raw observation scores.

    Parameters
    ----------
    var_name : str or None
        Observation variable; required for multiple response variables.

    Returns
    -------
    WAICResult
        Pointwise and aggregate WAIC estimates.
    """
    self._validate_axes()
    return waic(self.log_likelihood[self._likelihood_name(var_name)])

to_datatree

to_datatree(*, coords: Mapping[str, list] | None = None, dims: Mapping[str, list[str]] | None = None) -> DataTree

Convert retained tensors for the optional ArviZ diagnostics package.

PARAMETER DESCRIPTION
coords

Coordinate labels for named event or observation dimensions.

TYPE: Mapping[str, list] or None DEFAULT: None

dims

Dimension names per variable, excluding chain and draw axes.

TYPE: Mapping[str, list[str]] or None DEFAULT: None

RETURNS DESCRIPTION
DataTree

Posterior, likelihood, observed-data, and sampler-stat groups. Chain and draw axes are retained for both posterior kinds.

Source code in src/quivers/diagnostics/posterior.py
def to_datatree(
    self,
    *,
    coords: Mapping[str, list] | None = None,
    dims: Mapping[str, list[str]] | None = None,
) -> _DataTree:
    """Convert retained tensors for the optional ArviZ diagnostics package.

    Parameters
    ----------
    coords : Mapping[str, list] or None
        Coordinate labels for named event or observation dimensions.
    dims : Mapping[str, list[str]] or None
        Dimension names per variable, excluding chain and draw axes.

    Returns
    -------
    xarray.DataTree
        Posterior, likelihood, observed-data, and sampler-stat groups.
        Chain and draw axes are retained for both posterior kinds.
    """
    self._validate_axes()
    from quivers.diagnostics.arviz_io import (
        _ARVIZ_SAMPLE_STATS,
        _groups_to_datatree,
    )

    groups = {
        "posterior": self.samples,
        "log_likelihood": self.log_likelihood,
        "observed_data": self.observed_data,
        "sample_stats": self.sample_stats,
    }
    tree = _groups_to_datatree(
        groups,
        coords={
            name: list(values)
            for name, values in (self.coords if coords is None else coords).items()
        },
        dims={
            name: list(values)
            for name, values in (self.dims if dims is None else dims).items()
        },
        group_dims={
            group: {name: list(axes) for name, axes in names.items()}
            for group, names in self.group_dims.items()
        }
        if dims is None
        else None,
    )
    tree.attrs["quivers_posterior_kind"] = self.kind
    tree.attrs["quivers_posterior_format"] = "qvr-posterior/v1"
    tree.attrs["quivers_sample_stat_names"] = json.dumps(
        {_ARVIZ_SAMPLE_STATS.get(name, name): name for name in self.sample_stats},
        sort_keys=True,
    )
    tree.attrs["quivers_tensor_dtypes"] = json.dumps(
        {
            group: {
                (
                    _ARVIZ_SAMPLE_STATS.get(name, name)
                    if group == "sample_stats"
                    else name
                ): str(value.dtype)
                for name, value in values.items()
            }
            for group, values in groups.items()
        },
        sort_keys=True,
    )
    return tree

from_datatree classmethod

from_datatree(tree: DataTree, *, kind: Literal['mcmc', 'variational', 'prior'] | None = None) -> PosteriorAnalysis

Restore retained analysis tensors and labels from an ArviZ DataTree.

PARAMETER DESCRIPTION
tree

Canonical posterior, likelihood, observed-data, and sample-stat groups.

TYPE: DataTree

kind

Explicit provenance for external trees. QVR trees retain it in metadata; an override must agree. Chain count never determines provenance.

TYPE: ('mcmc', 'variational', 'prior') DEFAULT: "mcmc"

RETURNS DESCRIPTION
PosteriorAnalysis

Independently owned CPU tensors, dimensions, labels, and provenance. This restores analysis, rather than an executable predictor or optimizer.

RAISES DESCRIPTION
(ValueError, TypeError)

If provenance, format, labels, groups, or sampled axes are inconsistent.

Source code in src/quivers/diagnostics/posterior.py
@classmethod
def from_datatree(
    cls,
    tree: _DataTree,
    *,
    kind: Literal["mcmc", "variational", "prior"] | None = None,
) -> PosteriorAnalysis:
    """Restore retained analysis tensors and labels from an ArviZ DataTree.

    Parameters
    ----------
    tree : xarray.DataTree
        Canonical posterior, likelihood, observed-data, and sample-stat groups.
    kind : {"mcmc", "variational", "prior"} or None
        Explicit provenance for external trees. QVR trees retain it in metadata;
        an override must agree. Chain count never determines provenance.

    Returns
    -------
    PosteriorAnalysis
        Independently owned CPU tensors, dimensions, labels, and provenance.
        This restores analysis, rather than an executable predictor or optimizer.

    Raises
    ------
    ValueError, TypeError
        If provenance, format, labels, groups, or sampled axes are inconsistent.
    """
    from quivers.diagnostics.arviz_io import _posterior_from_datatree

    return _posterior_from_datatree(tree, kind=kind)

to_netcdf

to_netcdf(path: str | Path) -> None

Write retained analysis through the existing DataTree NetCDF format.

PARAMETER DESCRIPTION
path

Output NetCDF file. Requires the diagnostics extra.

TYPE: str or Path

Source code in src/quivers/diagnostics/posterior.py
def to_netcdf(self, path: str | Path) -> None:
    """Write retained analysis through the existing DataTree NetCDF format.

    Parameters
    ----------
    path : str or Path
        Output NetCDF file. Requires the diagnostics extra.
    """
    from quivers._atomic import atomic_output

    tree = self.to_datatree()
    with atomic_output(Path(path)) as temporary:
        encoding = {
            f"/{group}": {
                name: {"dtype": "float32"}
                for name, variable in node.dataset.data_vars.items()
                if variable.dtype == "float16"
            }
            for group, node in tree.children.items()
        }
        tree.to_netcdf(temporary, engine="netcdf4", encoding=encoding)

from_netcdf classmethod

from_netcdf(path: str | Path) -> PosteriorAnalysis

Read a QVR retained analysis without reconstructing a fitted evaluator.

PARAMETER DESCRIPTION
path

NetCDF file written from a QVR posterior DataTree.

TYPE: str or Path

RETURNS DESCRIPTION
PosteriorAnalysis

Owned CPU tensors and coordinate metadata, after axis validation.

RAISES DESCRIPTION
(ValueError, TypeError)

If the stored tree does not satisfy the retained analysis contract.

OSError

If the input file is missing or cannot be opened as NetCDF.

Source code in src/quivers/diagnostics/posterior.py
@classmethod
def from_netcdf(cls, path: str | Path) -> PosteriorAnalysis:
    """Read a QVR retained analysis without reconstructing a fitted evaluator.

    Parameters
    ----------
    path : str or Path
        NetCDF file written from a QVR posterior DataTree.

    Returns
    -------
    PosteriorAnalysis
        Owned CPU tensors and coordinate metadata, after axis validation.

    Raises
    ------
    ValueError, TypeError
        If the stored tree does not satisfy the retained analysis contract.
    OSError
        If the input file is missing or cannot be opened as NetCDF.
    """
    import xarray as xr

    with xr.open_datatree(path, engine="netcdf4") as tree:
        return cls.from_datatree(tree)

reloo

reloo(refit: Callable[[FoldSplit], FoldPrediction], *, var_name: str | None = None, k_threshold: float | None = None, seed: int = 0, pointwise: bool = True) -> object

Replace problematic PSIS terms with raw held-out refit predictions.

PARAMETER DESCRIPTION
refit

Existing source, formula, or user refit adapter. Each requested split excludes one original observation and scores that row alone.

TYPE: Callable[[FoldSplit], FoldPrediction]

var_name

Selected response; required when several responses are retained.

TYPE: str or None DEFAULT: None

k_threshold

Refit Pareto shapes above this value. None uses ArviZ's threshold based on retained sample count. Negative infinity refits all rows. Undefined tail fits are refitted unless the threshold is infinity.

TYPE: float or None DEFAULT: None

seed

Nonnegative base seed for independent, order-invariant row refits.

TYPE: int DEFAULT: 0

pointwise

Retain individual ELPD values and Pareto diagnostics in the result.

TYPE: bool DEFAULT: True

RETURNS DESCRIPTION
ELPDData

ArviZ's updated criterion, on the natural-log scale. Refit terms retain the supplied fitter's approximation and Monte Carlo error.

RAISES DESCRIPTION
(ValueError, TypeError)

If options, raw held-out scores, or observation axes are invalid.

ImportError

If the optional diagnostics dependencies are unavailable.

Source code in src/quivers/diagnostics/posterior.py
def reloo(
    self,
    refit: Callable[[FoldSplit], FoldPrediction],
    *,
    var_name: str | None = None,
    k_threshold: float | None = None,
    seed: int = 0,
    pointwise: bool = True,
) -> object:
    """Replace problematic PSIS terms with raw held-out refit predictions.

    Parameters
    ----------
    refit : Callable[[FoldSplit], FoldPrediction]
        Existing source, formula, or user refit adapter. Each requested
        split excludes one original observation and scores that row alone.
    var_name : str or None
        Selected response; required when several responses are retained.
    k_threshold : float or None
        Refit Pareto shapes above this value. None uses ArviZ's threshold
        based on retained sample count. Negative infinity refits all rows.
        Undefined tail fits are refitted unless the threshold is infinity.
    seed : int
        Nonnegative base seed for independent, order-invariant row refits.
    pointwise : bool
        Retain individual ELPD values and Pareto diagnostics in the result.

    Returns
    -------
    arviz.ELPDData
        ArviZ's updated criterion, on the natural-log scale. Refit terms
        retain the supplied fitter's approximation and Monte Carlo error.

    Raises
    ------
    ValueError, TypeError
        If options, raw held-out scores, or observation axes are invalid.
    ImportError
        If the optional diagnostics dependencies are unavailable.
    """
    from quivers.diagnostics._reloo import reloo

    return reloo(
        self,
        refit,
        var_name=var_name,
        k_threshold=k_threshold,
        seed=seed,
        pointwise=pointwise,
    )

loo

loo(*, var_name: str | None = None, pointwise: bool = True) -> object

Run optional ArviZ PSIS-LOO without recomputing model likelihoods.

PARAMETER DESCRIPTION
var_name

Observation variable; required for multiple responses.

TYPE: str or None DEFAULT: None

pointwise

Include observation-level ELPD and Pareto shape diagnostics.

TYPE: bool DEFAULT: True

RETURNS DESCRIPTION
ELPDData

ArviZ's PSIS-LOO result. Variational draws use relative efficiency one; this does not correct variational approximation error. If every likelihood column is constant across draws, exact uniform weights replace an undefined tail fit, and Pareto shapes are NaN.

Source code in src/quivers/diagnostics/posterior.py
def loo(self, *, var_name: str | None = None, pointwise: bool = True) -> object:
    """Run optional ArviZ PSIS-LOO without recomputing model likelihoods.

    Parameters
    ----------
    var_name : str or None
        Observation variable; required for multiple responses.
    pointwise : bool
        Include observation-level ELPD and Pareto shape diagnostics.

    Returns
    -------
    arviz.ELPDData
        ArviZ's PSIS-LOO result. Variational draws use relative efficiency
        one; this does not correct variational approximation error.
        If every likelihood column is constant across draws, exact uniform
        weights replace an undefined tail fit, and Pareto shapes are NaN.
    """
    import arviz as az

    name = self._likelihood_name(var_name)
    tree = self.to_datatree()
    scores = self.log_likelihood[name]
    if (
        scores.numel()
        and bool(torch.isfinite(scores).all())
        and torch.equal(scores, scores[0, 0].expand_as(scores))
    ):
        # Constant likelihood columns have exactly uniform importance
        # weights. No tail distribution exists to fit or diagnose.
        import xarray as xr

        likelihood = tree["log_likelihood"].dataset[name]
        count = scores.shape[0] * scores.shape[1]
        return az.loo(
            tree,
            var_name=name,
            pointwise=pointwise,
            reff=1.0,
            log_weights=xr.full_like(
                likelihood.transpose(..., "chain", "draw"), -math.log(count)
            ),
            pareto_k=xr.full_like(
                likelihood.isel(chain=0, draw=0, drop=True), math.nan
            ),
        )
    return az.loo(
        tree,
        var_name=name,
        pointwise=pointwise,
        reff=1.0 if self.kind != "mcmc" or not self.samples else None,
    )

waic

waic(log_likelihood: Tensor) -> WAICResult

Estimate WAIC from raw conditional log likelihoods.

PARAMETER DESCRIPTION
log_likelihood

Finite real scores with shape (chain, draw, *observation_shape). Event dimensions must already be reduced. Scores must refer to the same observation units across draws and exclude likelihood weights.

TYPE: Tensor

RETURNS DESCRIPTION
WAICResult

Log-mean-exp predictive scores minus sample log-score variances. The variance convention matches R's loo::waic (divisor S - 1).

RAISES DESCRIPTION
ValueError

If scores are nonfinite, empty, or supply fewer than two pooled draws.

TypeError

If scores are not a real floating-point tensor.

Source code in src/quivers/diagnostics/posterior.py
def waic(log_likelihood: torch.Tensor) -> WAICResult:
    """Estimate WAIC from raw conditional log likelihoods.

    Parameters
    ----------
    log_likelihood : torch.Tensor
        Finite real scores with shape ``(chain, draw, *observation_shape)``.
        Event dimensions must already be reduced. Scores must refer to the
        same observation units across draws and exclude likelihood weights.

    Returns
    -------
    WAICResult
        Log-mean-exp predictive scores minus sample log-score variances.
        The variance convention matches R's loo::waic (divisor S - 1).

    Raises
    ------
    ValueError
        If scores are nonfinite, empty, or supply fewer than two pooled draws.
    TypeError
        If scores are not a real floating-point tensor.
    """
    if (
        not isinstance(log_likelihood, torch.Tensor)
        or not log_likelihood.is_floating_point()
    ):
        raise TypeError("WAIC requires a real floating-point log-likelihood tensor")
    if log_likelihood.ndim < 3 or not all(log_likelihood.shape):
        raise ValueError("WAIC requires nonempty chain, draw, and observation axes")
    values = log_likelihood.detach().to(dtype=torch.float64)
    if not bool(torch.isfinite(values).all()):
        raise ValueError("WAIC requires finite log likelihoods at every posterior draw")
    draws = values.shape[0] * values.shape[1]
    if draws < 2:
        raise ValueError("WAIC requires at least two pooled posterior draws")
    pooled = values.flatten(0, 1)
    variance = pooled.var(dim=0, correction=1)
    elpd = torch.logsumexp(pooled, dim=0) - math.log(draws) - variance
    count = elpd.numel()
    return WAICResult(
        elpd_waic=elpd.sum().item(),
        p_waic=variance.sum().item(),
        waic=-2 * elpd.sum().item(),
        se=(count * elpd.flatten().var(correction=1)).sqrt().item()
        if count > 1
        else math.nan,
        se_p_waic=(count * variance.flatten().var(correction=1)).sqrt().item()
        if count > 1
        else math.nan,
        elpd_i=elpd,
        variance_i=variance,
        warning=bool((variance > 0.4).any()),
        num_samples=draws,
        num_observations=count,
    )

Cited sources

Aki Vehtari, Andrew Gelman, and Jonah Gabry. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27:1413–1432, 2017. URL: https://link.springer.com/article/10.1007/s11222-016-9696-4, doi:10.1007/s11222-016-9696-4.