Initialization recipes¶

init_spec ¶

Algebra-specific initialization recipes.

The Algebra's tensor / join operations determine where in its value space a k-step composition lands when entries are i.i.d. and roughly mean-zero. For each built-in algebra we record the parameters of a one-dimensional initial distribution that places the k-step output in a reference region where the closed-form approximation predicts larger gradients. The shipped recipes are:

ProductFuzzyAlgebra — noisy-OR over k cells lands at ½ when each cell is initialized at p ≈ ln 2 / k.
LogProbAlgebra — log-domain; centered at -ln k so the exponentiated value is 1 / k.
MarkovAlgebra — row logits centered at 0 (uniform prior over the row simplex).
MaxPlusAlgebra / TropicalAlgebra — additive semiring on the (extended) reals; centered at 0 with spread 1 / sqrt(k).
LukasiewiczAlgebra — bounded-sum t-conorm hits ½ at roughly k terms of p ≈ 1 / k.
BooleanAlgebra, GodelAlgebra — idempotent; init at the algebra's mid-point 0.5.
RealAlgebra, ProbabilityAlgebra, CountingAlgebra — sum-product semirings; we recommend unit-variance Normal init centered at 0 (real) or an algebra-specific 1 / k (probability / counting). These pieces are conservative defaults pending a fuller writeup of the semiring case.

The InitSpec produced by Algebra.init_spec is distribution-shaped (mean, std, lower, upper, distribution name). Sampling itself is delegated to torch.nn.init via apply_init_spec.

InitSpec `didactic-model` ¶

InitSpec(*, distribution: Literal['normal', 'uniform', 'constant'] = 'normal', mean: float = 0.0, std: float = 1.0, lower: float = float('-inf'), upper: float = float('inf'), rationale: str = '')

Bases: Model

Parameters of a one-dimensional initial distribution.

ATTRIBUTE	DESCRIPTION
`distribution`	One of `"normal"` / `"uniform"` / `"constant"`. TYPE: `str`
`mean`	Location parameter for `"normal"` / `"constant"`; the midpoint `(lower + upper) / 2` for `"uniform"`. TYPE: `float`
`std`	Scale parameter for `"normal"`; the half-width `(upper - lower) / 2` for `"uniform"`; `0.0` for `"constant"`. TYPE: `float`
`lower, upper`	Bounds for `"uniform"`; for `"normal"` / `"constant"` they record the algebra's value-space bounds if any (`-inf` / `inf` otherwise). TYPE: `float`
`rationale`	One-line explanation of the recipe, suitable for surfacing in a warning or in `qvr check` output. TYPE: `str`

recommend_init ¶

recommend_init(module: Module) -> dict[str, InitSpec]

Per-latent algebra-specific initialization recipe.

Walks the program's ChainShape, looks up the governing algebra, and returns a mapping from latent variable name to the InitSpec that places the chain at the algebra's mid-saturation point at depth-and-size.

PARAMETER	DESCRIPTION
`module`	The parsed QVR module whose program is analysed. TYPE: `Module`

RETURNS	DESCRIPTION
`dict[str, InitSpec]`	The recipe for each latent, keyed by variable name; empty when the module declares no governing algebra.

Source code in src/quivers/analysis/init_spec.py

def recommend_init(module: Module) -> dict[str, InitSpec]:
    """Per-latent algebra-specific initialization recipe.

    Walks the program's `ChainShape`, looks up the governing
    algebra, and returns a mapping from latent variable name to the
    `InitSpec` that places the chain at the algebra's
    mid-saturation point at depth-and-size.

    Parameters
    ----------
    module : Module
        The parsed QVR module whose program is analysed.

    Returns
    -------
    dict[str, InitSpec]
        The recipe for each latent, keyed by variable name; empty when
        the module declares no governing algebra.
    """
    shape = ChainShape.from_module(module)
    algebra = shape.algebra
    if algebra is None:
        return {}
    out: dict[str, InitSpec] = {}
    for step in shape.steps:
        if step.kind != "latent":
            continue
        size = step.intermediate_size or 1
        out[step.name] = _algebra_init_spec(algebra, step.depth, size)
    return out

apply_init_spec ¶

apply_init_spec(parameter: Parameter | Tensor, spec: InitSpec) -> None

Materialize an InitSpec onto a learnable tensor.

Delegates to torch.nn.init for normal / uniform and to a plain fill_ for constant. The tensor's shape is preserved; only its entries are overwritten in-place.

PARAMETER	DESCRIPTION
`parameter`	The tensor to overwrite in place. TYPE: `Parameter or Tensor`
`spec`	The recipe to draw the new entries from. TYPE: `InitSpec`