Pathfinder¶
Pathfinder (Zhang et al., 2022b) is a variational method that follows an L-BFGS optimization path of the unconstrained log joint, forms a normal approximation at each iterate from the quasi-Newton inverse-Hessian estimate, and keeps the one with the largest estimated evidence lower bound (ELBO). Multi-path Pathfinder runs several independent paths and resamples their pooled draws with Pareto-smoothed importance weights (Vehtari et al., 2024). The formal statement gives the approximation, the ELBO estimate, and the resampling step.
Pathfinder takes CmdStan's
argument names. It scores the same Jacobian-corrected
PotentialFn as the
Hamiltonian kernels, over the latents one recorded run reaches. Its result
PathfinderDraws keeps the
layout of independent draws, one pseudo-chain of num_samples draws, with
each draw's target and approximation log densities (Stan's lp__ and
lp_approx__), the path it came from, each path's ELBO and stopping
reason, and the Pareto shape of the pooled importance ratios.
PathfinderMethod applies it to a
bound model, and fit(..., algorithm="pathfinder") to a formula.
import torch
from quivers.inference import Pathfinder
from quivers.modeling import Model, PathfinderMethod
source = """object Row : FinSet _
object Out : Real 1
program regression(x : Tensor[Real]([Row]), y : Tensor[Real]([Row])) : Row -> Out
sample b <- Normal(loc=0.0, scale=1.0)
let mu = b * x
observe y : Row <- Normal(loc=mu, scale=1.0)
return b
export regression
"""
x = torch.linspace(-1.0, 1.0, 12)
y = 0.6 * x + 0.1 * torch.sin(7.0 * x)
fitted = Model(source).fit(
{"x": x, "y": y},
method=PathfinderMethod(
pathfinder=Pathfinder(num_paths=2, num_draws=200, num_psis_draws=400, seed=1)
),
)
draws = fitted.posterior
assert draws.samples["b"].shape == (1, 400)
assert draws.num_paths == 2 and draws.resampled
Cost per path: the L-BFGS evaluations, then for each of the L iterates
O(N J^2 + J^3) to factor the approximation by a thin QR and a Cholesky
decomposition, and K log-density evaluations at O(N J + J^2) per draw for
the ELBO, with N the unconstrained dimension, J the history size, and K
the ELBO draws. No N-by-N matrix is formed. Paths run one after another.
The line search enforces the strong Wolfe conditions with cubic interpolation (Nocedal and Wright, 2006). Near-zero Wolfe constants make it an exact line search, under which the approximation on a Gaussian target is the target itself once the history size reaches the dimension.
pathfinder
¶
Pathfinder variational inference over the unconstrained log joint.
Pathfinder [@zhang2022pathfinder] runs L-BFGS on the Jacobian-corrected unconstrained log density of a program from a random initial point, forms a normal approximation at every iterate of the optimization path from the iterate, its gradient, and the L-BFGS inverse-Hessian estimate, selects the approximation with the largest Monte Carlo estimate of the evidence lower bound (ELBO), and draws from it. Multi-path Pathfinder runs several independent paths and resamples their pooled draws with replacement, with probabilities given by Pareto-smoothed importance weights.
The pieces follow the paper's algorithms:
lbfgs_path: L-BFGS (Algorithm 7) with a strong Wolfe line search and Stan's convergence tests, recording every iterate and gradient;recover_diagonal: the diagonal inverse-Hessian estimate of Gilbert and Lemarechal along the path (Algorithm 3);bfgs_approximation: the compact BFGS representation of [@byrd1994representations] at one iterate and its factorization by a thin QR and a Cholesky decomposition (Algorithm 4);single_path: ELBO selection and draws (Algorithm 1);psis_log_weights: Pareto-smoothed importance sampling as the loo package computes it [@vehtari2024psis];Pathfinder: the driver over a compiled program (Algorithm 2), whose resultPathfinderDrawsthe fit, prediction, and artifact machinery reads as independent draws.
Termination
¶
Termination = Literal['absolute objective', 'relative objective', 'absolute gradient', 'relative gradient', 'parameter change', 'maximum iterations', 'line search failed']
Why an L-BFGS path stopped.
InitialValues
¶
InitialValues = Mapping[str, Tensor | float]
Natural-scale starting values keyed by latent site.
PathfinderTarget
¶
Bases: Protocol
A log density on a flat unconstrained vector.
PotentialFn is one, through Pathfinder.run; a test target can be any
object with these members.
log_density
¶
value_and_grad
¶
Return the log density and its gradient at z.
| PARAMETER | DESCRIPTION |
|---|---|
z
|
Position, shape
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
tuple[Tensor, Tensor]
|
Scalar log density and gradient of shape |
Source code in src/quivers/inference/pathfinder.py
LBFGSSettings
didactic-model
¶
LBFGSSettings(*, history_size: int = 5, max_iterations: int = 1000, init_alpha: float = 0.001, tol_obj: float = 1e-12, tol_rel_obj: float = 10000.0, tol_grad: float = 1e-08, tol_rel_grad: float = 10000000.0, tol_param: float = 1e-08, wolfe_c1: float = 0.0001, wolfe_c2: float = 0.9, max_line_search: int = 40)
Bases: ValueModel
L-BFGS settings, named as CmdStan's optimizer and Pathfinder arguments.
| PARAMETER | DESCRIPTION |
|---|---|
history_size
|
Number of position and gradient updates kept (CmdStan
TYPE:
|
max_iterations
|
Maximum L-BFGS iterations (
TYPE:
|
init_alpha
|
Step length tried first on the first iteration (
TYPE:
|
tol_obj
|
Stop when the objective changes by less than this (
TYPE:
|
tol_rel_obj
|
Stop when the relative objective change is below this many machine
epsilons (
TYPE:
|
tol_grad
|
Stop when the gradient norm is below this (
TYPE:
|
tol_rel_grad
|
Stop when
TYPE:
|
tol_param
|
Stop when the position moves by less than this (
TYPE:
|
wolfe_c1
|
Sufficient decrease constant of the strong Wolfe conditions.
TYPE:
|
wolfe_c2
|
Curvature constant of the strong Wolfe conditions. Near zero, the line search returns the exact minimizer along the direction, which cubic interpolation reaches in one step on a quadratic.
TYPE:
|
max_line_search
|
Maximum function evaluations of one line search.
TYPE:
|
OptimizationPath
didactic-model
¶
OptimizationPath(*, positions: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_densities: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), gradients: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), termination: Termination, num_evaluations: int)
Bases: ValueModel
The iterates of one L-BFGS run on -log p.
| PARAMETER | DESCRIPTION |
|---|---|
positions
|
Iterates
TYPE:
|
log_densities
|
TYPE:
|
gradients
|
TYPE:
|
termination
|
Why the run stopped.
TYPE:
|
num_evaluations
|
Log-density and gradient evaluations, line searches included.
TYPE:
|
DiagonalRecovery
didactic-model
¶
DiagonalRecovery(*, alphas: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), accepted: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), steps: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), gradient_steps: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY))
Bases: ValueModel
Output of the paper's α-RECOVER (Algorithm 3).
| PARAMETER | DESCRIPTION |
|---|---|
alphas
|
Diagonal initial inverse-Hessian estimate after each step, shape
TYPE:
|
accepted
|
Boolean, whether step
TYPE:
|
steps
|
Position updates
TYPE:
|
gradient_steps
|
Updates
TYPE:
|
NormalApproximation
didactic-model
¶
NormalApproximation(*, mean: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), alpha: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), q: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), cholesky: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), beta: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), gamma: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_det: float)
Bases: ValueModel
The normal approximation at one iterate, in factored form.
The covariance is Sigma = diag(alpha) + beta gamma beta' (the
compact BFGS inverse-Hessian estimate) and is held through the thin QR
factorization Q R = diag(alpha)^(-1/2) beta and the Cholesky factor
L of I + R gamma R', so Sigma = T T' with
T = diag(alpha)^(1/2) (Q L Q' + I - Q Q'). Sampling and scoring cost
O(N J + J^2) per draw for history size J.
| PARAMETER | DESCRIPTION |
|---|---|
mean
|
TYPE:
|
alpha
|
Diagonal part, shape
TYPE:
|
q
|
Orthonormal factor, shape
TYPE:
|
cholesky
|
Lower Cholesky factor of
TYPE:
|
beta
|
TYPE:
|
gamma
|
Middle matrix of the compact representation, shape
TYPE:
|
log_det
|
TYPE:
|
sample
¶
Map standard normal noise to draws (Algorithm 4, line 11).
| PARAMETER | DESCRIPTION |
|---|---|
noise
|
Standard normal
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
|
Source code in src/quivers/inference/pathfinder.py
log_prob
¶
Score draws under the approximation.
The standard normal coordinates are recovered with the inverse of
I + Q (L - I) Q', which is I + Q (L^(-1) - I) Q', so the cost
is O(N J + J^2) per draw.
| PARAMETER | DESCRIPTION |
|---|---|
draws
|
Points, shape
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
Log densities, shape |
Source code in src/quivers/inference/pathfinder.py
SinglePath
didactic-model
¶
SinglePath(*, path: OptimizationPath = field(opaque=True, extras=OPAQUE_BY_IDENTITY), elbo: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), best_iteration: int, draws: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_density: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_approx: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), failed: bool)
Bases: ValueModel
One Pathfinder run (Algorithm 1).
| PARAMETER | DESCRIPTION |
|---|---|
path
|
The L-BFGS iterates.
TYPE:
|
elbo
|
ELBO estimate at iterates
TYPE:
|
best_iteration
|
Iterate
TYPE:
|
draws
|
Unconstrained draws from the selected approximation, shape
TYPE:
|
log_density
|
Target log density at each draw (NaN without
TYPE:
|
log_approx
|
Approximation log density at each draw.
TYPE:
|
failed
|
Whether no iterate gave a usable approximation. A failed path keeps
its last iterate as its draws with
TYPE:
|
PathfinderDraws
didactic-model
¶
PathfinderDraws(*, samples: Mapping[str, Tensor] = field(opaque=True, extras=OPAQUE_BY_IDENTITY), num_samples: int, log_density: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), log_approx: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), path_index: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), path_elbo: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), path_iterations: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), path_best_iteration: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), path_termination: tuple[str, ...] = (), num_evaluations: Tensor = field(opaque=True, extras=OPAQUE_BY_IDENTITY), pareto_k: float = nan, resampled: bool = False, single_paths: Mapping[str, Tensor] = field(default_factory=dict, opaque=True, extras=OPAQUE_BY_IDENTITY))
Bases: PriorDraws
Pathfinder draws, read by the fit machinery as independent draws.
The samples share PriorDraws's layout (1, draw, *event_shape):
importance-resampled draws are not a Markov chain and carry no chain
diagnostics. draw_provenance names them variational.
| PARAMETER | DESCRIPTION |
|---|---|
samples
|
Constrained draws
TYPE:
|
num_samples
|
Number of draws.
TYPE:
|
log_density
|
Unconstrained log joint (Stan's
TYPE:
|
log_approx
|
Approximation log density (
TYPE:
|
path_index
|
Path each draw came from, shape
TYPE:
|
path_elbo
|
Largest ELBO estimate of each path,
TYPE:
|
path_iterations
|
L-BFGS iterations of each path.
TYPE:
|
path_best_iteration
|
Selected iterate of each path.
TYPE:
|
path_termination
|
Why each path's L-BFGS run stopped.
TYPE:
|
num_evaluations
|
Log-density evaluations of each path.
TYPE:
|
pareto_k
|
Pareto shape of the pooled importance ratios; NaN without resampling.
TYPE:
|
resampled
|
Whether the draws were importance resampled.
TYPE:
|
single_paths
|
With
TYPE:
|
Pathfinder
didactic-model
¶
Pathfinder(*, num_paths: int = 4, num_draws: int = 1000, num_psis_draws: int = 1000, num_elbo_draws: int = 25, psis_resample: bool = True, calculate_lp: bool = True, save_single_paths: bool = False, lbfgs: LBFGSSettings = field(default_factory=LBFGSSettings), init_radius: float = 2.0, init_strategy: Literal['uniform', 'zero'] = 'uniform', initial_values: InitialValues | tuple[InitialValues, ...] | None = field(default=None, opaque=True, extras=OPAQUE_BY_IDENTITY), seed: int = 0)
Bases: ValueModel
Single- and multi-path Pathfinder over a compiled program.
Parameter names follow CmdStan's pathfinder method; brms maps
chains to num_paths. With one path the draws are that path's
num_draws draws. With several, the num_paths * num_draws pooled
draws are resampled with replacement to num_psis_draws draws with
Pareto-smoothed importance weights p / q (Algorithm 2), unless
psis_resample or calculate_lp is False, in which case every
pooled draw is returned. Paths run one after another, each from its own
stream derived from seed.
| PARAMETER | DESCRIPTION |
|---|---|
num_paths
|
Independent runs
TYPE:
|
num_draws
|
Draws per path
TYPE:
|
num_psis_draws
|
Draws after resampling
TYPE:
|
num_elbo_draws
|
Draws per ELBO estimate
TYPE:
|
psis_resample
|
Whether to importance resample across paths.
TYPE:
|
calculate_lp
|
Whether to evaluate the target at the returned draws.
TYPE:
|
save_single_paths
|
Whether to keep each path's draws in the result.
TYPE:
|
lbfgs
|
L-BFGS settings (
TYPE:
|
init_radius
|
Initial points are uniform on
TYPE:
|
init_strategy
|
Uniform initial points, or the origin.
TYPE:
|
initial_values
|
Natural-scale values replacing the named sites' initial coordinates, shared or one mapping per path.
TYPE:
|
seed
|
Base seed of every path's streams.
TYPE:
|
run
¶
run(model: MonadicProgram, x: Tensor, observations: dict[str, Tensor]) -> PathfinderDraws
Run Pathfinder on a program's unconstrained log joint.
The latents are those one recorded run of the program reaches, as
for MCMC, and the target is the same Jacobian-corrected
PotentialFn the Hamiltonian kernels use.
| PARAMETER | DESCRIPTION |
|---|---|
model
|
The program.
TYPE:
|
x
|
One-row program input.
TYPE:
|
observations
|
Observed values and data.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
PathfinderDraws
|
Constrained draws and per-path diagnostics. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the input has several rows, the data lie outside their support, an initial value is invalid, an initial point has a nonfinite log density, or every path fails. |
Source code in src/quivers/inference/pathfinder.py
run_target
¶
run_target(target: PathfinderTarget, dim: int, *, registry: LatentRegistry | None = None) -> PathfinderDraws
Run Pathfinder on a log density over R^dim.
| PARAMETER | DESCRIPTION |
|---|---|
target
|
The log density.
TYPE:
|
dim
|
Dimension
TYPE:
|
registry
|
The program's registry; its sites name the result's samples.
Without one, the result holds the unconstrained draws as site
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
PathfinderDraws
|
Draws and per-path diagnostics. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If every path fails. |
Source code in src/quivers/inference/pathfinder.py
1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 | |
lbfgs_path
¶
lbfgs_path(target: PathfinderTarget, initial: Tensor, settings: LBFGSSettings) -> OptimizationPath
Maximize log p by L-BFGS, recording every iterate.
The search direction is the two-loop recursion over the last
history_size pairs with the scaled identity
(s'y / y'y) I as the initial inverse Hessian; a pair enters the
history when s'y > eps |y|^2. The first step tries length
init_alpha along the gradient, later steps length one. The line
search enforces the strong Wolfe conditions by bracketing and cubic
interpolation [@nocedal2006numerical]. The run stops on the first of
Stan's tests that holds, in Stan's order: absolute and relative objective
change, absolute and relative gradient, and parameter change.
| PARAMETER | DESCRIPTION |
|---|---|
target
|
The log density.
TYPE:
|
initial
|
Starting position, shape
TYPE:
|
settings
|
Optimizer settings.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
OptimizationPath
|
Iterates, log densities, gradients, and the stopping reason. Cost is O(L J N) besides the evaluations. |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the log density or its gradient at |
Source code in src/quivers/inference/pathfinder.py
372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 | |
recover_diagonal
¶
recover_diagonal(path: OptimizationPath) -> DiagonalRecovery
Recover the diagonal inverse-Hessian estimates along a path.
Algorithm 3 of [@zhang2022pathfinder]: starting from alpha = 1, each
step with s'z > 1e-12 |z|^2 updates alpha elementwise by the
diagonal BFGS formula of Gilbert and Lemarechal,
1 / alpha_n = a / (b alpha_n) + z_n^2 / b - a s_n^2 / (b c alpha_n^2)
with a = z' diag(alpha) z, b = z' s, c = s' diag(alpha)^(-1) s
[@gilbert1989numerical], and is marked accepted. The paper's printed condition has its
inequality reversed; the condition here is the one its text states,
which keeps steps whose gradient change is not 10^12 times larger than
the position change along it.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
L-BFGS iterates and gradients.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
DiagonalRecovery
|
Per-step diagonals, acceptance marks, and updates; O(L N). |
Source code in src/quivers/inference/pathfinder.py
compact_factors
¶
Compact BFGS representation of the inverse Hessian.
Equation (2) of [@zhang2022pathfinder], after Byrd, Nocedal and Schnabel
(1994, eq. 2.6): beta = [diag(alpha) Z, S] and
gamma = [[0, -E^(-1)], [-E^(-T), E^(-T) (diag(eta) + Z' diag(alpha) Z) E^(-1)]]
with E the upper triangle of S'Z and eta its diagonal, so that
diag(alpha) + beta gamma beta' equals the BFGS updates of
diag(alpha) by the pairs in order.
| PARAMETER | DESCRIPTION |
|---|---|
alpha
|
Initial diagonal, shape
TYPE:
|
s
|
Position updates as columns, shape
TYPE:
|
z
|
Gradient updates of
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
tuple[Tensor, Tensor]
|
|
Source code in src/quivers/inference/pathfinder.py
normal_approximation
¶
normal_approximation(position: Tensor, gradient: Tensor, alpha: Tensor, s: Tensor, z: Tensor) -> NormalApproximation | None
Factor the normal approximation at one iterate (Algorithm 4).
| PARAMETER | DESCRIPTION |
|---|---|
position
|
Iterate
TYPE:
|
gradient
|
TYPE:
|
alpha
|
Diagonal estimate, shape
TYPE:
|
s
|
Selected update pairs as columns, shape
TYPE:
|
z
|
Selected update pairs as columns, shape
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
NormalApproximation or None
|
The factored approximation, or None when the estimate is not positive definite. Cost O(N m^2 + m^3). |
Source code in src/quivers/inference/pathfinder.py
bfgs_approximation
¶
bfgs_approximation(path: OptimizationPath, recovery: DiagonalRecovery, iteration: int, history_size: int) -> NormalApproximation | None
The normal approximation at iterate iteration of a path.
It uses the last history_size accepted pairs among steps
1..iteration and the diagonal recovered at that iterate.
| PARAMETER | DESCRIPTION |
|---|---|
path
|
The L-BFGS iterates.
TYPE:
|
recovery
|
α-RECOVER output for the path.
TYPE:
|
iteration
|
Iterate
TYPE:
|
history_size
|
Number
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
NormalApproximation or None
|
The approximation, or None when it is not positive definite. |
Source code in src/quivers/inference/pathfinder.py
stream
¶
stream(seed: int, index: int) -> Generator
Derive an independent generator from a seed by SplitMix64.
The mixing is the one MCMC uses to give each chain its stream.
| PARAMETER | DESCRIPTION |
|---|---|
seed
|
Base seed.
TYPE:
|
index
|
Stream index.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Generator
|
A CPU generator seeded with the mixed value. |
Source code in src/quivers/inference/pathfinder.py
single_path
¶
single_path(target: PathfinderTarget, initial: Tensor, *, seed: int, settings: LBFGSSettings, num_elbo_draws: int, num_draws: int, calculate_lp: bool = True) -> SinglePath
Run single-path Pathfinder from one initial point.
The ELBO at iterate l averages log p(phi) - log q(phi) over
num_elbo_draws draws of stream stream(seed, l); the returned
draws come from stream stream(seed, -1).
| PARAMETER | DESCRIPTION |
|---|---|
target
|
The log density.
TYPE:
|
initial
|
Starting position, shape
TYPE:
|
seed
|
Base of the path's random streams.
TYPE:
|
settings
|
L-BFGS settings, including the history size.
TYPE:
|
num_elbo_draws
|
Draws per ELBO estimate
TYPE:
|
num_draws
|
Draws returned
TYPE:
|
calculate_lp
|
Whether to evaluate the target at the returned draws.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
SinglePath
|
The path, its ELBO profile, and the draws. Cost is the L-BFGS
evaluations plus |
Source code in src/quivers/inference/pathfinder.py
972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 | |
psis_log_weights
¶
Pareto-smooth importance log ratios as loo's psis with r_eff = 1.
The largest M = ceil(min(0.2 S, 3 sqrt(S))) ratios are replaced by
the quantiles (i - 1/2) / M of a generalized Pareto distribution fit
to their exceedances over the next largest ratio, and every ratio is
truncated at the largest raw ratio [@vehtari2024psis]. Ratios of
-inf keep zero weight.
| PARAMETER | DESCRIPTION |
|---|---|
log_ratios
|
Log importance ratios
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
tuple[Tensor, float]
|
Normalized smoothed log weights and the Pareto shape |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If no ratio is finite. |
Source code in src/quivers/inference/pathfinder.py
draw_provenance
¶
draw_provenance(posterior: MCMCResult | PriorDraws | Guide) -> Literal['mcmc', 'variational', 'prior']
Name where retained draws come from.
| PARAMETER | DESCRIPTION |
|---|---|
posterior
|
A fit's posterior.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
{'mcmc', 'variational', 'prior'}
|
|
Source code in src/quivers/inference/pathfinder.py
Cited sources¶
Lu Zhang, Bob Carpenter, Andrew Gelman, and Aki Vehtari. Pathfinder: Parallel quasi-Newton variational inference. Journal of Machine Learning Research, 23(306):1–49, 2022b. URL: https://jmlr.org/papers/v23/21-0889.html.
Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao, and Jonah Gabry. Pareto Smoothed Importance Sampling. Journal of Machine Learning Research, 25(72):1–58, 2024. URL: https://www.jmlr.org/papers/v25/19-556.html.
Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, 2 edition, 2006. doi:10.1007/978-0-387-40065-5.