Log-Space Families¶
A Beta or
Gamma variable with a small shape parameter
keeps most of its mass within a few multiples of the smallest positive floating-point number, where
scoring it through a sigmoid or an exponential loses its tails. These distributions live on the
real line instead. LogitBeta is the distribution of the
logit of a Beta variable, and
LogGamma that of the log of a Gamma variable, each with
its exact density and a sampler that stays in log space. The
continuous families guide
says when to use them.
log_space
¶
Distributions of the logit of a Beta variable and the log of a Gamma variable.
A Beta or Gamma site with a small shape parameter puts most of its mass within a few multiples of the smallest floating-point number of zero (or of one). Scored on the constrained scale, through a sigmoid or an exponential and the Jacobian of that map, such a site loses its tails: the sigmoid saturates long before the density of the logit does, and a Gamma sampler that returns the smallest normal number in place of a smaller draw no longer samples the declared prior.
The two families here live on the real line and carry the density of the transformed variable directly.
LogitBeta
The distribution of x = logit(p) for p ~ Beta(a, b), with
density a * logsigmoid(x) + b * logsigmoid(-x) - log B(a, b).
LogGamma
The distribution of y = log(g) for g ~ Gamma(a, r), with
density a * (log r + y) - r * exp(y) - log Gamma(a).
Both densities are exact for every finite argument: no intermediate
quantity is formed on the constrained scale. Sampling stays in log space
through the shape-augmentation identity: if G' ~ Gamma(a + 1) and
U ~ Uniform(0, 1) are independent, then G' * U^(1/a) ~ Gamma(a)
(Marsaglia and Tsang 2000), so log G' + log(U) / a is a log-Gamma draw
that never forms the Gamma variable itself. A logit-Beta draw is the
difference of two such draws, since G_a / (G_a + G_b) ~ Beta(a, b)
for independent unit-rate Gamma variables. G' has shape at least
one, so it never approaches the underflow threshold, and its implicit
reparameterization gradient carries through, which makes both samplers
reparameterized.
LogGamma
¶
Bases: Distribution
The log of a Gamma variable: y = log(g) for g ~ Gamma(a, r).
The density on the real line is
log p(y) = a * (log r + y) - r * exp(y) - lgamma(a),
exact for every finite y and every positive shape. Draws are
made in log space by shape augmentation (see
log_standard_gamma),
so a shape of 1e-8 yields draws near -1e8 rather than the
clamped smallest normal number a constrained-scale sampler returns.
TensorFlow Probability ships the same family as ExpGamma.
| PARAMETER | DESCRIPTION |
|---|---|
concentration
|
The Gamma shape
TYPE:
|
rate
|
The Gamma rate
TYPE:
|
validate_args
|
Whether to check parameter constraints and the support of scored values.
TYPE:
|
Source code in src/quivers/continuous/log_space.py
expand
¶
Broadcast the parameters to a larger batch shape.
| PARAMETER | DESCRIPTION |
|---|---|
batch_shape
|
The batch shape to expand to.
TYPE:
|
_instance
|
The instance torch's
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
LogGamma
|
The expanded distribution. |
Source code in src/quivers/continuous/log_space.py
rsample
¶
Draw reparameterized samples in log space.
| PARAMETER | DESCRIPTION |
|---|---|
sample_shape
|
Leading sample dimensions. |
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
Draws of shape |
Source code in src/quivers/continuous/log_space.py
log_prob
¶
The log density at value.
| PARAMETER | DESCRIPTION |
|---|---|
value
|
Points on the real line.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
|
Source code in src/quivers/continuous/log_space.py
LogitBeta
¶
LogitBeta(concentration1: _Parameter, concentration0: _Parameter, validate_args: bool | None = None)
Bases: Distribution
The logit of a Beta variable: x = logit(p) for p ~ Beta(a, b).
The density on the real line is
log p(x) = a * logsigmoid(x) + b * logsigmoid(-x) - log B(a, b),
exact for every finite x: at a = 1e-3, b = 1 and x =
-800 it is about -7.71, where scoring sigmoid(x) under a
Beta density with the sigmoid's Jacobian saturates and returns about
-99. A draw is the difference of two log-Gamma draws made in log
space (see
log_standard_gamma),
so it is finite and reparameterized for shapes far below where a
Beta sampler returns exactly zero or one.
| PARAMETER | DESCRIPTION |
|---|---|
concentration1
|
The Beta shape
TYPE:
|
concentration0
|
The Beta shape
TYPE:
|
validate_args
|
Whether to check parameter constraints and the support of scored values.
TYPE:
|
Source code in src/quivers/continuous/log_space.py
expand
¶
Broadcast the parameters to a larger batch shape.
| PARAMETER | DESCRIPTION |
|---|---|
batch_shape
|
The batch shape to expand to.
TYPE:
|
_instance
|
The instance torch's
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
LogitBeta
|
The expanded distribution. |
Source code in src/quivers/continuous/log_space.py
rsample
¶
Draw reparameterized samples as a difference of log-Gamma draws.
| PARAMETER | DESCRIPTION |
|---|---|
sample_shape
|
Leading sample dimensions. |
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
Draws of shape |
Source code in src/quivers/continuous/log_space.py
log_prob
¶
The log density at value.
| PARAMETER | DESCRIPTION |
|---|---|
value
|
Points on the real line.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
|
Source code in src/quivers/continuous/log_space.py
entropy
¶
entropy() -> Tensor
log B(a, b) - a (digamma(a) - digamma(a + b)) - b (digamma(b) - digamma(a + b)).
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
The differential entropy, one per batch element. |
Source code in src/quivers/continuous/log_space.py
log_standard_gamma
¶
Draw log(G) for G ~ Gamma(concentration, 1) in log space.
The draw is log G' + log(U) / concentration with G' ~
Gamma(concentration + 1, 1) and U ~ Uniform(0, 1], which has
the law of the log of a Gamma variable for every positive
concentration and never forms that variable. Gradients reach the
concentration through the implicit reparameterization of G' and
through the division.
| PARAMETER | DESCRIPTION |
|---|---|
concentration
|
Positive shape parameters, one per draw.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
Tensor
|
One log-Gamma draw per entry of |