API

BernoulliLikelihood(l=logistic)

Bernoulli likelihood is to be used if we assume that the uncertainity associated with the data follows a Bernoulli distribution. The link l needs to transform the input f to the domain [0, 1]

\[ p(y|f) = \operatorname{Bernoulli}(y | l(f))\]

On calling, this would return a Bernoulli distribution with l(f) probability of true.

CategoricalLikelihood(l=BijectiveSimplexLink(softmax))

Categorical likelihood is to be used if we assume that the uncertainty associated with the data follows a Categorical distribution.

Assuming a distribution with n categories:

n-1 inputs (bijective link)

One can work with a bijective transformation by wrapping a link (like softmax) into a BijectiveSimplexLink and only needs n-1 inputs:

\[ p(y|f_1, f_2, \dots, f_{n-1}) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_{n-1}, 0))\]

The default constructor is a bijective link around softmax.

n inputs (non-bijective link)

One can also pass directly the inputs without concatenating a 0:

\[ p(y|f_1, f_2, \dots, f_n) = \operatorname{Categorical}(y | l(f_1, f_2, \dots, f_n))\]

This variant is over-parametrized, as there are n-1 independent parameters embedded in a n dimensional parameter space. For more details, see the end of the section of this Wikipedia link where it corresponds to Variant 1 and 2.

ExponentialLikelihood(l=exp)

Exponential likelihood with scale given by l(f).

\[ p(y|f) = \operatorname{Exponential}(y | l(f))\]

GammaLikelihood(α::Real=1.0, l=exp)

Gamma likelihood with fixed shape α.

\[ p(y|f) = \operatorname{Gamma}(y | α, l(f))\]

On calling, this returns a Gamma distribution with shape α and scale invlink(f).

GaussianLikelihood(σ²)

Gaussian likelihood with σ² variance. This is to be used if we assume that the uncertainity associated with the data follows a Gaussian distribution.

\[ p(y|f) = \operatorname{Normal}(y | f, σ²)\]

On calling, this would return a normal distribution with mean f and variance σ².

HeteroscedasticGaussianLikelihood(l=exp)

Heteroscedastic Gaussian likelihood. This is a Gaussian likelihood whose mean and variance are functions of latent processes.

\[ p(y|[f, g]) = \operatorname{Normal}(y | f, sqrt(l(g)))\]

On calling, this would return a normal distribution with mean f and variance l(g). Where l is link going from R to R^+

PoissonLikelihood(l=exp)

Poisson likelihood with rate defined as l(f).

\[ p(y|f) = \operatorname{Poisson}(y | θ=l(f))\]

This is to be used if we assume that the uncertainity associated with the data follows a Poisson distribution.

NegativeBinomialLikelihood(param::NBParam, invlink::Union{Function,Link})

There are many possible parametrizations for the Negative Binomial likelihood. We follow the convention laid out in p.137 of [^Hilbe'11] and provide some common parametrizations. The NegativeBinomialLikelihood has a special structure; the first type parameter NBParam defines in what parametrization the latent function is used, and contains the other (scalar) parameters. NBParam itself has two subtypes:

• NBParamProb for parametrizations where f -> p, the probability of success of a Bernoulli event
• NBParamMean for parametrizations where f -> μ, the expected number of events

NBParam predefined types

NBParamProb types with p = invlink(f) the probability of success or failure

• NBParamSuccess: Here p = invlink(f) is the probability of success. This is the definition used in Distributions.jl.
• NBParamFailure: Here p = invlink(f) is the probability of a failure

NBParamMean types with μ = invlink(f) the mean/expected number of events

• NBParamI: Mean is linked to f and variance is given by μ(1 + α)
• NBParamII: Mean is linked to f and variance is given by μ(1 + α * μ)
• NBParamPower: Mean is linked to f and variance is given by μ(1 + α * μ^ρ)

To create a new parametrization, you need to:

• create a new type struct MyNBParam{T} <: NBParam; myparams::T; end;
• dispatch (l::NegativeBinomialLikelihood{<:MyNBParam})(f::Real), which must return a NegativeBinomial from Distributions.jl.

NegativeBinomial follows the parametrization of NBParamSuccess, i.e. the first argument is the number of successes and the second argument is the probability of success.

Examples

julia> NegativeBinomialLikelihood(NBParamSuccess(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.8807970779778824)
julia> NegativeBinomialLikelihood(NBParamFailure(10), logistic)(2.0)
NegativeBinomial{Float64}(r=10.0, p=0.11920292202211757)

julia> d = NegativeBinomialLikelihood(NBParamI(3.0), exp)(2.0)
NegativeBinomial{Float64}(r=2.4630186996435506, p=0.25)
julia> mean(d) ≈ exp(2.0)
true
julia> var(d) ≈ exp(2.0) * (1 + 3.0)
true

[^Hilbe'11]: Hilbe, Joseph M. Negative binomial regression. Cambridge University Press, 2011.

NBParamSuccess(successes)

Negative Binomial parametrization with successes the number of successes and invlink(f) the probability of success. This corresponds to the definition used by Distributions.jl.

\[ p(y|\text{successes}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{successes})}{y! \Gamma(\text{successes})} p^\text{successes} (1 - p)^y\]

NBParamFailure(failures)

Negative Binomial parametrization with failures the number of failures and invlink(f) the probability of success. This corresponds to the definition used by Wikipedia.

\[ p(y|\text{failures}, p=\text{invlink}(f)) = \frac{\Gamma(y+\text{failures})}{y! \Gamma(\text{failures})} p^y (1 - p)^{\text{failures}}\]

NBParamI(α)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α) where α > 0.

NBParamII(α)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ) where α > 0.

NBParamPower(α, ρ)

Negative Binomial parametrization with mean μ = invlink(f) and variance v = μ(1 + α * μ^ρ) where α > 0 and ρ > 0.

Link(f)

General construction for a link with a function f.

ChainLink(links)

Create a composed chain of different links.

BijectiveSimplexLink(link)

Wrapper to preprocess the inputs by adding a 0 at the end before passing it to the link link. This is a necessary step to work with simplices. For example with the SoftMaxLink, to obtain a n-simplex leading to n+1 categories for the CategoricalLikelihood, one needs to pass n+1 latent GP. However, by wrapping the link into a BijectiveSimplexLink, only n latent are needed.

LogLink()

log link, f:ℝ⁺->ℝ . Its inverse is the ExpLink.

ExpLink()

exp link, f:ℝ->ℝ⁺. Its inverse is the LogLink.

InvLink()

inv link, f:ℝ/{0}->ℝ/{0}. It is its own inverse.

SqrtLink()

sqrt link, f:ℝ⁺∪{0}->ℝ⁺∪{0}. Its inverse is the SquareLink.

SquareLink()

^2 link, f:ℝ->ℝ⁺∪{0}. Its inverse is the SqrtLink.

LogitLink()

log(x/(1-x)) link, f:[0,1]->ℝ. Its inverse is the LogisticLink.

LogisticLink()

1/(1+exp(-x)) link. f:ℝ->[0,1]. Its inverse is the LogitLink.

ProbitLink()

ϕ⁻¹(y) link, where ϕ⁻¹ is the invcdf of a Normal distribution, f:[0,1]->ℝ. Its inverse is the NormalCDFLink.

NormalCDFLink()

ϕ(y) link, where ϕ is the cdf of a Normal distribution, f:ℝ->[0,1]. Its inverse is the ProbitLink.

SoftMaxLink()

softmax link, i.e f(x)ᵢ = exp(xᵢ)/∑ⱼexp(xⱼ). f:ℝⁿ->Sⁿ⁻¹, where Sⁿ⁻¹ is an (n-1)-simplex It has no defined inverse

expected_loglikelihood(
    quadrature,
    lik,
    q_f::AbstractVector{<:Normal},
    y::AbstractVector,
)

This function computes the expected log likelihood:

\[ ∫ q(f) log p(y | f) df\]

where p(y | f) is the process likelihood. This is described by lik, which should be a callable that takes f as input and returns a Distribution over y that supports loglikelihood(lik(f), y).

q(f) is an approximation to the latent function values f given by:

\[ q(f) = ∫ p(f | u) q(u) du\]

where q(u) is the variational distribution over inducing points. The marginal distributions of q(f) are given by q_f.

quadrature determines which method is used to calculate the expected log likelihood.

Extended help

q(f) is assumed to be an MvNormal distribution and p(y | f) is assumed to have independent marginals such that only the marginals of q(f) are required.

expected_loglikelihood(::DefaultExpectationMethod, lik, q_f::AbstractVector{<:Normal}, y::AbstractVector)

The expected log likelihood, using the default quadrature method for the given likelihood. (The default quadrature method is defined by default_expectation_method(lik), and should be the closed form solution if it exists, but otherwise defaults to Gauss-Hermite quadrature.)