Probability Distributions

Math.NET Numerics provides a wide range of probability distributions. Given the distribution parameters they can be used to investigate their statistical properties or to sample non-uniform random numbers.

All the distributions implement a common set of operations such as evaluating the density (PDF) and the cumulative distribution (CDF) at a given point, or to compute the mean, standard deviation and other properties. Because it is often numerically more stable and faster to compute such statistical quantities in the logarithmic domain, we also provide a selection of them in the log domain with the "Ln" suffix, e.g. DensityLn for the logarithmic density.

using MathNet.Numerics.Distributions;
using MathNet.Numerics.Random;

// create a parametrized distribution instance
var gamma = new Gamma(2.0, 1.5);

// distribution properties
double mean = gamma.Mean;
double variance = gamma.Variance;
double entropy = gamma.Entropy;

// distribution functions
double a = gamma.Density(2.3); // PDF
double b = gamma.DensityLn(2.3); // ln(PDF)
double c = gamma.CumulativeDistribution(0.7); // CDF

// non-uniform number sampling
double randomSample = gamma.Sample();

Both probability functions and sampling are also available as static functions for simpler usage scenarios:

// distribution parameters must be passed as arguments
double a2 = Gamma.PDF(2.0, 1.5, 2.3);
double randomSample2 = Gamma.Sample(2.0, 1.5);

Distribution Parameters

There are many ways to parametrize a distribution in the literature. When using the default constructor, read carefully which parameters it requires. For distributions where multiple ways are common there are also static methods, so you can use the one that fits best. For example, a normal distribution is usually parametrized with mean and standard deviation, but if you'd rather use mean and precision:

var normal = Normal.WithMeanPrecision(0.0, 0.5);

Since probability distributions can also be sampled to generate random numbers with the configured distribution, all constructors optionally accept a random generator as last argument.

var gamma2 = new Gamma(2.0, 1.5, new MersenneTwister());

// the random generator can also be replaced on an existing instance
gamma2.RandomSource = new Mrg32k3a();

A few more examples, this time in F#:

// some probability distributions
let normal = Normal.WithMeanVariance(3.0, 1.5, a)
let exponential = Exponential(2.4)
let gamma = Gamma(2.0, 1.5, Random.crypto())
let cauchy = Cauchy(0.0, 1.0, Random.mrg32k3aWith 10 false)
let poisson = Poisson(3.0)
let geometric = Geometric(0.8, Random.system())

Some of the distributions also have routines for maximum-likelihood parameter estimation from a set of samples:

let estimation = LogNormal.Estimate([| 2.0; 1.5; 2.1; 1.2; 3.0; 2.4; 1.8 |])
let mean, variance = estimation.Mean, estimation.Variance
let moreSamples = estimation.Samples() |> Seq.take 10 |> Seq.toArray

or in C#:

LogNormal estimation = LogNormal.Estimate(new [] {2.0, 1.5, 2.1, 1.2, 3.0, 2.4, 1.8});
double mean = estimation.Mean, variance = estimation.Variance;
double[] moreSamples = estimation.Samples().Take(10).ToArray();

Sampling a Probability Distribution

Each distribution provides methods to generate random numbers from that distribution. These random variate generators work by accessing the distribution's member RandomSource to provide uniform random numbers. By default, this member is an instance of System.Random but one can easily replace this with more sophisticated random number generators from MathNet.Numerics.Random (see Random Numbers for details).

// sample some random numbers from these distributions
// continuous distributions sample to floating-point numbers:
let continuous =
  [ yield normal.Sample()
    yield exponential.Sample()
    yield! gamma.Samples() |> Seq.take 10 ]

// discrete distributions on the other hand sample to integers:
let discrete =
  [ poisson.Sample()
    poisson.Sample()
    geometric.Sample() ]

Instead of creating a distribution object we can also sample directly with static functions. Note that no intermediate value caching is possible this way and parameters must be validated on each call.

// using the default number generator (SystemRandomSource.Default)
let w = Rayleigh.Sample(1.5)
let x = Hypergeometric.Sample(100, 20, 5)

// or by manually providing the uniform random number generator
let u = Normal.Sample(Random.system(), 2.0, 4.0)
let v = Laplace.Samples(Random.mersenneTwister(), 1.0, 3.0) |> Seq.take 100 |> List.ofSeq

If you need to sample not just one or two values but a large number of them, there are routines that either fill an existing array or return an enumerable. The variant that fills an array is generally the fastest. Routines to sample more than one value use the plural form Samples instead of Sample.

Let's sample 100'000 values from a laplace distribution with mean 1.0 and scale 2.0 in C#:

var samples = new double[100000];
Laplace.Samples(samples, 1.0, 2.0);

Let's do some random walks in F# (TODO: Graph):

Seq.scan (+) 0.0 (Normal.Samples(0.0, 1.0)) |> Seq.take 10 |> Seq.toArray
Seq.scan (+) 0.0 (Cauchy.Samples(0.0, 1.0)) |> Seq.take 10 |> Seq.toArray

Distribution Functions and Properties

Distributions can not just be used to generate non-uniform random samples. Once parametrized they can compute a variety of distribution properties or evaluate distribution functions. Because it is often numerically more stable and faster to compute and work with such quantities in the logarithmic domain, some of them are also available with the Ln-suffix.

// distribution properties of the gamma we've configured above
let gammaStats =
  ( gamma.Mean,
    gamma.Variance,
    gamma.StdDev,
    gamma.Entropy,
    gamma.Skewness,
    gamma.Mode )

// probability distribution functions of the normal we've configured above.
let nd = normal.Density(4.0)  (* PDF *)
let ndLn = normal.DensityLn(4.0)  (* ln(PDF) *)
let nc = normal.CumulativeDistribution(4.0)  (* CDF *)
let nic = normal.InverseCumulativeDistribution(0.7)  (* CDF^(-1) *)

// Distribution functions can also be evaluated without creating an object,
// but then you have to pass in the distribution parameters as first arguments:
let nd2 = Normal.PDF(3.0, sqrt 1.5, 4.0)
let ndLn2 = Normal.PDFLn(3.0, sqrt 1.5, 4.0)
let nc2 = Normal.CDF(3.0, sqrt 1.5, 4.0)
let nic2 = Normal.InvCDF(3.0, sqrt 1.5, 0.7)

Composing Distributions

Specifically for F# there is also a Sample module that allows a somewhat more functional view on distribution sampling functions by having the random source passed in as last argument. This way they can be composed and transformed arbitrarily if curried:

/// Transform a sample from a distribution
let s1 rng = tanh (Sample.normal 2.0 0.5 rng)

/// But we really want to transform the function, not the resulting sample:
let s1f rng = Sample.map tanh (Sample.normal 2.0 0.5) rng

/// Exactly the same also works with functions generating full sequences
let s1s rng = Sample.mapSeq tanh (Sample.normalSeq 2.0 0.5) rng

/// Now with multiple distributions, e.g. their product:
let s2 rng = (Sample.normal 2.0 1.5 rng) * (Sample.cauchy 2.0 0.5 rng)
let s2f rng = Sample.map2 (*) (Sample.normal 2.0 1.5) (Sample.cauchy 2.0 0.5) rng
let s2s rng = Sample.mapSeq2 (*) (Sample.normalSeq 2.0 1.5) (Sample.cauchySeq 2.0 0.5) rng

// Taking some samples from the composed function
Seq.take 10 (s2s (Random.system())) |> Seq.toArray