Math.NET Numerics | Math.NET Project | GitHub


Probability Distributions

Math.NET Numerics provides a wide range of probability distributions. Given the distribution parameters they can be used to investigate their statistical properties or to sample non-uniform random numbers.

All the distributions implement a common set of operations such as evaluating the density (PDF) and the cumulative distribution (CDF) at a given point, or to compute the mean, standard deviation and other properties. Because it is often numerically more stable and faster to compute such statistical quantities in the logarithmic domain, we also provide a selection of them in the log domain with the "Ln" suffix, e.g. DensityLn for the logarithmic density.

using MathNet.Numerics.Distributions;
using MathNet.Numerics.Random;

// create a parametrized distribution instance
var gamma = new Gamma(2.0, 1.5);

// distribution properties
double mean = gamma.Mean;
double variance = gamma.Variance;
double entropy = gamma.Entropy;

// distribution functions
double a = gamma.Density(2.3); // PDF
double b = gamma.DensityLn(2.3); // ln(PDF)
double c = gamma.CumulativeDistribution(0.7); // CDF

// non-uniform number sampling
double randomSample = gamma.Sample();

Both probability functions and sampling are also available as static functions for simpler usage scenarios:

// distribution parameters must be passed as arguments
double a2 = Gamma.PDF(2.0, 1.5, 2.3);
double randomSample2 = Gamma.Sample(2.0, 1.5);

Continuous Distributions

Discrete Distributions

Multivariate Distributions

Distribution Parameters

There are many ways to parametrize a distribution in the literature. When using the default constructor, read carefully which parameters it requires. For distributions where multiple ways are common there are also static methods, so you can use the one that fits best. For example, a normal distribution is usually parametrized with mean and standard deviation, but if you'd rather use mean and precision:

var normal = Normal.WithMeanPrecision(0.0, 0.5);

Since probability distributions can also be sampled to generate random numbers with the configured distribution, all constructors optionally accept a random generator as last argument.

var gamma2 = new Gamma(2.0, 1.5, new MersenneTwister());

// the random generator can also be replaced on an existing instance
gamma2.RandomSource = new Mrg32k3a();

A few more examples, this time in F#:

// some probability distributions
let normal = Normal.WithMeanVariance(3.0, 1.5, a)
let exponential = Exponential(2.4)
let gamma = Gamma(2.0, 1.5, Random.crypto())
let cauchy = Cauchy(0.0, 1.0, Random.mrg32k3aWith 10 false)
let poisson = Poisson(3.0)
let geometric = Geometric(0.8, Random.system())

Some of the distributions also have routines for maximum-likelihood parameter estimation from a set of samples:

let estimation = LogNormal.Estimate([| 2.0; 1.5; 2.1; 1.2; 3.0; 2.4; 1.8 |])
let mean, variance = estimation.Mean, estimation.Variance
let moreSamples = estimation.Samples() |> Seq.take 10 |> Seq.toArray

or in C#:

LogNormal estimation = LogNormal.Estimate(new [] {2.0, 1.5, 2.1, 1.2, 3.0, 2.4, 1.8});
double mean = estimation.Mean, variance = estimation.Variance;
double[] moreSamples = estimation.Samples().Take(10).ToArray();

Sampling a Probability Distribution

Each distribution provides methods to generate random numbers from that distribution. These random variate generators work by accessing the distribution's member RandomSource to provide uniform random numbers. By default, this member is an instance of System.Random but one can easily replace this with more sophisticated random number generators from MathNet.Numerics.Random (see Random Numbers for details).

// sample some random numbers from these distributions
// continuous distributions sample to floating-point numbers:
let continuous =
  [ yield normal.Sample()
    yield exponential.Sample()
    yield! gamma.Samples() |> Seq.take 10 ]

// discrete distributions on the other hand sample to integers:
let discrete =
  [ poisson.Sample()
    poisson.Sample()
    geometric.Sample() ]

Instead of creating a distribution object we can also sample directly with static functions. Note that no intermediate value caching is possible this way and parameters must be validated on each call.

// using the default number generator (SystemRandomSource.Default)
let w = Rayleigh.Sample(1.5)
let x = Hypergeometric.Sample(100, 20, 5)

// or by manually providing the uniform random number generator
let u = Normal.Sample(Random.system(), 2.0, 4.0)
let v = Laplace.Samples(Random.mersenneTwister(), 1.0, 3.0) |> Seq.take 100 |> List.ofSeq

If you need to sample not just one or two values but a large number of them, there are routines that either fill an existing array or return an enumerable. The variant that fills an array is generally the fastest. Routines to sample more than one value use the plural form Samples instead of Sample.

Let's sample 100'000 values from a laplace distribution with mean 1.0 and scale 2.0 in C#:

var samples = new double[100000];
Laplace.Samples(samples, 1.0, 2.0);

Let's do some random walks in F# (TODO: Graph):

Seq.scan (+) 0.0 (Normal.Samples(0.0, 1.0)) |> Seq.take 10 |> Seq.toArray
Seq.scan (+) 0.0 (Cauchy.Samples(0.0, 1.0)) |> Seq.take 10 |> Seq.toArray

Distribution Functions and Properties

Distributions can not just be used to generate non-uniform random samples. Once parametrized they can compute a variety of distribution properties or evaluate distribution functions. Because it is often numerically more stable and faster to compute and work with such quantities in the logarithmic domain, some of them are also available with the Ln-suffix.

// distribution properties of the gamma we've configured above
let gammaStats =
  ( gamma.Mean,
    gamma.Variance,
    gamma.StdDev,
    gamma.Entropy,
    gamma.Skewness,
    gamma.Mode )

// probability distribution functions of the normal we've configured above.
let nd = normal.Density(4.0)  (* PDF *)
let ndLn = normal.DensityLn(4.0)  (* ln(PDF) *)
let nc = normal.CumulativeDistribution(4.0)  (* CDF *)
let nic = normal.InverseCumulativeDistribution(0.7)  (* CDF^(-1) *)

// Distribution functions can also be evaluated without creating an object,
// but then you have to pass in the distribution parameters as first arguments:
let nd2 = Normal.PDF(3.0, sqrt 1.5, 4.0)
let ndLn2 = Normal.PDFLn(3.0, sqrt 1.5, 4.0)
let nc2 = Normal.CDF(3.0, sqrt 1.5, 4.0)
let nic2 = Normal.InvCDF(3.0, sqrt 1.5, 0.7)

Composing Distributions

Specifically for F# there is also a Sample module that allows a somewhat more functional view on distribution sampling functions by having the random source passed in as last argument. This way they can be composed and transformed arbitrarily if curried:

/// Transform a sample from a distribution
let s1 rng = tanh (Sample.normal 2.0 0.5 rng)

/// But we really want to transform the function, not the resulting sample:
let s1f rng = Sample.map tanh (Sample.normal 2.0 0.5) rng

/// Exactly the same also works with functions generating full sequences
let s1s rng = Sample.mapSeq tanh (Sample.normalSeq 2.0 0.5) rng

/// Now with multiple distributions, e.g. their product:
let s2 rng = (Sample.normal 2.0 1.5 rng) * (Sample.cauchy 2.0 0.5 rng)
let s2f rng = Sample.map2 (*) (Sample.normal 2.0 1.5) (Sample.cauchy 2.0 0.5) rng
let s2s rng = Sample.mapSeq2 (*) (Sample.normalSeq 2.0 1.5) (Sample.cauchySeq 2.0 0.5) rng

// Taking some samples from the composed function
Seq.take 10 (s2s (Random.system())) |> Seq.toArray
val normal : obj
val exponential : obj
val gamma : obj
val cauchy : obj
val poisson : obj
val geometric : obj
val estimation : obj
val mean : obj
val variance : obj
val moreSamples : obj []
module Seq from Microsoft.FSharp.Collections
<summary>Contains operations for working with values of type <see cref="T:Microsoft.FSharp.Collections.seq`1" />.</summary>
val take : count:int -> source:seq<'T> -> seq<'T>
<summary>Returns the first N elements of the sequence.</summary>
<remarks>Throws <c>InvalidOperationException</c> if the count exceeds the number of elements in the sequence. <c>Seq.truncate</c> returns as many items as the sequence contains instead of throwing an exception.</remarks>
<param name="count">The number of items to take.</param>
<param name="source">The input sequence.</param>
<returns>The result sequence.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
<exception cref="T:System.ArgumentException">Thrown when the input sequence is empty.</exception>
<exception cref="T:System.InvalidOperationException">Thrown when count exceeds the number of elements in the sequence.</exception>
val toArray : source:seq<'T> -> 'T []
<summary>Builds an array from the given collection.</summary>
<param name="source">The input sequence.</param>
<returns>The result array.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
val continuous : obj list
val discrete : obj list
val w : obj
val x : obj
val u : obj
val v : obj list
Multiple items
module List from Microsoft.FSharp.Collections
<summary>Contains operations for working with values of type <see cref="T:Microsoft.FSharp.Collections.list`1" />.</summary>
<namespacedoc><summary>Operations for collections such as lists, arrays, sets, maps and sequences. See also <a href="https://docs.microsoft.com/dotnet/fsharp/language-reference/fsharp-collection-types">F# Collection Types</a> in the F# Language Guide. </summary></namespacedoc>


--------------------
type List<'T> = | ( [] ) | ( :: ) of Head: 'T * Tail: 'T list interface IReadOnlyList<'T> interface IReadOnlyCollection<'T> interface IEnumerable interface IEnumerable<'T> member GetReverseIndex : rank:int * offset:int -> int member GetSlice : startIndex:int option * endIndex:int option -> 'T list static member Cons : head:'T * tail:'T list -> 'T list member Head : 'T member IsEmpty : bool member Item : index:int -> 'T with get ...
<summary>The type of immutable singly-linked lists.</summary>
<remarks>Use the constructors <c>[]</c> and <c>::</c> (infix) to create values of this type, or the notation <c>[1;2;3]</c>. Use the values in the <c>List</c> module to manipulate values of this type, or pattern match against the values directly. </remarks>
<exclude />
val ofSeq : source:seq<'T> -> 'T list
<summary>Builds a new list from the given enumerable object.</summary>
<param name="source">The input sequence.</param>
<returns>The list of elements from the sequence.</returns>
val scan : folder:('State -> 'T -> 'State) -> state:'State -> source:seq<'T> -> seq<'State>
<summary>Like fold, but computes on-demand and returns the sequence of intermediary and final results.</summary>
<param name="folder">A function that updates the state with each element from the sequence.</param>
<param name="state">The initial state.</param>
<param name="source">The input sequence.</param>
<returns>The resulting sequence of computed states.</returns>
<exception cref="T:System.ArgumentNullException">Thrown when the input sequence is null.</exception>
val gammaStats : obj * obj * obj * obj * obj * obj
val nd : obj
val ndLn : obj
val nc : obj
val nic : obj
val nd2 : obj
val sqrt : value:'T -> 'U (requires member Sqrt)
<summary>Square root of the given number</summary>
<param name="value">The input value.</param>
<returns>The square root of the input.</returns>
val ndLn2 : obj
val nc2 : obj
val nic2 : obj
val s1 : rng:'a -> float
 Transform a sample from a distribution
val rng : 'a
val tanh : value:'T -> 'T (requires member Tanh)
<summary>Hyperbolic tangent of the given number</summary>
<param name="value">The input value.</param>
<returns>The hyperbolic tangent of the input.</returns>
val s1f : rng:'a -> 'b
 But we really want to transform the function, not the resulting sample:
val s1s : rng:'a -> 'b
 Exactly the same also works with functions generating full sequences
val s2 : rng:'a -> obj
 Now with multiple distributions, e.g. their product:
val s2f : rng:'a -> 'b
val s2s : rng:'a -> 'b