V. 2.6.

Package umontreal.iro.lecuyer.hups

Monte Carlo and quasi-Monte Carlo


Interface Summary
PointSetIterator Objects of classes that implement this interface are iterators that permit one to enumerate (or observe) the successive points of a point set and the successive coordinates of these points.
PointSetRandomization This interface is used to randomize a PointSet.

Class Summary
AntitheticPointSet This container class provides antithetic points.
BakerTransformedPointSet This container class embodies a point set to which a Baker transformation is applied.
CachedPointSet This container class caches a point set by precomputing and storing its points locally in an array.
ContainerPointSet This acts as a generic base class for all container classes that contain a point set and apply some kind of transformation to the coordinates to define a new point set.
CycleBasedPointSet This abstract class provides the basic structures for storing and manipulating a highly uniform point set defined by a set of cycles.
CycleBasedPointSetBase2 Similar to CycleBasedPointSet, except that the successive values in the cycles are stored as integers in the range {0,..., 2k -1}, where 1 <= k <= 31.
DigitalNet This class provides the basic structures for storing and manipulating linear digital nets in base b, for an arbitrary base b >= 2.
DigitalNetBase2 A special case of DigitalNet for the base b = 2.
DigitalNetBase2FromFile This class allows us to read the parameters defining a digital net in base 2 either from a file, or from a URL address on the World Wide Web.
DigitalNetFromFile This class allows us to read the parameters defining a digital net either from a file, or from a URL address on the World Wide Web.
DigitalSequence This abstract class describes methods specific to digital sequences.
DigitalSequenceBase2 This abstract class describes methods specific to digital sequences in base 2.
EmptyRandomization This class implements an empty PointSetRandomization.
F2wCycleBasedLFSR This class creates a point set based upon a linear feedback shift register sequence.
F2wCycleBasedPolyLCG This class creates a point set based upon a linear congruential sequence in the finite field F2w[z]/P(z).
F2wNetLFSR This class implements a digital net in base 2 starting from a linear feedback shift register generator.
F2wNetPolyLCG This class implements a digital net in base 2 starting from a polynomial LCG in F2w[z]/P(z).
F2wStructure This class implements methods and fields needed by the classes F2wNetLFSR, F2wNetPolyLCG, F2wCycleBasedLFSR and F2wCycleBasedPolyLCG.
FaureSequence This class implements digital nets or digital sequences formed by the first n = bk points of the Faure sequence in base b.
HaltonSequence This class implements the sequence of Halton, which is essentially a modification of Hammersley nets for producing an infinite sequence of points having low discrepancy.
HammersleyPointSet This class implements Hammersley point sets, which are defined as follows.
KorobovLattice This class implements Korobov lattices, which represents the same point sets as in class LCGPointSet, but implemented differently.
KorobovLatticeSequence This class implements Korobov lattice sequences, defined as follows.
LCGPointSet Implements a recurrence-based point set defined via a linear congruential recurrence of the form xi = axi-1mod n and ui = xi/n.
LMScrambleShift This class implements a PointSetRandomization that performs a left matrix scrambling and adds a random digital shift.
NiedSequenceBase2 This class implements digital sequences constructed from the Niederreiter sequence in base 2.
NiedXingSequenceBase2 This class implements digital sequences based on the Niederreiter-Xing sequence in base 2.
PaddedPointSet This container class realizes padded point sets, constructed by taking some coordinates from a point set P1, other coordinates from a point set P2, and so on.
PointSet This abstract class defines the basic methods for accessing and manipulating point sets.
RadicalInverse This class implements basic methods for working with radical inverses of integers in an arbitrary basis b.
RandomShift This class implements a PointSetRandomization.
RandomStart This class implements a PointSetRandomization that randomizes a sequence with a random starting point.
RandShiftedPointSet This container class embodies a point set to which a random shift modulo 1 is applied (i.e., a single uniform random point is added to all points, modulo 1, to randomize the inner point set).
Rank1Lattice This class implements point sets specified by integration lattices of rank 1.
RQMCPointSet This class is used for randomized quasi-Monte Carlo (RQMC) simulations.
SMScrambleShift This class implements a PointSetRandomization that performs a striped matrix scrambling and adds a random digital shift.
SobolSequence This class implements digital nets or digital sequences in base 2 formed by the first n = 2k points of a Sobol' sequence.
SubsetOfPointSet This container class permits one to select a subset of a point set.

Package umontreal.iro.lecuyer.hups Description

Monte Carlo and quasi-Monte Carlo

This package provides classes implementing highly uniform point sets (HUPS) over the s-dimensional unit hypercube [0, 1)s, and tools for their randomization. The terminology low-discrepancy sequence (LDS) is often used for infinite sequences of points such that the discrepancy between the distribution of the first n points of the sequence and the uniform distribution converges to zero at a certain rate when n -> ∞. HUPS and LDS are used for quasi-Monte Carlo integration, as we now briefly explain. See, e.g., for further details.

Suppose we want to estimate the integral of a function f defined over the s-dimensional unit hypercube,

μ = ∫[0, 1)s f ($\displaystyle \mbox{\boldmath$u$}$)d[tex2html_wrap_indisplay1030].

Practically any mathematical expectation that can be estimated by simulation can be written in this way, usually for a very complicated f and sometimes for s = ∞. Indeed, the source of randomness of stochastic simulations is usually a stream of real numbers u = (u0, u1, u2,...) whose purpose is to imitate i.i.d. U(0, 1) random variables. These real numbers are transformed in complicated ways to produce the estimator. Thus, the dimension s of the integral represents the number of calls to the uniform random number generator if that number is deterministic. If it is random and unbounded, we take s = ∞. In the latter case, however, we can assume that the actual number of calls is finite with probability one (otherwise the simulation may never end).

We consider an estimator of μ of the form

Qn = $\displaystyle {\frac{{1}}{{n}}}$i=0n-1f ([tex2html_wrap_indisplay1043]i),

which is the average of f over the point set Pn = {u0,...,un-1}⊂[0, 1)s.

With the Monte Carlo (MC) method, the ui's are i.i.d. random vectors uniformly distributed over [0, 1)s. Then, Qn is an unbiased estimator of μ with variance σ2/n, where

σ2 = ∫[0, 1)sf2([tex2html_wrap_indisplay1061])d[tex2html_wrap_indisplay1062] - μ2,

and it obeys a central-limit theorem if σ2 < ∞.

Quasi-Monte Carlo (QMC) methods use point sets Pn that are more evenly distributed over the unit hypercube than typical random points. We call them highly uniform point sets (HUPS). The aim is to reduce the size of the integration error Qn - μ. Two important classes of methods for constructing such point sets are digital nets and integration lattices. Both are implemented in this package, in various flavors.

Elementary constructions

To give an idea of how HUPS and LDS can be constructed, we start with a simple one-dimensional example. If s = 1 and n is fixed, very simple highly uniform constructions are the point sets Pn = {0, 1/n, ...,(n - 1)/n} and the shifted version P'n = {1/(2n), 3/(2n), ...,(2n - 1)/(2n)}.

In s > 1 dimensions, the simplest extensions would be as follows. Let n = ds for some integer d and define Pn as the Cartesian product of s copies of the one-dimensional sets Pd; that is, Pn = {(u0,..., us-1) : uj∈{0, 1/d, ...,(d - 1)/d} for each j}, and similarly for P'n. The point sets thus obtained are regular rectangular grids. Unfortunately, this approach breaks down rapidly when s gets large, because n must increase exponentially fast with s for fixed d. Another important drawback is that when Pn is projected over lower-dimensional subspaces, several points are projected onto each other and become redundant.

A better idea is to construct a point set Pn in s dimensions such that each one-dimensional projection of Pn is the set of values {0, 1/n, ...,(n - 1)/n}. Of course, these values should not be visited in the same order for all coordinates, because otherwise all the points would lie on the diagonal line going from (0,..., 0) to (1,..., 1). In other words, for each coordinate j, 0 <= j < s, we must define a different permutation of the integers {0,..., n - 1} and visit the values {0, 1/n, ...,(n - 1)/n} in the order determined by that permutation. The trick is to select those permutations in a way that Pn itself is highly uniform over [0, 1)s in a well-defined sense (to be determined). This is what most construction methods attempt to achieve. Before looking at concrete ways of defining such permutations, we introduce a related issue: what to do if n is not fixed.

For s = 1, a simple way of filling up the unit interval [0, 1) uniformly is via the low-discrepancy sequence 0, 1/2, 1/4, 3/4, 1/8, 5/8, 3/8, 7/8, 1/16, 9/16, ..., called the van der Corput sequence in base 2. More generally, select an integer b >= 2, called the base. The radical inverse function in base b, ψb : N -> [0, 1), is defined as follows. If i is a k-digit integer in base b with digital b-ary expansion

i = a0 + a1b + ... + ak-1bk-1,


ψb(i) = a0b-1 + a1b-2 + ... + ak-1b-k.

For a given b, ψb(0), ψb(1), ψb(2),... is called the van der Corput sequence in base b. This sequence fills up the unit interval [0, 1) quite uniformly. For example, for b = 2 we obtain the sequence mentioned above and for b = 3 we obtain 0, 1/3, 2/3, 1/9, 4/9, 7/9, 2/9, 5/9, 8/9, 1/27, 10/27, 19/27, .... Moreover, for two relatively prime bases b1 and b2, the two sequences have no value in common except 0.

For s > 1, one could either take different (relatively prime) bases for the different coordinates, or take the same basis b but permute the successive values using a different permutation for each coordinate. These permutations are usually selected in a way that for every integer k, the first bk values that are enumerated remain the same (they are the values of ψb(i) for i = 0,..., bk - 1), but they are enumerated in a different order. Several digital net constructions (to be defined later) fit this framework.

If we decide to take different bases, the most natural choice is to take the jth smallest prime, bj, as a base for coordinate j - 1; that is, base 2 for coordinate 0, base 3 for coordinate 1, base 5 for coordinate 2, and so on. The infinite sequence thus defined, where point i is

[tex2html_wrap_indisplay1128]i = (ψb1(i), ψb2(i),..., ψbs(i))

for i >=  0, was proposed in and is called the Halton sequence. One drawback of this sequence is that for large s, the base bs becomes quite large.

In the case where n is fixed, we can always take i/n as the first coordinate of point i. In particular, the Hammersley point set with n points in s dimensions contains the points

[tex2html_wrap_indisplay1138]i = (i/n, ψb1(i), ψb2(i),..., ψbs-1(i)),

for i = 0,..., n - 1. Historically, Halton sequences were defined as extensions of Hammersley point sets.

Digital nets

Digital nets and sequences are an important class of HUPS and LDS constructions. Most concrete implementations, e.g., those proposed by Sobol', Faure, Niederreiter, and Niederreiter and Xing, are linear digital nets and sequences, defined as follows (see also).

Let b >= 2 be an arbitrary integer (usually a prime number), called the base. A net that contains n = bk points in s dimensions is defined via s generator matrices C0,...,Cs-1, which are (in theory) ∞×k matrices whose elements are in Zb = {0,..., b - 1}. The matrix Cj is used for coordinate j of all the points, for j >=  0. To define the ith point ui, for i = 0,..., bk - 1, write the digital expansion of i in base b and multiply the vector of its digits by Cj to obtain the digits of the expansion of ui, j, the jth coordinate of ui. That is,

i = $\scriptstyle \ell$=0k-1ai,[tex2html_wrap_indisplay1174]b[tex2html_wrap_indisplay1175],  
$\displaystyle \pmatrix$ui, j, 1$\displaystyle \cr$ui, j, 2[tex2html_wrap_indisplay1179] $\displaystyle \vdots$ [tex2html_wrap_indisplay1181] = Cj[tex2html_wrap_indisplay1184]ai, 0[tex2html_wrap_indisplay1185]ai, 1[tex2html_wrap_indisplay1186] [tex2html_wrap_indisplay1187] [tex2html_wrap_indisplay1188]ai, k-1[tex2html_wrap_indisplay1189],  
ui, j = [tex2html_wrap_indisplay1193]=1ui, j,[tex2html_wrap_indisplay1194]b-[tex2html_wrap_indisplay1195],  
[tex2html_wrap_indisplay1197]i = (ui, 0,..., ui, s-1).  

In practice, the expansion in is truncated to the first r digits for some positive integer r, so each matrix Cj is actually truncated to a r×k matrix. Typically r is equal to k, or is slightly larger, or is selected so that br is near 231.

Usually, the first k lines of each Cj form a nonsingular k×k matrix. Then, the n output values for coordinate j, u0, j,..., un-1, j, when truncated to their first k fractional digits in base b, are a permutation of the numbers 0, 1/n,...,(n - 1)/n. Different coordinates simply use different permutations, implemented via the matrices Cj.

When the first k lines of Cj form the identity and the other lines are zero, the first n output values are the first n elements of the van der Corput sequence in base b. If we reverse the order of the columns of that matrix Cj (i.e., column c will contain a one in line k - c + 1 and zeros elsewhere, for 0 <= c < k), we obtain the output values 0, 1/n,...,(n - 1)/n in that order. With a slight abuse of language, we shall call this first matrix (with the identity followed by lines of zeros) the identity and the second one (with the columns in reverse order) the reflected identity. It is customary to take C0 as the identity for digital sequences, and often for digital nets as well. But for digital nets (where n is fixed in advance), one can take C0 as the reflected identity instead, then C1 as the identity, and so on. That is, the matrix Cj for the digital net is taken as the matrix Cj-1 of the digital sequence. Our package often gives the choice.

For digital sequences, the matrices Cj actually have an infinite number of columns, although only the first k columns are needed to generate the first bk points. So in practice, we never need to store more than a finite number of columns at a time. Whenever we find that we need more than bk points for the current value of k, we can simply increase k and add the corresponding columns to the matrices Cj.

The classes DigitalNet and DigitalSequence implement generic digital nets and sequences. Specific instances are constructed in subclasses of these two classes.

Lattice Rules

An integration lattice is a discrete (but infinite) subset of Rs of the form

Ls = {v = ∑j=1shjvj such that each hjZ},

where v1,...,vsRs are linearly independent over R and ZsLs. This last condition means that Ls must contain all integer vectors, and this implies that Ls is periodic with period 1 along each of the s coordinates. The approximation of μ by Qn with the point set Pn = Ls∩[0, 1)s is called a lattice rule. The value of n is the number of points of the lattice that are in the unit hypercube [0, 1)s.

Let V be the matrix whose rows are the basis vectors v1, ... ,vs and V-1 its inverse. One has ZsLs if and only if all entries of V-1 are integer. When this holds, n = det(V-1) and all entries of V are multiples of 1/n.

The rank of the lattice is the smallest r such that one can find a basis of the form v1,...,vr,er+1, ... ,es, where ej is the jth unit vector in s dimensions. In particular, a lattice rule of rank 1 has a basis of the form v1 = (a1,..., as)/n and vj = ej for j > 1, where ajZn for each j. It is a Korobov rule if v1 has the special form v1 = (1,  a,  a2mod n,  ...,  as-1mod n)/n for some aZn. The point set Pn of a Korobov lattice rule can also be written as Pn = {(x1,..., xs)/n such that x1Zn and xj = axj-1mod n for all j > 1}. This is the set of all vectors of successive values produced by a linear congruential generator (LCG) with modulus n and multiplier a, from all possible initial states, including 0. In this case, the points are easy to enumerate by using the recurrence.

Cycle-based point sets

Certain types of point sets are defined pretty much like random number generators: choose a finite state space S, a transition function f : S -> S, an output function g : S -> [0, 1), and define

Pn = {[tex2html_wrap_indisplay1288] = (u0, u1,...) : s0∈S, sj = f (sj-1), and uj = g(sj) for all j}.

This is the set of all vectors of successive output values produced by the recurrence defined by f and the output function g, from all possible initial states. The value of n is the cardinality of S and the dimension s is infinite. We could also have n = ∞ (an infinite sequence) if S is infinite but denumerable and ordered (so we know in which order to enumerate the points).

Let us assume that n is finite and that for each s0∈S, the recurrence sj = f (sj-1) is purely periodic, i.e., there is always an integer j such that sj = s0. The smallest such j, called the period length, depends in general on s0. Thus, the state space S is partitioned into a finite number of cycles. The successive coordinates of any point uPn are periodic with period length equal to the length of the cycle that contains s0 (and the following sj's).

One way of implementing such a point set while avoiding to recompute f and g each time a coordinate is needed is to store explicitly all the cycles of the recurrence, in the form of a list of cycles. We can store either the successive uj's directly, or the successive sj's, over each cycle. The class CycleBasedPointSet provides the framework for doing that.

For example, a Korobov lattice point set is defined via the recurrence xj = axj-1mod n and output function uj = xj/n. If n is prime and a is a primitive element modulo n, then there are two cycles: one of period 1 that contains only 0, and the other of period n - 1. For more general n and a, there will be more cycles. The class LCGPointSet constructs this type of point set and stores explicitly the successive values of uj over the different cycles.

There are cases where n is a power of two, say n = 2k, and where the state sj is represented as a k-bit string. In that context, it is often more convenient to store the successive sj's instead of the successive uj's, over the set of cycles (e.g., if a random digital shift in base 2 is to be applied to randomize the points, it can be performed by applying a bitwise xor directly to sj). When generating the coordinates, the sj's can be interpreted as 2k-bit integers and multiplied by 2-k to produce the output. This is supported by the class CycleBasedPointSetBase2. Special instances of this class are usually based on linear recurrences modulo 2 and they include the Korobov-type polynomial lattice rules.

Randomized quasi-Monte Carlo

In their original versions, these HUPS are deterministic, and the corresponding QMC methods give a deterministic integration error that is difficult to estimate. In randomized QMC methods, Pn is randomized, preferably in a way that it retains its high uniformity over [0, 1)s when taken as a set, while each of its points has the uniform distribution over [0, 1)s when taken individually. Then, Qn becomes an unbiased estimator of μ, hopefully with smaller variance than the standard MC estimator. To estimate the variance and compute a confidence interval on μ, one can apply m independent randomizations to the same Pn, and compute bar(X)m and Sm, x2, the sample mean and sample variance of the m corresponding (independent) copies of Qn. Then, E[bar(X)m] = μ and E[Sm, x2] = Var[Qn] = mVar[bar(X)m].

Two examples of such randomizations are the random shift modulo 1, proposed in and implemented in class RandShiftedPointSet, and the random digital shift in base b, described and implemented in class DigitalNet. These randomizations are also incorporated directly in certain types of point sets such as CycleBasedPointSet, CycleBasedPointSetBase2, etc.

In the random shift modulo 1, we generate a single point u uniformly over [0, 1)s and add it to each point of Pn, coordinate-wise, modulo 1. Since all points of Pn are shifted by the same amount, the set retains most of its structure and uniformity.

For the random digital shift in base b, we generate again a single u = (u1,..., us) uniformly over [0, 1)s, write the digital expansion in base b of each of its coordinates, say uj = ∑[tex2html_wrap_inline1364]=1dj,[tex2html_wrap_inline1365]b-[tex2html_wrap_inline1366], then add dj,[tex2html_wrap_inline1368] modulo b to the $ \ell$th digit of the digital expansion in base b of the jth coordinate of each point uiPn. For b = 2, the digit-wise addition modulo b becomes a bitwise exclusive-or, which is fast to perform on a computer.

An interesting property of the digital shift in base b is that if the hypercube [0, 1)s is partitioned into bq1+ ... +qs rectangular boxes of the same size by partitioning the jth axis into bqj equal parts for each j, for some integers qj >=  0 (such a partition is called a q-equidissection in base b of the unit hypercube, where q = (q1,..., qs)), then the number of boxes that contain m points, for each integer m, is unchanged by the randomization. In particular, if each box contains the same number of points of Pn before the randomization, then it also does after the randomization. In this case, we say that Pn is q-equidistributed in base b. Several other randomization methods exist and most are adapted to special types of point sets; see, e.g., DigitalNet and.

Point set implementations and enumeration tools

Let ui = (ui, 0, ui, 1,..., ui, s-1) be the elements of the point set Pn, for i = 0,..., n - 1. Both the number of points n and the dimension s can be finite or infinite. The point set can be viewed as a two-dimensional array whose element (i, j) contains ui, j, the coordinate j of point i. In the implementations of typical point sets, the values ui, j are not stored explicitly in a two-dimensional array, but pertinent information is organized so that the points and their coordinates can be generated efficiently. The base class for point sets is the abstract class PointSet.

To enumerate the successive points or the successive coordinates of a given point, we use point set iterators, which resemble the iterators defined in Java collections, except that they loop over bi-dimensional sets. Their general behavior is defined in the interface PointSetIterator. Several independent iterators can coexist at any given time for the same point set. Each one maintains a current point index and a current coordinate index, which are incremented by one when the iterator advances to the next point or the next coordinate. Both are initialized to 0. Each subclass of PointSet has its own implementation of PointSetIterator and has a method iterator that creates and returns a new point set iterator of the correct type.

An important feature of the PointSetIterator interface is that it extends the RandomStream interface. This means that any point set iterator can be used in place of a random stream that is supposed to generate i.i.d. U(0, 1) random variables, anywhere in a simulation program. It then becomes very easy to replace the (pseudo)random numbers by the coordinates ui, j of a randomized HUPS without changing the internal code of the simulation program.

Transformed point sets and containers

HUPS are often transformed either deterministically or randomly. Deterministic transformations can be applied to improve the uniformity, or to eliminate some points or coordinates (i.e., selecting subsets), or to concatenate point sets (padding), or to take an antithetic version of a point set, etc. Random transformations are used for randomized QMC. They are also useful when the average of a uniformity measure of interest over the outcomes of a certain type of randomization is much better than the worst case and may be better than the uniformity measure of the original point set. When a point set is transformed, we often want to keep the original as well, and we may want to apply different types of transformations or different independent randomizations to the same point set.

This can be achieved via container point sets, which are defined in terms of another point set to which they keep a reference and apply certain transformations. ContainerPointSet is the base class for such containers. One example is RandShiftedPointSet, which applies a random shift modulo 1 to the point set that it contains. Of course, the contained point set can be a container itself and this can be done recursively, but too many levels of recursiveness may impair the performance (speed).


To be done...

V. 2.6.

To submit a bug or ask questions, send an e-mail to Pierre L'Ecuyer.