Construction of measures using random variables

A common problem in measure theory is that we want to build a measure whose values are already morally clear on some small family of sets. We do not usually discover a measure by assigning values to arbitrary sets one by one. Instead, we begin with a simple class of sets where the desired answer is obvious, and then we need a mechanism that extends this partial information to a genuine countably additive measure on a sigma algebra. The construction of Lebesgue measure is the basic example. We know that the interval $[a,b]$ should have measure $b-a$ , and that finite disjoint unions of intervals should have measure equal to the sum of their lengths. We also expect translation-invariance: moving a set should not change its length. These requirements are natural and geometric, but they only describe the measure on very simple sets. The actual problem is to construct a measure on a much larger class of subsets of $\mathbb R$ .

The usual construction proceeds through outer measure. Given an arbitrary set $A\subseteq\mathbb R$ , one covers it by countably many intervals and takes the infimum of the total lengths:

$\displaystyle m^*(A)=\inf\left \{\sum_{j=1}^{\infty}|I_j|: A\subseteq\bigcup_{j=1}^{\infty}I_j\right\}.$

This outer measure is defined for every subset of the real line, but it is not countably additive on every subset. The next step is to identify the good sets, namely those sets $E$ which split every set $A$ cleanly:

$\displaystyle m^*(A)=m^*(A\cap E)+m^*(A\setminus E).$

Carathéodory’s theorem says that these good sets form a sigma algebra, and that the outer measure restricted to this sigma algebra is a genuine measure. This gives Lebesgue measure. The Borel sets are contained in this sigma algebra, and in fact the Lebesgue measurable sets form a larger class. So the construction begins with the simple geometric idea of length, passes through an outer measure, and then extracts the sets on which countable additivity behaves correctly.

This is one way of constructing measures: start with values on elementary sets and extend. But probability theory uses another method just as often. Once we have one measure space, we can create new measures by mapping it into other spaces. Suppose $(\Omega,\mathcal F,\mu)$ is a measure space, $(S,\mathcal G)$ is a measurable space, and $X:\Omega\to S$ is measurable. Then $X$ transports the measure $\mu$ from $\Omega$ to $S$ . The transported measure is defined by

$\displaystyle X_*\mu(A)=\mu(X^{-1}(A)),\qquad A\in\mathcal G.$

This formula is the pushforward construction. It says that to measure a set $A$ in the target space, we pull it back to the original space and measure the set of points that land inside $A$ . In probability language, if $\mu=\mathbb P$ , then the measurable map $X:\Omega\to S$ is a random variable, or more generally a random element of $S$ , and the measure $X_*\mathbb P$ is the law or distribution of $X$ .

This is a better definition of “random variable” than the informal phrase “a variable whose value is random.” A random variable is a measurable map out of a probability space. Its law is the measure obtained by pushing probability forward through that map. The original point $\omega\in\Omega$ may be thought of as the hidden source of randomness; the random variable $X(\omega)$ is the observable quantity. Often the observable quantity is what matters, not the underlying sample point. If two random variables have the same law, then for many purposes they are the same random object, even if they are defined on different probability spaces.

The integral form of the pushforward construction explains why the law is so useful. If $f\to\mathbb R$ is measurable, then

$\displaystyle \int_S f(s) d(X_*\mu)(s)=\int_\Omega f(X(\omega)) d\mu(\omega).$

In probability notation, this says $\mathbb E[f(X)]=\int f d\mu_X$ , where $\mu_X$ is the law of $X$ . Thus once the law of $X$ is known, all expectations of functions of $X$ can be computed without referring back to the original sample space. This is why distributions are not just bookkeeping devices. They are the measures that contain exactly the information needed to integrate observables depending on the random variable.

The simplest example is a biased coin. Take $[0,1]$ with Lebesgue measure, and define $X(\omega)=1$ if $0\le\omega\le p$ , and $X(\omega)=0$ if $p<\omega\le1$ . Then the preimage of ${1}$ has length $p$ , and the preimage of ${0}$ has length $1-p$ . Therefore the law of $X$ is $(1-p)\delta_0+p\delta_1$ . We have constructed a Bernoulli measure not by an extension theorem, but by transporting Lebesgue measure through a simple map. This small example is representative. Once Lebesgue measure on $[0,1]$ is available, many probability distributions can be obtained as pushforwards of it.

If $U$ is uniformly distributed on $[0,1]$ , then $-\log U$ has the exponential distribution with rate $1$ , since $\Pr(-\log U\le x)=1-e^{-x}$ for $x\ge0$ . More generally, if $F$ is a distribution function, then the generalized inverse construction $X=F^{-1}(U)$ , where $\displaystyle F^{-1}(u)=\inf\{x\in\mathbb R: F(x)\ge u\}.$ , gives a random variable with distribution function $F$ . In the continuous strictly increasing case, the verification is simply $\Pr(F^{-1}(U)\le x)=\Pr(U\le F(x))=F(x)$ . The general case requires a little care because of jumps and flat parts, but the same idea remains: the unit interval is being rearranged so that the amount of mass placed to the left of $x$ is exactly $F(x)$ .

This suggests a useful principle: for many purposes, $[0,1]$ with Lebesgue measure is a universal source of randomness. In standard Borel spaces, every probability law can be realized as the law of some measurable function defined on $[0,1]$ . This does not mean that the probability space is irrelevant in every problem. It means that if one only wants to realize a single law, one can often choose a convenient background space. The law is the abstract measure; the random variable is a concrete representation of that law.

There is another reason this viewpoint is powerful. A good construction of a random variable is not just a way of proving that a measure exists. It can give us a way to sample from the measure, compare two measures, and put several random objects on the same probability space so that their relationship becomes visible. In practice, this is often how probabilists think. A probability law tells us the distribution of an object, but a sampling rule tells us how to realize that object from some basic source of randomness. Once we have such a realization, we can build several objects together and study them pathwise, not only distributionally.

For example, suppose $0\le p\le q\le1$ . We can construct a Bernoulli random variable with parameter $p$ by taking a uniform random variable $U\sim U[0,1]$ and setting $X=\mathbf 1_{{U\le p}}$ . Similarly, we can construct a Bernoulli random variable with parameter $q$ by setting $Y=\mathbf 1_{{U\le q}}$ using the same uniform random variable. Then $X$ has the Bernoulli- $p$ law, and $Y$ has the Bernoulli- $q$ law, but now we have more than the two marginal distributions. Since $p\le q$ , we have

$\displaystyle X\le Y \quad\text{for every value of }U.$

This is much stronger than saying merely that $\Pr(X=1)\le\Pr(Y=1)$ . We have built the two random variables on the same probability space in such a way that one is always less than or equal to the other. This is a coupling. A coupling is a joint construction of random variables with prescribed marginal laws. The marginals tell us what each variable looks like separately; the coupling tells us how they are related when placed on the same space. This is one of the main uses of finding a good sampling representation. If we had constructed the Bernoulli- $p$ variable on one probability space and the Bernoulli- $q$ variable on another unrelated probability space, the inequality $X\le Y$ would not even make sense pointwise. They would live in different worlds. By realizing both variables as functions of the same uniform random variable, we put them into one world where comparison becomes possible. The laws are the same as before, but the representation gives extra structure.

This is why choosing the probability space is not completely innocent. If we care only about one random variable, many sample spaces may represent the same law equally well. But if we care about comparison, dependence, monotonicity, convergence, or simultaneous construction, then the choice of sample space matters. A clever sample space can put many random objects together so that their relationship is visible point by point. This is the philosophy behind many coupling arguments in probability: rather than compare two distributions only by formulas, we build random variables with those distributions on the same space and compare the random variables directly.

The infinite coin-toss measure is a richer example because the target space is infinite-dimensional. The outcome of infinitely many fair coin tosses is a sequence $(\omega_1,\omega_2,\ldots)$ with each $\omega_i\in\{0,1\}$ , so the natural space is $\{0,1\}^{\mathbb N}$ . We want a probability measure for which each coordinate is fair and the coordinates are independent. In concrete terms, if we prescribe finitely many coordinates, say $\omega_{i_1}=a_1,\ldots,\omega_{i_k}=a_k$ , then the probability should be $2^{-k}$ . These finite-coordinate events are cylinder events, and the product sigma algebra is generated by them. One can construct this measure by declaring the probabilities of cylinder sets and applying an extension theorem. But there is also a direct pushforward construction. Write $\omega\in[0,1]$ in binary expansion,

$\displaystyle \omega=\sum_{k=1}^{\infty}\frac{\omega_k}{2^k},\qquad \omega_k\in{0,1}.$

The binary digits define a map from $[0,1]$ to $\{0,1\}^{\mathbb N}$ . There is a minor ambiguity at dyadic rationals, such as $1/2=0.1000\ldots=0.0111\ldots$ , but these points form a countable set and hence have Lebesgue measure zero. After choosing one convention on this null set, we push forward Lebesgue measure by the binary-digit map. The resulting measure is the fair infinite coin-toss measure. If we prescribe the first $k$ digits, the preimage is one dyadic interval of length $2^{-k}$ . If we prescribe $k$ digits in arbitrary positions, the preimage is a union of dyadic intervals whose total length is still $2^{-k}$ . Thus independence and fairness are encoded in the geometry of binary expansions.

This example is worth pausing over. The space $[0,1]$ and the space ${0,1}^{\mathbb N}$ look completely different. One is an interval; the other is a space of infinite sequences. But as probability spaces, the interval contains enough randomness to generate the infinite coin-toss law. The binary expansion map transfers the measure from one world to the other. We wanted a measure on an infinite product space, and we obtained it as the law of a random sequence constructed from a uniform point in the interval. This is the kind of idea that appears everywhere in probability: instead of constructing a measure directly on a complicated target space, construct a random object with values in that space and take its law. One often constructs a random object as a function of some basic source of randomness, and then studies the law of that object.

The pushforward viewpoint also clarifies independence. Suppose $X:\Omega\to S$ and $Y:\Omega\to T$ are random variables. Each has its own law: $\mu_X=X_\mathbb P$ and $\mu_Y=Y_\mathbb P$ . But the pair $(X,Y)$ is also a random variable, now taking values in $S\times T$ , and its law $\mu_{X,Y}=(X,Y)*\mathbb P$ is the joint distribution. The variables $X$ and $Y$ are independent exactly when the joint law factors as the product of the marginals: $\mu{X,Y}=\mu_X\otimes\mu_Y$ . In more elementary terms, this says $\Pr(X\in A,Y\in B)=\Pr(X\in A)\Pr(Y\in B)$ for measurable $A$ and $B$ . But the measure-theoretic statement is more structural: independence means that the pushforward measure of the pair is the product measure of the individual pushforwards.

This also shows why knowing the separate distributions of $X$ and $Y$ is not enough to know their joint behavior. The same marginals can be coupled in many different ways. If $U$ is uniform on $[0,1]$ , then the pair $(U,U)$ has both coordinates uniformly distributed, but the two coordinates are perfectly dependent; the joint law is concentrated on the diagonal ${(x,x):0\le x\le1}$ . If $U$ and $V$ are independent uniforms, then $(U,V)$ has two-dimensional Lebesgue measure on the unit square. The one-dimensional marginals are the same in both cases, but the joint laws are completely different. This is why probability theory is not just the study of individual measures, but of joint laws, dependence, independence, and couplings.

A random variable also has an informational meaning. If $X:\Omega\to S$ is measurable, then it pulls back the sigma algebra of $S$ to a sigma algebra on $\Omega$ . This sigma algebra, denoted $\sigma(X)$ , consists of all events whose truth can be decided by knowing the value of $X$ . If $X$ is real-valued, then events like ${X\le a}$ , ${X\in B}$ , and ${X^2+X>3}$ belong to $\sigma(X)$ . They are all questions about the original sample point $\omega$ that can be answered once $X(\omega)$ is known. Thus random variables do two things at once: they push measures forward and they pull information backward. This is one of the reasons they are so fundamental.

The informational viewpoint becomes indispensable in martingale theory and stochastic processes. A filtration is an increasing family of sigma algebras $\mathcal F_1\subseteq\mathcal F_2\subseteq\cdots$ , interpreted as information accumulating over time. A process $X_n$ is adapted to this filtration if $X_n$ is measurable with respect to $\mathcal F_n$ , meaning its value is known at time $n$ . Conditional expectation is also best understood as projection onto a smaller information sigma algebra. So probability is not merely measure theory with total mass one; it is measure theory together with a language of observation, information, dependence, and time.

Pushforward measures also construct singular and fractal measures, not just familiar distributions with densities. Consider the middle-thirds Cantor set. A point in the Cantor set can be written in ternary expansion using only the digits $0$ and $2$ : $x=\sum_{k=1}^{\infty}2\omega_k3^{-k}$ , where each $\omega_k\in{0,1}$ . Start with infinitely many fair coin tosses $(\omega_1,\omega_2,\ldots)$ and define

$\displaystyle X=\sum_{k=1}^{\infty}\frac{2\omega_k}{3^{k}}$ .

The law of $X$ is the Cantor measure. It is a probability measure supported entirely on the Cantor set, which has Lebesgue measure zero. It has no atoms, so no single point has positive mass. Thus it is singular continuous: singular with respect to Lebesgue measure, but not discrete. This is an excellent example of how a very simple random procedure can produce a subtle measure. We did not define the Cantor measure by writing down a density, because no density with respect to Lebesgue measure exists. We defined it as the distribution of a random series.

This random-series perspective is much broader. Let $\xi_1,\xi_2,\ldots$ be independent random signs, each equal to $1$ or $-1$ with probability $1/2$ , and consider a series

$\displaystyle X=\sum_{n=1}^{\infty}a_n\xi_n$

with $\sum |a_n|<\infty$ . This series defines a measurable map from the sign sequence space $\{-1,1\}^{\mathbb N}$ into $\mathbb R$ , and its law is the pushforward of the infinite product coin-toss measure. For different choices of the coefficients $a_n$ , one obtains very different measures. If $a_n=2^{-n}$ , the resulting distribution is closely related to Lebesgue measure on an interval. If $a_n=3^{-n}$ , one obtains a Cantor-type singular measure. If $a_n=\lambda^n$ , one obtains Bernoulli convolutions, a deep family of measures whose absolute continuity or singularity depends delicately on the number $\lambda$ . Thus a measure may be constructed not by describing its values on sets, but by describing a random infinite process and taking its distribution.

The same idea scales up to stochastic processes. A stochastic process $(X_t), {t\in T}$ can be regarded as a single random object taking values in a path space. Instead of viewing each $X_t$ separately, define $X(\omega)=(X_t(\omega)), {t\in T}$ . Then $X$ is a random element of $S^T$ , and its law is a probability measure on the path space. Infinite coin tossing is the case $T=\mathbb N$ and $S={0,1}$ . Brownian motion is a more sophisticated example: it can be understood as a probability measure on the space $C([0,\infty),\mathbb R)$ of continuous real-valued functions. The coordinate maps $B_t(\omega)=\omega(t)$ are random variables, and the law of the whole process is a measure on the function space. This measure is Wiener measure.

There is a close relation here between extension constructions and pushforward constructions. For Brownian motion, one often begins by specifying all finite-dimensional distributions: the joint laws of $(B_{t_1},\ldots,B_{t_k})$ for finite sets of times. These finite-dimensional laws must be consistent when one forgets coordinates. Kolmogorov’s extension theorem then constructs a probability measure on an infinite product space. Further regularity arguments show that the measure is concentrated on continuous paths. In this way, the old extension method reappears, but now in a probabilistic form: finite-dimensional probability rules are extended to an infinite-dimensional measure. Once the measure exists, the coordinate maps become random variables, and the whole process becomes a random path. Thus extension and pushforward are not competing philosophies; they work together. Extension builds large probability spaces, and pushforward extracts laws of random objects from them.

Weak convergence is another place where the law, rather than the underlying sample space, becomes the main object. If $X_n$ are random variables with laws $\mu_n$ , and $X$ has law $\mu$ , then $X_n\Rightarrow X$ means that $\mu_n$ converges weakly to $\mu$ . On a metric space, this means that for every bounded continuous test function $f$ , one has $\int f,d\mu_n\to\int f,d\mu$ . Equivalently, $\mathbb E[f(X_n)]\to\mathbb E[f(X)]$ . This is another way of saying that convergence in distribution is convergence of pushforward measures. The random variables $X_n$ may even live on different probability spaces. For convergence in distribution, what matters is not a pointwise relationship between them, but the convergence of their laws. This is also why characteristic functions are so useful in probability. For a real-valued random variable $X$ , the characteristic function $\phi_X(t)=\mathbb E e^{itX}$ is the integral of the test function $x\mapsto e^{itx}$ against the law of $X$ . The law is being studied through its integrals against oscillatory functions. In limit theorems, characteristic functions are especially powerful because independence turns sums into products: the characteristic function of $X+Y$ is the product of the characteristic functions when $X$ and $Y$ are independent. Taking logarithms then turns products into sums. Thus the Fourier transform of the pushforward measure becomes a natural analytic tool for understanding sums of independent random variables.

At this point, one sees why random variables are not secondary objects in probability theory. They are the morphisms of probability theory: the maps that carry probability from one measurable space to another. They construct distributions by pushforward. They generate information sigma algebras by pullback. They form joint laws. They encode independence by product measures. They build stochastic processes as probability measures on path spaces. They allow us to discuss weak convergence without fixing a single universal sample space. Measure theory provides the underlying language of sets, sigma algebras, and integration; probability theory adds the dynamic language of random variables, laws, conditioning, independence, and limiting behavior.

There is one subtle correction worth making. It is sometimes said informally that a random variable “does not depend on the probability space.” Literally this is false. A random variable is a measurable function from a probability space, so it requires a domain and a probability measure. What is true is that the law of a random variable can often be studied independently of the particular representation. A Bernoulli random variable can be realized on a two-point space, on the unit interval by a threshold map, or as the first binary digit of a uniform point in $[0,1]$ . These are different functions on different spaces, but they have the same law. If we only care about the distribution of one random variable, the representation may be irrelevant. But if we care about several random variables at once, the joint representation matters, because dependence lives in the joint law, not in the separate marginal laws.

The final picture is this. There are two great ways to construct measures. One is extension: define a premeasure or outer measure from simple geometric or finite-dimensional rules, and extend it to a sigma algebra. This gives Lebesgue measure, product measures, and many infinite-dimensional probability measures. The other is transport: start with a measure already understood, apply a measurable map, and take the pushforward. In probability, this measurable map is a random variable, and the pushed-forward measure is its law. Once Lebesgue measure on $[0,1]$ exists, it becomes a universal reservoir from which many other probability measures can be produced: Bernoulli laws, exponential laws, arbitrary real distributions, infinite coin-toss measures, Cantor measures, random-series measures, and laws of stochastic processes. Thus probability theory is not merely the study of measures with total mass one. It is the study of how measures are transported by maps, how random objects generate laws, how information is encoded by sigma algebras, how dependence is represented by joint distributions, and how finite-dimensional intuition becomes infinite-dimensional measure. Random variables are central because they are the mechanism by which probability moves from one space to another.

Share this:

Related

Leave a Reply Cancel reply