Constructive Proof of Chernoff Bounds

Chernoff bounds allow us to get the concentration of sum of independent binary random variables. (We can relax the independence to negative correlation/association etc). The standard proof goes by considering the idea of Bernstein to use moment generating function which allows us to decouple the variables. That is using the fact that exponentials behave nicely under addition.

\displaystyle e^{t(X_1+X_2+\cdots X_n)} =\prod_{i=1}^n e^{tX_i}

Applying Markov’s inequality and optimizing the parameter t, we get the Chernoff bounds.

The following paper gives a constructive proof of the bound. Here is how the proof goes. If the probability that the sum of the variables is much larger than the expectation is not too small, then we can by sampling a subset of the variables, we get that the probability that the subset of variables are all 1 is not too small. By appropriate large enough choice of the size of subset of variables, we get a contradiction because by independence (or relaxed conditions like negative correlation) we get that this probability should be very small.

Note that the proof is essentially by counting!

Click to access RANDOM2010.pdf

Posted in $.

Leave a comment