The Law of Large Numbers - Gambler's fallacy

The “Law of Large Numbers” (a.k.a. Bernoulli's law) is a statistical theorem that is defined as follows - The average of a large number of independent measurements of a random quantity tends toward the theoretical average of that quantity.

Simply stated, the law of large numbers states that if you repeat a random experiment, such as tossing a coin or rolling a die, many, many, many times, your outcomes should on average be equal to the theoretical average.

If you toss a “fair coin”, there is a 50% chance that it would be heads. However, it does not necessarily mean that if you toss the coin 2 times, you will get heads one time. However, if you repeat this experiment many, many times – you should get closer to 50% (number of times you get heads).

Mathematician John Kerrich tossed a coin 10,000 times while interned in a prison camp in Denmark during World War II. At various stages of the experiment, the relative frequency would climb or fall below the theoretical probability of 0.5, but as the number of tosses increased, the relative frequency tended to vary less and stay near 0.5, or 50 percent.

Insurance companies rely on this theorem. Out of a large group of policyholders the insurance company can fairly accurately predict not by name but by number, the number of policyholders who will suffer a loss. In other words, the more cars you insure, the more accurately you can predict the number of cars likely to be stolen. However, note that the individual policyholder cannot accurately predict (whether he will be in an accident this year or not).

Casinos also use this principle. The gambler's fallacy, also known as the Monte Carlo fallacy, is the false belief that odds increase or decrease depending upon recent occurrences. So, if black has come up 5 times in succession at a roulette table, the gambler is almost sure that the next one is red to even out the pattern. As discussed earlier, while it is true that the pattern will even out in the “long” run, it may not be necessarily true in the “short” run. Casinos take advantage of it (since they have the advantage of the long run).

Another area of confusion is how large is “large”? Is it 100 samples or 1000 samples or 10,000 samples? The technical definition is closer to infinity. However, from a practical standpoint, it is subjective and depends on the circumstance and the span of control.

A key statement to be understood is “A chance event is uninfluenced by the events which have gone before”. A coin toss, rolling a die, roulette is a game of independent trials. While using “samples” is certainly an effective statistical technique, be aware of the effect of the “Law of Large Numbers”. Do not fall prey to the gambler’s fallacy.