The Gambler's Fallacy and The Law of Small Numbers

Brian Keng

2015-11-01 11:08

Games and gambling have been part of human cultures around the world for millennia. Nowadays, the connection between games of chance and mathematics (in particular probability) are so obvious that it is taught to school children. However, the mathematics of games and gambling only started to formally develop in the 17th century with the works of multiple mathematicians such as Fermat and Pascal. It is then no wonder that many incorrect beliefs around gambling have formed that are "intuitive" from a layman's perspective but fail to pass muster when applying the rigor of mathematics.

In this post, I want to discuss how surprisingly easy it is to be fooled into the wrong line of thinking even when approaching it using mathematics. We'll take a look from both a theoretical (mathematics) point of view looking at topics such as the Gambler's Fallacy and the law of small numbers as well as do some simulations using code to gain some insight into the problem. This post was inspired by a paper I recently came across a paper by Miller and Sanjurjo^[1] that explains the surprising result of how easily we can be fooled.

Probability as Extended Logic

Brian Keng

2015-10-14 20:30

Modern probability theory is typically derived from the Kolmogorov axioms, using measure theory with concepts like events and sample space. In one way, it's intuitive to understand how this works as Laplace wrote:

The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible, when [the cases are] equally possible. ... Probability is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

However, the intuition of this view of probability breaks down when we want to do more complex reasoning. After learning probability from the lens of coins, dice and urns full of red and white balls, I still didn't feel that I had have a strong grasp about how to apply it to other situations -- especially ones where it was difficult or too abstract to apply the idea of "a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible". And then I read Probability Theory: The Logic of Science by E. T. Jaynes.

Jaynes takes a drastically different approach to probability, not with events and sample spaces, but rather as an extension of Boolean logic. Taking this view made a great deal of sense to me since I spent a lot of time studying and reasoning in Boolean logic. The following post is my attempt to explain Jaynes' view of probability theory, where he derives it from "common sense" extensions to Boolean logic. (Spoiler alert: he ends up with pretty much the same mathematical system as Kolmogorov's probability theory.) I'll stay away from any heavy derivations and stick with the intuition, which is exactly where I think this view of probability theory is most useful.

Accessible Satisfiability

Brian Keng

2015-10-03 20:28

Even though it was just a scant few years ago, my research in grad school seems like it was from another lifetime. Nowadays I deal with data and most of my code revolves around manipulating and extracting interesting insights from it. However in my previous life I spent quite a bit of time dealing with satisfiability problems. So before I start writing about data and related topics, I thought I'd kick it old school and write about some topics from my formative years.

Functions and Procedures

Brian Keng

2015-09-06 03:04

I have something to admit: I've never done any serious programming in a functional language. Yes, yes, I've done some small school assignments in Scheme (a dialect of Lisp), even helping out my friend from another university with his Scheme assignment but nothing real 1. Does this make me a worse programmer? Probably not, I imagine most developers haven't done anything real with functional programming. Although that's probably not what you'd expect from reading Hacker News, where you don't know programming if you haven't tried Clojure or Haskell 2.

My position is much more pragmatic: I'm interested in tools and techniques that help me solve problems faster, cleaner and with less headache. Ideas and concepts from functional programming languages are supposed to help with at least some of that -- and they do; they're just not a panacea for all your programming woes. Like most things a solid grasp of the fundamentals goes a long way. So the topic of this post is about something pretty fundamental: a subroutine. In particular, two important kinds of subroutines: procedures and pure functions.

Common Sense and Investing

Brian Keng

2015-08-30 10:44

Preface: Second post and I'm already off topic according to my subtitle. I do have another post that is more topic appropriate but it's been pretty hectic so I haven't had time to finish it. Although, investing is definitely a kind of technical topic involving data and numbers, so one could argue it is still on topic. And argue I shall! Enjoy!

A lot of smart people I meet frequently mention that they have no clue about investing or other related financial matters. They're quite right that modern finance can get complicated really fast such as option pricing models (that require differential equations and other such complicated maths) but as in most things though, a little common sense goes a long way. So I want to share an analogy that I think will help people understand a rational way to invest i.e. value investing. In particular, the discussion will be geared towards investing in publicly traded stocks but the general idea applies to most asset classes.