Probability as Extended Logic

Modern probability theory is typically derived from the Kolmogorov axioms, using measure theory with concepts like events and sample space. In one way, it's intuitive to understand how this works as Laplace wrote:

The probability of an event is the ratio of the number of cases favorable to it, to the number of all cases possible, when [the cases are] equally possible. ... Probability is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

However, the intuition of this view of probability breaks down when we want to do more complex reasoning. After learning probability from the lens of coins, dice and urns full of red and white balls, I still didn't feel that I had have a strong grasp about how to apply it to other situations -- especially ones where it was difficult or too abstract to apply the idea of "a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible". And then I read Probability Theory: The Logic of Science by E. T. Jayne.

Jayne takes a drastically different approach to probability, not with events and sample spaces, but rather as an extension of Boolean logic. Taking this view made a great deal of sense to me since I spent a lot of time studying and reasoning in Boolean logic. The following post is my attempt to explain Jayne's view of probability theory, where he derives it from "common sense" extensions to Boolean logic. (Spoiler alert: he ends up with pretty much the same mathematical system as Kolmogorov's probability theory.) I'll stay away from any heavy derivations and stick with the intuition, which is exactly where I think this view of probability theory is most useful.

Read more…

Accessible Satisfiability

Even though it was just a scant few years ago, my research in grad school seems like it was from another lifetime. Nowadays I deal with data and most of my code revolves around manipulating and extracting interesting insights from it. However in my previous life I spent quite a bit of time dealing with satisfiability problems. So before I start writing about data and related topics, I thought I'd kick it old school and write about some topics from my formative years.

Read more…

Functions and Procedures

I have something to admit: I've never done any serious programming in a functional language. Yes, yes, I've done some small school assignments in Scheme (a dialect of Lisp), even helping out my friend from another university with his Scheme assignment but nothing real [1]. Does this make me a worse programmer? Probably not, I imagine most developers haven't done anything real with functional programming. Although that's probably not what you'd expect from reading Hacker News, where you don't know programming if you haven't tried Clojure or Haskell [2].

My position is much more pragmatic: I'm interested in tools and techniques that help me solve problems faster, cleaner and with less headache. Ideas and concepts from functional programming languages are supposed to help with at least some of that -- and they do; they're just not a panacea for all your programming woes. Like most things a solid grasp of the fundamentals goes a long way. So the topic of this post is about something pretty fundamental: a subroutine. In particular, two important kinds of subroutines: procedures and pure functions.

Read more…

Common Sense and Investing

Preface: Second post and I'm already off topic according to my subtitle. I do have another post that is more topic appropriate but it's been pretty hectic so I haven't had time to finish it. Although, investing is definitely a kind of technical topic involving data and numbers, so one could argue it is still on topic. And argue I shall! Enjoy!

A lot of smart people I meet frequently mention that they have no clue about investing or other related financial matters. They're quite right that modern finance can get complicated really fast such as option pricing models (that require differential equations and other such complicated maths) but as in most things though, a little common sense goes a long way. So I want to share an analogy that I think will help people understand a rational way to invest i.e. value investing. In particular, the discussion will be geared towards investing in publicly traded stocks but the general idea applies to most asset classes.

Read more…

A First Post

Hi all, this the first post in my technical blog. For a long time I've resisted making a blog about anything technical because, to be frank, I didn't feel like I was qualified. What makes me more qualified now? Nothing really, I'm just a bit less afraid of being wrong.

So maybe a good place to start is what I used to setup this blog and what the domain/title mean. For my personal blog (, where I discuss random worldly wisdom that I come across, I use WordPress. For this one, I used something a bit more technical (because why not?):

Github Pages
Great place to host a static website (free) but also has some of that "hacker cred". It's quite easy to use (if you've used git before) and also quite well documented on how to get your domain setup.

Nikola (static site generator)
The recommended static site generator with Github Pages is Jekyll but one thing I wanted for this site is to make use of IPython Notebooks. After a bit of searching, I found Nikola to be a good match because it supports IPython Notebooks out of the box (as opposed to Jekyll which required some fiddling), and seemd to have a reasonably simple setup. The really interesting line from the documentation is this quote: DON'T READ THIS MANUAL. IF YOU NEED TO READ IT I FAILED, JUST USE THE THING. Exactly the sentiment I was going for.

Bootstrap3 theme and Readable Bootstrap theme.

Bootstrap3 is the default theme for Nikola, conveniently using Bootstrap, the most popular HTML/CSS/JS framework for the web. Since I'm not a very good designer using high quality themes/framework is definitely ideal.

The default Nikola theme, however, was not too my liking. A bit too spartan for me. Fortunately, Nikola has a really cool feature where you can take an existing Bootstrap theme from and skin the default Nikola Bootstrap theme. I chose the Readable theme because I'm a huge fan of making things easy-to-read (you should see how large the font size is on my devices, frequently referred to as "old people font size").

Even with the new Bootstrap theme, the site layout didn't quite look right. So I fiddled around with the layout and placement of items, pretty much copying the layout of another blog I really enjoy programming in the twenty-first century by James Hague. It was interesting playing around with HTML templates and CSS but pretty much confirmed that it's not really what I like doing. Getting to a satisfactory layout was as much effort as I really wanted to put in.

Read more…

I'm Brian Keng, a former academic, current data scientist and engineer. This is the place where I write about all things technical.

Twitter: @bjlkeng



RSS feed

Signup for Email Blog Posts