Poker Coder

One Year of the Odds Oracle!

2012-01-09T09:18:00.000-08:00

Summing it Up

It has been just about a year since the first closed beta of the Odds Oracle 0.1 came out. As of the writing of this article, we are on version 1.8.5 and just on the cusp of 2.0. User feedback has been overwhelmingly positive, and from a business perspective things are going reasonably well. This couldn't have been possible without the constant support, suggestions, questions and encouragement of Odds Oracle users - I thank you all! Below is a summary of what we have achieved together, and what I feel are the good, great, and not-so-great parts of the Odds Oracle as it stands now.

The Good

"Show PQL" buttons everywhere help users to make the leap from the user-interface to custom PQL queries.
Multiple input/output sources (hand history importer, command line, spreadsheets) have made it more convenient to run simulations.
Saved ranges/macros save time for users with complicated pre-constructed ranges
Multi-core support has been a (ahem) core design principle from the get-go. This has made it relatively easy for the Odds Oracle to take full advantage of the latest hardware.
Hand vs. Range equity graphs. These are what people usually want when they graph equities - this is now the default graphing algorithm.
Interactive hand stats on flop/turn - while not pretty to look at, this allows slicing-and-dicing of hand types in a way that would be too cumbersome otherwise.

The Great

"Ask the Oracle". Being able to ask probability questions using pull-down menus is awesome.
PQL was a big gamble, and it has paid off. PQL powers almost everything in the Odds Oracle and the website. As a result, there is a fairly straightforward correspondence between elements in the UI and generated PQL queries (this is something that will be made more explicit in 2.0)
Generic range syntax has turned out to be a big winner. Having one syntax available for everything has proven a boon, especially for people who play multiple games.
The core re-write of December, 2010 was definitely worth it. I've learned a lot since propokertools.com opened its doors several years ago, and starting over with this accumulated knowledge has made the code-base smaller, faster, and easier to work with. For users, that means new features can come out faster.

The Not So Great

The focus on the 0.x and 1.x series of the Odds Oracle has been to create an ever-increasing set of questions that can be answered. As a result, not enough time and effort has been spent on creating a better user experience. Specifically:

The range pickers are rudimentary at best.
The feature set is too spread out over different windows (range pickers, hand explorers, main window, equity/graph/ask the oracle/common questions buttons...). As a result, the common denominator of most of the Odds Oracle (that would be PQL) is somewhat obscured.
Ask the Oracle is too limited in its current form. Ideally, there should be a 1:1 correspondence between PQL and the UI - any PQL query you can write you should be able to construct in the UI (no, the PQL runner and CLI don't count!)

What's in Store for 2.0

My goals for 2.0 in priority order are:

Create a streamlined UI, allowing users to do most of what they want in a single window without the need to write custom PQL.
Create best-in-class range pickers.
Continue to increase the power of PQL by adding more core functions and features.
Spend some time working on marketing.
Create more instructional materials (videos, docs, ebooks?)
Make the UI look prettier.

If you have comments, questions, complaints, burning feature requests, or just want to shoot the breeze, please feel free to leave a comment here or on the free ProPokerTools Forum.

Here's to a productive 2012!

- Dan @ ProPokerTools

AK vs half-dead pair (User Request)

2011-04-27T14:30:00.000-07:00

I got a fun question from a user today. "How does AK (suited or unsuited) fare against any pair assuming that one of the pair's set outs are dead?"

We can answer this with a little PQL hackery. I say "hackery" because there isn't a way to connect dead cards with other hand ranges in PQL at the moment. We will perform an all in equity simulation with AK vs any pair. If the pair gets quads by the river, we throw out the sim, since one of the set cards is supposed to be dead. If the pair does not get a matching card by the river, we keep the sim, since the dead card never appeared. If the pair hits exactly one of its set cards but doesn't have quads, we have a 50/50 chance that the sim should be thrown out. Here's where the hack comes in - we can just throw out all sims in this case where one of the pair cards is a heart, as 50% of pairs have a heart in them. Note that if the pair is Aces or Kings, then we should not be able to hit the board at all, so we need to exclude that as well.

Here is the PQL query I came up with:

select avg(riverEquity(hero))
from game='holdem', hero='AK', villain='AA-22', syntax='Generic'
where
case handBoardIntersections(villain, river)
  when 0 then true
  when 2 then inRange(villain,'*!h!AA-KK')
end
and not (handtype(villain, river) = quads)

Answer - AK has 48% equity.

Whence PQL?

2011-04-19T12:31:00.000-07:00

"A-ha!" vs. "Ho-hum"

In the popular imagination, innovation comes from a flash of genius - a sudden, almost supernatural insight that often catches the inventor himself unawares. In reality, most innovation is far more mundane - small, incremental (and often obvious) improvements combine to make something new possible. Indeed, almost everything "new" on propokertools.com has found its genesis in incremental improvements - new games, graphs, and tools that build on what came before. After all, the very first version of propokertools.com was an answer to the question "What would an online version of PokerStove look like?"

PQL is the exception - it is the closest I've come to a "Eureka" moment yet.

Three as One

Here are three questions that have been answerable at one time or another via propokertools:

Given that player one has XX, player two has YY, and the board is ZZZ, what is the all-in equity for player one?
Given that player one has XX, player two has YY, and the board is ZZZ, how often does player one have at least two pair by the river?
Given that player one has XX, player two has YY, and the board is ZZZ, what does the graph of equities look like for player one?

Notice anything? These questions all have the same setup, namely "Given that p1 has ..., p2 has...". Let's call this the situation. After each situation, there is a question, which we can refer to as, well, the question. The three questions can all be rewritten as:

Given situation S, what is the answer to question Q?

In programmer-speak, we can say that we have abstracted out the concept of a situation and a question. This process of abstraction is one of the principal (some would say the principal) jobs of a programmer. And there is nothing particularly noteworthy about the abstraction above - any programmer worth her salt would spot it almost immediately.

Exposing the Abstraction

I spent a few weeks rewriting propokertools in December of 2010 (yes, all of propokertools (except for razz) got rewritten). One of the first things I did after throwing out several years' worth of accumulated cruft was to see how easy it would be to answer new poker questions with my re-written code. I began to write small programs to answer questions like "Given situation S, how often does player one win when he was an underdog before the flop?" and "Given situation R, how often is the winning hand a straight or better?". These programs were, for the most part, satisfyingly short and sweet. At first, I thought I would expose these little mini-programs as individual tools on the website. Then it hit me.

Why not let the users ask the questions?

Enter the DSL

"DSL" stands for "domain-specific language". A DSL is a language, usually small, used to describe a very particular area, or domain. For a somewhat contrived example of a DSL in action, one need only step up to the counter at a popular chain of coffee-shops during the morning rush. "Half-caf grande iced americano" means "using half regular beans and half decaffeinated beans, brew an espresso and dump it in a large to-go container, adding ice cubes and cold water." The key behind a successful DSL is that it is concise and unambiguous. An even more concise version of our coffee DSL can be seen behind the counter, as the people at the cash-registers write only a few characters of text on each coffee cup to completely describe what is to be brewed.

Another DSL you may be familiar with is the ProPokerTools "range of hands" DSL. For instance, in omaha, "AA**" means a hand with at least two aces in it.

Why Invent What You Can Steal?

To allow users to ask custom questions, I needed to create a new DSL. I had a number of requirements:

Need to be able to ask multiple questions
Need to be able to compose questions using boolean operators (such as AND, OR, and NOT)
Need to be able to fully describe the situation
Need a semantics that is easy to explain

As it happens, there is a language that has been used for decades in another domain that provides pretty much all of what we want. That language is SQL, or Structured Query Language. SQL is the industry-standard way to query databases. While the correspondence between SQL and our poker situation is not perfect, it is good enough, and given the availability of free tools around SQL, it became the obvious choice. Thus was born PQL - Poker Query Language.

Eating Your Own Dog Food

The most obvious way to make sure PQL was truly useful was to use it myself. As a result, all of the major features on the website and in the downloadable Odds Oracle are powered by PQL queries.

2011-01-03T10:52:00.000-08:00

Downloadable ProPokerTools 0.1 is done!

I promised myself I would release a downloadable version of ProPokerTools in 2010. The bad news? I did not reach my goal. The good news? I am only three days late. A random pool of users have been selected from those who submitted a survey from the propokertools website. Check your inbox!

Generating all 3-way holdem matchups

2010-12-01T15:45:00.001-08:00

By user request, I have generated equities for all 3-way races between holdem hand types (ex 22 vs. AJs vs KQo) and stored the values in a file. My new quad-core laptop is still catching its breath from the effort. If you would like the results (warning - 15+Meg file), please email propokertools support and I'll send it your way.

2010-07-05T13:53:00.000-07:00

Another (smaller) razz fix

Fixed a bug today involving dependent probabilities in razz. In a nutshell, hands with range syntax in both the first two and last cards were overweighting paired cards on seventh street. Thanks to the folks on this thread for bringing this to my attention.

Fixing Razz

2010-04-06T10:24:00.000-07:00

Trying to Get it Right

Joel Spolsky wrote a famous piece on how to do job interviews. His mantra throughout the article is that you want to hire people who are "Smart" and "Get Things Done" (I hope he hasn't copyrighted these phrases yet). To these two attributes, I would humbly add "Tries to Get It Right", though I'm hardly the first one to suggest this. When hiring a software engineer, you want someone who does his best to get it right the first time, and if necessary, gets it right the second or third or fourth time if things go wrong.

And things will go wrong. When it is discovered, for instance, that something he has written is based on faulty assumptions, your engineer shouldn't react too defensively, at least not for long. Ideally, he should look like someone just punched him in the gut, and then he should scramble to fix whatever it is that needs fixing, throwing away reams of code as necessary.

Enough with the Digression, What Needs Fixing?

Thanks to a number of posters on the 2+2 forums (the thread is here - the discussion starts toward the middle of page two), a highly counter-intuitive result was brought to my attention involving randomized razz simulations. In a sense, it isn't a bug - the code is actually performing the way I intended it. No, it is worse than a bug - it is a behavior, hidden from the user, which gives results different from what, in almost all cases, was probably intended.

The Old Approach

On the propokertools razz simulator, you can enter ranges of hands for each player. For instance, if you have a2 with a 9 up, and you see an early raiser showing a seven with a bunch of low cards following, you might put him on a range of 'a seven up with two downcards seven or lower without any duplicates'. Here is a link to this situation in propokertools with some dead cards added, assuming everyone but the raiser folds to you. Our simulation gives a29 around 42% all-in equity in this situation.

"But how are the random hands generated?" you ask. Fair enough. Essentially, for each possible rank in each hand, a selection is made randomly. Then, some statistics are performed to ensure a fair probability for that choice (please pardon the hand-waving here as this part of the algorithm is not important for the purposes of this discussion). Then, the next rank is chosen, etc. etc.

For instance, for our a29 hand, we always choose an ace, a two, and a nine, since no 'range of hands' syntax was employed for any of the cards.

For the (7-7-7) hand, we first choose a rank for the first downcard at random (lets say 6), then the second downcard( lets say 4), and then we always choose a 7 for the third card. Great, we chose 647. (If the second card chosen happened to also be a 6, we would have had to start over, as our range stipulates no duplicate ranks in the first three cards).

So, we have two 'real' hands a29 vs 647 - we can now deal out the rest of the cards for each hand and see who won (or if there was a tie).

Wash/rinse/repeat 600,000 times and VOILA! 42% equity.

Investigating the Old Approach

Let's say we race "(3-3-)45678" vs "A345678" with no dead cards. The Old Approach involves picking a random rank for each of the "3-" cards.

For the first card, if we randomly pick a deuce, then our choices for the next card are an ace or a three, with equal probability. We win with the ace and lose with the three, giving 50% equity on average when we pick a deuce for the first card.

For the first card, if we randomly pick an ace, we then have four ways to pick a deuce and three ways to pick a three for the second card. Picking a deuce for the second card nets a win, and a three gets a tie. So, on average, we win 4 times and tie 3 times for every seven random ace picks.

For the first card, if we randomly pick a three, the situation is the opposite of picking an ace. We lose 4 times and tie 3 times for every seven times this happens.

It should be obvious at this point that the Old Approach gives us 50% equity on average, which matches our intuition nicely.

Now, for the problem scenario lets race "(3-2-)45678" vs "A345678". Notice that the only thing I did was change the second rank of the first hand from '3-' to '2-'. This causes the equities to change from being even to being 2 to 1. That's right - with the Old Approach, the "(3-2-)45678" gets 66.66% equity. I now quote from jbrennan's post on 2+2 which efficiently explains why this is the case:

the simulator would seem to do this:

A2 -- valid hand
A3 -- NOT valid (second card not a deuce or less)
2A -- valid hand
23 -- NOT valid (second card not a deuce or less)
3A -- valid hand
32 -- valid hand

Since A2, 2A, or 32 can each happen 12 ways, and 3A can happen 9 ways, we end up with 45 possible hands -- 24 winners, 12 losers, and 9 ties. That matches up exactly with the output of the simulator

The Problem with the Old Approach

The difference in equity between the '(3-3-)' '(3-2-) hands is highly counter-intuitive. In both cases, it seems pretty clear that the person running the simulation intended 'two downcards 3 or lower that are different'. That person most certainly did NOT mean 'look at the first downcard - if it is a three or lower, look at the second downcard - if a nonmatching ace or deuce, play the hand'.

Downcards should not be ordered.

The New Approach

I scrapped the old random razz hand generation code and instead retrofitted the existing, well-used code for generating hands in hold'em and omaha. Here is the algorithm that translates a 'suitless' razz simulation into a 'suitfull' one.

Examine each rank in the simulation.
For cards where only one rank is possible (dead cards, ranks in hands like 'a23'), pick a card from the deck with the appropriate rank and assign it as part of the range. Remove that card from the deck.
For cards where more than one rank is possible, assign a range consisting of the specified ranks and any suit.
Remove any sequences of ranks in hands that violate the 'no pairs' constraints (the parentheses used to force no duplicates)
Race!

Here is an example of the procedure applied to "A345678" vs "(3-3-)45678"

Assigning arbitrary suits to non-range-based ranks, we have "As3s4s5s6s7s8s" vs "(3-3-)4h5h6h7h8h, and As3s4s5s6s7s8s4h5h6h7h8h are removed from the deck.
Assigning ranges, we have "As3s4s5s6s7s8s" vs "[3*|2*|a*][3*|2*|a*]4s5s6s7s8s" (where * indicates any suit). (I will not list all of the possibilities for the second hand here).
Remove hands that violate the constraints (hands like 33..., 22... from hand two).
We're done!

You can walk through these steps yourself and see that the ranges do not change if you alter the (3-3) to read (3-2) - the equities are 50% for both simulations.

ProPokerTools gets a Makeover

2010-04-02T10:23:00.001-07:00

ProPokerTools gets a Makeover

ProPokerTools has been around for a while now, and I've been humbled by the continued encouragement and support of the poker community. I've generally been proud of the work I've done on the site, and in general I would categorize it as a successful web app.

Except for one aspect - the, um, WEB part of it. Yeah.

Though undoubtedly useful, the design felt dated from day one. In all honesty, I had only a rudimentary understanding of how to design web interfaces when I began. Scratch that - I had NO IDEA how to design a web interface. Now I can confidently say that my web design skills have advanced from nonexistent to merely mediocre. Progress!

I hope you like the new implementation of the ProPokerTools Simulator. I certainly find it easier to use and nicer to look at, but then again, not everyone is like me. If you have any feedback, any feedback at all, please email support at propokertools dot com or leave a comment here.

Depending on how things go (and how donations/advertising works out), I may be hiring a friend who is actually GOOD at ui design to do a once over of the site. At the very least I should be able to get a decent logo out of him. Stay tuned.

New Front-end, Same Back-end Goodness

I'm using a more modern technology for serving up the ProPokerTools web pages, but the actual simulation code has not changed. I made this change for two reasons:

1) I actually know a few things about the newer framework, as I work with it in one of my day jobs.
2) I can implement changes to the site far, far more rapidly than before.

So far the switchover has proved (relatively) painless. Kudos to the folks at RimuHosting for keeping my servers running.

Well that's embarassing...

2006-08-10T09:08:00.000-07:00

Last night I did something I rarely do. I complained about my bad run at poker - to my wife, no less (who is an angel and very supportive). "I just feel like I can't beat this game anymore," is something I recall saying. "Did I ever know how to play?" She was very nice, but had to leave for an outing with a friend. So what did I do? I played poker online, of course.

I entered a $33 limit hold'em tournament, which was already a mistake, as I had intended to enter a no-limit tournament. There were 60 players. Going into the final table, I was one of the top four stacks. By the time we got down to two players, my opponent had a 9-1 chip lead.

I won.

Moral of the story? I guess I should complain about my bad luck more often...

Downloadable Hand Orderings

2006-07-26T16:13:00.000-07:00

By popular demand, here are the generated hand orderings as described in this blog entry :

Towards a Meaningful Ordering of Hands

2006-07-15T15:50:00.000-07:00

Overview

Many poker books include chapters with hand rankings, and many software tools (including some at propokertools.com) support specifying the top N% of hands. Almost immediately the question arises - how do we create a meaningful ordering of hands?

One possibility is to use human judgement in creating hand rankings. This has worked fairly well in strategy books on hold'em, partly because the game is reasonably well understood, and partly because there are only a small number of distinct hold'em hands (169 to be exact). For omaha, it is not practical to list all the hands in a chart, as there are 16,432 distinct hands. Some authors have come up with point systems for evaluating omaha hand strength, and others have given general guidelines.

A second possibility is to race each possible hand against a set of random hands (say, two or three). Up until recently, propokertools.com used this approach. It is very easy to generate an ordering this way with less than a hundred lines of computer code, and the result will have no human biases. Upon inspection, however, the results leave something to be desired. This is not surprising, as random hands are rarely played in practice.

A third possibility is described by Billings, Davidson, Schaeffer and Szafron in "The challenge of poker" (a paper in "Artificial Intelligence"). At propokertools.com, we use hand orderings inspired by their approach. We created an evolutionary computer simulation, whereby the set of "good hands" is gradually refined, and hands are ranked against other "good hands".

The Evolution Program

To generate the ordering for each game:

Perform the following computation a total of ten times:
For each possible hand h:
Do the following many times (where "many" doubles each iteration):

Deal the hand h, and add one chip to the pot

Deal one random hand b, and add one chip to the pot (to simulate a "blind" hand)

Deal eight random hands. For each "good" hand (defined later), add one chip to the pot (to simulate a player calling). For all the rest, fold the hand

Deal a board, and award the pot to the winner(s)

We define a "good" hand as one which, on average, wins more than one chip from the pot. Put another way, a "good" hand is one that does better than break-even. For the first iteration, all hands are considered "good" hands. As the simulation is run, the set of good hands shrinks, and by the seventh iteration or so it becomes fairly stable.

Some Interesting Results

Top 15 hands by game

Hold'em : AA, KK, QQ, JJ, AKs, TT, AKo, AQs, 99, AJs

Omaha hi (all double suited): AAJJ, AATT, AAQQ, AAJT, AA55, AAKK, AA44, AA66, AA99, AA77

Omaha hi/lo: AxAy2x3y, AxAy2x4y, AxAy2x5y, AxAy2x2y, AxAy3x4y, AxAy2z3x, AxAy2x6y, AxAy2x3z, AxAy3x5y, AxAy2z4x

Speaking subjectively, these results look pretty good to me. For hold'em, I might put AQo ahead of AJs. For omaha, the numerical results are very close - this is not surprising given the relatively large set of hands possible. A small bit of random variation could easily have moved some hands up or down a few spots.

Percentage of "good" hands by game after the last iteration

Hold'em: 13% of hands are "good" (last "good" hand is KJo)

Omaha hi: 15% of hands are "good" (last "good" hand is QxQyJz3x)

Omaha hi/lo: 15.5% of hands are "good" (last "good" hand is AxJxTx4x)

Profitability of "best" hand by game

Hold'em: AA rates 2.046

Omaha hi: AAJJ double suited rates 1.568

Omaha hi/lo: AA23 double suited rates 1.571

Limitations of this Approach

As with any approach based on all hands going to river, caution must be used when interpreting the results. Furthermore, as our method uses random results, the generated orderings will be different each time our evolution program is run. In the future we will be performing experiments to see how much variation there is from run to run. We also plan on running the evolution program for games with less than 10 players to see what differences arise.

Mission statement

2006-03-08T10:45:00.000-08:00

Before I jump head first into my first "real" post, I thought it would be good to describe the purpose of this blog. In approximate order of importance, are

Commentary on the continuing development of ProPokerTools, my web-based poker application suite
Examples of how to use ProPokerTools in a blog
Random commentary.

That's pretty much it. Hope you enjoy it!