Equity Squared

Until recently, there have been two flavors of hand rankings provided by ProPokerTools. The first kind is created using a  program (described in previous posts) to 'evolve' an ordering over time. The second kind is simply all-in equity versus a random hand. Equity-squared is a third way to order hands, one that has been used extensively by the poker artificial intelligence community.

The easiest way to see how equity-squared works is to compare it to all-in equity.  One way to compute all-in equity vs. a random hand is to generate all possible boards, count our wins and ties vs. every possible opposing hand, and divide by the total number of matchups. This gives us a number between 0 and 1, where 0 always loses and 1 always wins. For example, in the case of AsTd, the answer turns out to be 0.62722, which means it has 62.722% all-in equity.

When we compute equity-squared, we again generate all possible boards. For each board, we compute our equity vs. every possible opposing hand. Then, we square the result (hence the name 'equity squared'). Our total equity squared is the average of all the equity-squared values for every board.

Why is this interesting, you may ask? The answer has to do with the power of strong draws. Imagine a theoretical hand that will make the nuts 50% of the time but make essentially nothing the other 50% of the time. Now imagine a second hand that always ends with exactly average  equity. It should be clear that while both hands have the same average all-in equity, they have a very different quality; the first hand is a very strong draw, while the second is an average value hand. 

The effect of using equity-squared is to increase the value of hands that can make strong draws. For a hand like JsTs, it has a good all-in equity rank, placing 24th out of 100. But using equity squared, its place moves up to 15. Conversely, a hand like As8h has a significantly better all-in equity rank, scoring a 16. But it doesn't make a very strong hand nearly as often as the JsTs, so its equity-squared rank is only 19. 

You can view the complete equity-squared ranking for hold'em here: http://propokertools.com/orderings/heequitysquaredordering.txt

6-Handed Hand Orderings

In a previous blog post, I explained the algorithm used to generate hand-orderings. ProPokerTools has had hand orderings for hold'em, omaha, omaha-8, big-O, and big-O8 for quite some time now. The existing orderings were generated for a full-ring game; 9 players for the big-O games, and 10 players for the others. Now I would like to share some results from my latest hand-ordering adventure.

Are you ready for a full-out nerd-out data-dump? I thought so. Let's begin.

6-Handed vs. Full Ring.

I have just finished generating hand-orderings for 6-handed versions of five flop games. The results are below. "Distance" refers to how far a particular hand is in one ordering vs. the other. The distance is given both in the number of positions and a percentage of the entire ordering.

game	average distance	median distance	max distance	moved more than 5%
holdem	3.98 (2.35%)	3 (1.78%)	16 (9.57%)	22 (13.02%)
omaha	540.46 (3.29%)	362 (2.20%)	5050 (30.73%)	3730 (22.70%)
omaha-8	461.35 (2.81%)	319 (1.94%)	3361 (20.45%)	3008 (18.31%)
big-O	3975.40 (2.96%)	2883 (2.14%)	33860 (25.18%)	26436 (19.66%)
big-O8	3642.96 (2.71%)	2409 (1.79%)	36030 (26.71%)	22938 (17.06%)

We can see a couple of obvious patterns. On average, most hands do not move up or down more than a couple of percentage points. However, in all cases there are at least 10% of hands that move five percent or more.

Hold'em 6-handed vs. 10-handed

Let's take a closer look at the differences for 6-handed hold'em vs. 10-handed holdem since we only have 169 hand classes to deal with. Here is a listing of all the changes between the two orderings going from 10-handed to 6-handed, where a positive number is a promotion and a negative number is a demotion. I have only listed hands with a difference of eight places or more to reduce the noise. Suited hands are in parentheses.

• A9 10
• A8 8
• (98) -8
• A5 8
• (87) -10
• A6 15
• A3 10
• A2 9
• Q8 10
• (54) -16
• (64) -12
• (T4) -10
• Q7 9
• J7 9
• (53) -12
• Q6 14
• K2 8
• T7 11
• (43) -15
• (63) -11
• Q5 9
• Q4 9
• J6 11
• T6 12
• (42) -10
• Q2 9
• J3 8

Two patterns jump out; high-card strength is more important 6-handed, and suitedness and connectedness is less valuable 6-handed. Both of these patterns match most players' intuitions on hand strengths, so I'm very pleased with this result.

Run-to-Run Consistency

The algorithm used to generated hand-orderings is a stochastic process - many random trials are used to create a reasonably stable ordering. I decided to explore how much random noise is involved by generating the 6-handed hold'em ordering a total of ten times. The table below shows the difference between the first run and the other nine runs.

run	average distance	median distance	max distance	moved more than 5%
1	0.93 (0.55%)	1 (0.59%)	6 (3.55%)	0 (0%)
2	0.88 (0.52%)	1 (0.59%)	6 (3.55%)	0 (0%)
3	1.02 (0.60%)	1 (0.59%)	8 (4.73%)	0 (0%)
4	1.03 (0.61%)	1 (0.59%)	9 (5.33%)	1 (0.59%)
5	0.89 (0.53%)	1 (0.59%)	9 (5.33%)	1 (0.59%)
6	0.95 (0.56%)	1 (0.59%)	6 (3.55%)	0 (0%)
7	0.96 (0.57%)	1 (0.59%)	7 (4.14%)	0 (0%)
8	1.04 (0.62%)	1 (0.59%)	8 (4.73%)	0 (0%)
9	0.98 (0.58%)	1 (0.59%)	6 (3.55%)	0 (0%)

As expected, there is some noise from run to run. But in all cases there was at most a single hand that moved more than 5%. This reinforces my oft-repeated claim that the hand orderings are useful as a rough guide, but shouldn't be trusted at too fine a level of detail.

Generations

The hand-ordering algorithm repeats the same process over a number of generations, with each generation becoming more 'accurate' with a doubling of the number of random trials. The idea is that after a while the ordering becomes relatively stable. Below is a listing of each generation in the 6-handed omaha ordering and how it compares to the prior generation.

generation	average distance	median distance	max distance	moved more than 5%
1 (first)	n/a	n/a	n/a	n/a
2	1930.77 (11.75%)	1439 (8.76%)	13311 (81.00%)	10998 (66.93%)
3	1417.24 (8.62%)	1033 (6.29%)	9779 (59.51%)	9404 (57.23%)
4	1029.30 (6.26%)	737 (4.49%)	9423 (56.35%)	7653 (46.57%)
5	740.68 (4.51%)	530 (3.23%)	5704 (34.71%)	5802 (35.31%)
6	524.36 (3.19%)	372 (2.26%)	4288 (26.10%)	3630 (22.09%)
7	372.15 (2.26%)	267 (1.62%)	2795 (17.01%)	1882 (11.45%)
8	260.11 (1.58%)	185 (1.13%)	1983 (12.07%)	647 (3.94%)
9	185.86 (1.13%)	131 (0.80%)	1213 (7.38%)	119 (0.72%)
10 (final)	132.76 (0.81%)	94 (0.57%)	1193 (7.26%)	8 (0.05%)

It is clear from the data that 10 generations gives a relatively high level of stability, with only a tiny fraction of hands moving more than 5%. Hold'em, omaha, and omaha-8 orderings all ran for a full 10 generations.

The 5-card omaha variants ran only for 6 generations [UPDATE 6/12/2012 - 8 generations]  due to the increased computational load (and my desire to get my computer back after the weekend), hence they are inherently less accurate than their brethren. However, this is probably mitigated by the very large number of hand classes in the 5-card variants (134,459 to be exact) which should cause some of the inaccuracies to average out (for example, if JT(987) is rated a bit high, perhaps hand classes such as JT(98)7, J(T98)7 and (JT9)87 will average it out).

How Many Hands In Common?

The table below shows the number of hands in common between the 6-handed and full-ring orderings for several hand ranges.

game	top 10%	11%-20%	21%-30%	31%-50%	51%-100%
holdem	90.91%	81.82%	84.38%	89.47%	98.20%
omaha	92.06%	77.63%	68.80%	78.70%	94.92%
omaha-8	94.01%	87.20%	79.48%	82.67%	95.81%
big-O	92.57%	79.15%	69.34%	80.76%	95.79%
big-O8	94.92%	87.57%	80.45%	83.41%	95.80%

The data shows that premium hands in one ordering are almost always premium hands in the other. The biggest differences appear in the medium-strength hands, those in approximately the top 15%-35%.

Some All-in Equity Results

The data below shows hand race results for each game. Each entry lists the full-ring equity and then the 6-handed equity. Equities are for the top 10% hand.

game	10% vs. 11%-20%	10% vs. 21%-35%	10% vs. 11%-20% vs. 21%-35%
holdem	63.95%/65.10%	66.55%/67.02%	47.37%/47.90%
omaha	58.47%/59.18%	60.25%/60.84%	40.84%/41.40%
omaha-8	55.23%/55.37%	57.71%/57.95%	39.37%/39.34%
big-O	55.91%/56.06%	57.36%/57.58%	38.63%/38.89%
big-O8	54.58%/54.66%	57.59%/57.68%	38.99%/38.96%

We can see that in all cases but one, the equity for the top 10% hand increases under the 6-handed ordering, although the improvement is never very significant.

Ordering Files

Below are the data files for the new 6-handed hand orderings:

• hold'em 6-handed
• omaha 6-handed
• omaha hi/lo 6-handed
• Big-O 6-handed
• Big-O hi/lo 6-handed

Data Overload

I think that's enough data for now. If there are some more data points you would like me to collect, feel free to comment here and I'll see what I can do.

One Year of the Odds Oracle!

Summing it Up

It has been just about a year since the first closed beta of the Odds Oracle 0.1 came out. As of the writing of this article, we are on version 1.8.5 and just on the cusp of 2.0. User feedback has been overwhelmingly positive, and from a business perspective things are going reasonably well. This couldn't have been possible without the constant support, suggestions, questions and encouragement of Odds Oracle users - I thank you all! Below is a summary of what we have achieved together, and what I feel are the good, great, and not-so-great parts of the Odds Oracle as it stands now.

The Good

• "Show PQL" buttons everywhere help users to make the leap from the user-interface to custom PQL queries.
• Multiple input/output sources (hand history importer, command line, spreadsheets) have made it more convenient to run simulations.
• Saved ranges/macros save time for users with complicated pre-constructed ranges
• Multi-core support has been a (ahem) core design principle from the get-go. This has made it relatively easy for the Odds Oracle to take full advantage of the latest hardware.
• Hand vs. Range equity graphs. These are what people usually want when they graph equities - this is now the default graphing algorithm.
• Interactive hand stats on flop/turn - while not pretty to look at, this allows slicing-and-dicing of hand types in a way that would be too cumbersome otherwise.

The Great

• "Ask the Oracle". Being able to ask probability questions using pull-down menus is awesome.
• PQL was a big gamble, and it has paid off. PQL powers almost everything in the Odds Oracle and the website. As a result, there is a fairly straightforward correspondence between elements in the UI and generated PQL queries (this is something that will be made more explicit in 2.0)
• Generic range syntax has turned out to be a big winner. Having one syntax available for everything has proven a boon, especially for people who play multiple games.
• The core re-write of December, 2010 was definitely worth it. I've learned a lot since propokertools.com opened its doors several years ago, and starting over with this accumulated knowledge has made the code-base smaller, faster, and easier to work with. For users, that means new features can come out faster.

The Not So Great

The focus on the 0.x and 1.x series of the Odds Oracle has been to create an ever-increasing set of questions that can be answered. As a result, not enough time and effort has been spent on creating a better user experience. Specifically:

• The range pickers are rudimentary at best.
• The feature set is too spread out over different windows (range pickers, hand explorers, main window, equity/graph/ask the oracle/common questions buttons...). As a result, the common denominator of most of the Odds Oracle (that would be PQL) is somewhat obscured.
• Ask the Oracle is too limited in its current form. Ideally, there should be a 1:1 correspondence between PQL and the UI - any PQL query you can write you should be able to construct in the UI (no, the PQL runner and CLI don't count!)

What's in Store for 2.0

My goals for 2.0 in priority order are:

• Create a streamlined UI, allowing users to do most of what they want in a single window without the need to write custom PQL.
• Create best-in-class range pickers.
• Continue to increase the power of PQL by adding more core functions and features.
• Spend some time working on marketing.
• Create more instructional materials (videos, docs, ebooks?)
• Make the UI look prettier.

If you have comments, questions, complaints, burning feature requests, or just want to shoot the breeze, please feel free to leave a comment here or on the free ProPokerTools Forum.

Here's to a productive 2012!

- Dan @ ProPokerTools

AK vs half-dead pair (User Request)

I got a fun question from a user today. "How does AK (suited or unsuited) fare against any pair assuming that one of the pair's set outs are dead?"

We can answer this with a little PQL hackery. I say "hackery" because there isn't a way to connect dead cards with other hand ranges in PQL at the moment. We will perform an all in equity simulation with AK vs any pair. If the pair gets quads by the river, we throw out the sim, since one of the set cards is supposed to be dead. If the pair does not get a matching card by the river, we keep the sim, since the dead card never appeared. If the pair hits exactly one of its set cards but doesn't have quads, we have a 50/50 chance that the sim should be thrown out. Here's where the hack comes in - we can just throw out all sims in this case where one of the pair cards is a heart, as 50% of pairs have a heart in them. Note that if the pair is Aces or Kings, then we should not be able to hit the board at all, so we need to exclude that as well.

Here is the PQL query I came up with:

select avg(riverEquity(hero))
from game='holdem', hero='AK', villain='AA-22', syntax='Generic'
where
case handBoardIntersections(villain, river)
  when 0 then true
  when 2 then inRange(villain,'*!h!AA-KK')
end
and not (handtype(villain, river) = quads)

Answer - AK has 48% equity.

Whence PQL?

"A-ha!" vs. "Ho-hum"

In the popular imagination, innovation comes from a flash of genius - a sudden, almost supernatural insight that often catches the inventor himself unawares. In reality, most innovation is far more mundane - small, incremental (and often obvious) improvements combine to make something new possible. Indeed, almost everything "new" on propokertools.com has found its genesis in incremental improvements - new games, graphs, and tools that build on what came before. After all, the very first version of propokertools.com was an answer to the question "What would an online version of PokerStove look like?"

PQL is the exception - it is the closest I've come to a "Eureka" moment yet.

Three as One

Here are three questions that have been answerable at one time or another via propokertools:

• Given that player one has XX, player two has YY, and the board is ZZZ, what is the all-in equity for player one?
• Given that player one has XX, player two has YY, and the board is ZZZ, how often does player one have at least two pair by the river?
• Given that player one has XX, player two has YY, and the board is ZZZ, what does the graph of equities look like for player one?

Notice anything? These questions all have the same setup, namely "Given that p1 has ..., p2 has...". Let's call this the situation. After each situation, there is a question, which we can refer to as, well, the question. The three questions can all be rewritten as:

• Given situation S, what is the answer to question Q?

In programmer-speak, we can say that we have abstracted out the concept of a situation and a question. This process of abstraction is one of the principal (some would say the principal) jobs of a programmer. And there is nothing particularly noteworthy about the abstraction above - any programmer worth her salt would spot it almost immediately.

Exposing the Abstraction

I spent a few weeks rewriting propokertools in December of 2010 (yes, all of propokertools (except for razz) got rewritten). One of the first things I did after throwing out several years' worth of accumulated cruft was to see how easy it would be to answer new poker questions with my re-written code. I began to write small programs to answer questions like "Given situation S, how often does player one win when he was an underdog before the flop?" and "Given situation R, how often is the winning hand a straight or better?". These programs were, for the most part, satisfyingly short and sweet. At first, I thought I would expose these little mini-programs as individual tools on the website. Then it hit me.

Why not let the users ask the questions?

Enter the DSL

"DSL" stands for "domain-specific language". A DSL is a language, usually small, used to describe a very particular area, or domain. For a somewhat contrived example of a DSL in action, one need only step up to the counter at a popular chain of coffee-shops during the morning rush. "Half-caf grande iced americano" means "using half regular beans and half decaffeinated beans, brew an espresso and dump it in a large to-go container, adding ice cubes and cold water." The key behind a successful DSL is that it is concise and unambiguous. An even more concise version of our coffee DSL can be seen behind the counter, as the people at the cash-registers write only a few characters of text on each coffee cup to completely describe what is to be brewed.

Another DSL you may be familiar with is the ProPokerTools "range of hands" DSL. For instance, in omaha, "AA**" means a hand with at least two aces in it.


Why Invent What You Can Steal?

To allow users to ask custom questions, I needed to create a new DSL. I had a number of requirements:

• Need to be able to ask multiple questions
• Need to be able to compose questions using boolean operators (such as AND, OR, and NOT)
  
• Need to be able to fully describe the situation
• Need a semantics that is easy to explain

As it happens, there is a language that has been used for decades in another domain that provides pretty much all of what we want. That language is SQL, or Structured Query Language. SQL is the industry-standard way to query databases. While the correspondence between SQL and our poker situation is not perfect, it is good enough, and given the availability of free tools around SQL, it became the obvious choice. Thus was born PQL - Poker Query Language.

Eating Your Own Dog Food

The most obvious way to make sure PQL was truly useful was to use it myself. As a result, all of the major features on the website and in the downloadable Odds Oracle are powered by PQL queries.

Downloadable ProPokerTools 0.1 is done!

I promised myself I would release a downloadable version of ProPokerTools in 2010. The bad news? I did not reach my goal. The good news? I am only three days late. A random pool of users have been selected from those who submitted a survey from the propokertools website. Check your inbox!

Generating all 3-way holdem matchups

By user request, I have generated equities for all 3-way races between holdem hand types (ex 22 vs. AJs vs KQo) and stored the values in a file. My new quad-core laptop is still catching its breath from the effort. If you would like the results (warning - 15+Meg file), please email propokertools support and I'll send it your way.