Rybka's Monte Carlo analysis

Mind you, I'm not trying to make anyone feel badly or anything, but more than one of these e-mails say that the feature is "underdocumented". I'll tell you straight away, there's not a lot here which requires a boatload of documentation. Basically, here's what "Monte Carlo analysis" is: you start with a game position, activate the feature, and then the Rybka engine plays a whole lot of games against itself (starting from that position), but instead of storing the games in a database (as the "Shootout" feature does), it stores all of the moves of those games in tree form.

That's it. Think of it as being similar to the Shootout feature (with which you're hopefully already familiar) with a couple of main differences: the results are stored as a tree, and the engine keeps playing games against itself endlessly until you stop it (instead of Shootout's pre-programmed "cap" on the number of games played).

Let's see this analysis mode in action to get a better grasp of what it does. We'll start with a position from a game played in 1901 in -- you guessed it -- Monte Carlo. My "old friend" David Janowski (who I wrote about at length in one of my e-books) has the Black pieces against M. Didier. White has just played 14.c3 to reach the following position:

We should keep in mind here that this game takes place in 1901. These were the days when Janowski was a feared and respected chess competitor, very unlike his later gambling-addicted dissolute self of the mid-1920's. What does David Janowski do in this position?

I'll admit that as I replayed this game, I never saw this coming. Instead of living with the two White Knights supporting each other, Janowski sacrifices the exchange by playing 14...Rxb6 15.Nxb6 Qxb6 16.cxd4 exd4. My curiosity piqued, I decided to see how a Rybka Monte Carlo analysis would turn out starting from the position after 14.c3.

To start a Monte Carlo analysis from a particular position, go to the Tools menu, select "Analysis", and then "Monte Carlo" from the submenu (note that this function will be in grey half-tone and unavailable unless you've already loaded a Rybka engine). The next thing you know, something startling happens: the game in your Notation pane disappears, to be replaced by the "Load book" button. Superimposed over all is the following dialogue:

You have two types of settings in this dialogue. The first is the search depth, with a default of "5". This controls how far ahead (in half-moves, or "plies") the engine will look before making a move. For example, if you leave this at "5", the engine will look 2.5 moves ahead before making a move. Remember, the engine is going to be playing a lot of games against itself and storing the moves in the form of a tree, so the search depth is important. You must realize, however, that there's something of a tradeoff here; the higher you set the search depth, the more time the engine will need to make each move -- so you're trading time for depth. On the other hand, setting a lower search depth means that many more games will be played in a given amount of time, but that the moves themselves are likely to be more superficial.

Keep in mind, too, that you should use only odd numbers for the search depth, because chess engines tend to develop a tactical "blind spot" when made to analyze at even ply depths. Rule of thumb: odd numbers good, even numbers bad.

The second setting is the "width" of the tree. This is similar in some ways to the "Branching factor" in Deep Position Analysis and is another "space for time" tradeoff. If you create a "Narrow" tree, you won't see many alternative moves displayed in your game tree but the overall process of playing games and generating the tree will be faster. "Broad" trees show more alternatives but take longer to generate (it requires more processor time and thus slows down the chess engine).

Let's go for a "Medium" tree and a search depth of "7", click "OK" and see what happens next.

The software will load the engine and you'll get a dialogue to that effect. Then -- nothing? Patience, please. For a minute or so it'll look like nothing's going on, but then you'll see the tree view (your former Notation pane) become populated with data:

This looks a lot like the game tree/opening book display with which you're already acquainted. This one acts a bit differently in the data it displays. You still have "N" for the number of games played (ten so far when I took this screen shot) and a numerical percentage of how well the move did from White's perspective; we're starting to see that Janowski's exchange sac was probably unsound. "Prob" is the probability of a move being played; since 14...Rxb6 showed up in every game Rybka played against itself so far, the value here is "100%".

Note that you can actually play through the moves of the tree while it's still being generated, too. Use the cursor keys on your keyboard to navigate through the moves (left and right to move forward and backward, up and down to select a move when multiple moves are displayed).

The bar near the top of the display also provides useful information:

This display reminds us of the engine which is being used for the analysis and the search depth we set at the start of the process. We see how many games have been played so far (54 at the time I made the screen capture) and the results of those games. In this case, White won twenty-five of them while Black won just three; twenty-six of the games were drawn. The next value is the success rate, given from White's perspective (70.4%). "TP" is a "performance rating" value, again presented from White's viewpoint. And I'll confess that the remaining values have me stumped so far.

You can let Monte Carlo analysis run for as long as you like. Remember that the more games you let the engine play, the more reliable will be the statistical results.

Let's take a last look at the statistical Monte Carlo analysis for our game Didier-Janowski after White's 14.c3:

What we're seeing is that after 100 Rybka-generated games Janowski's exchange sac was really the only move that makes sense (100% Prob value and no other moves showing in the tree). But it's not a great move: White scores 68% from the position after 14.Rxb6 (getting just over 2/3rds of the available points, with wins being a point and draws being a half-point, of course). In fact, Black won just seven out of 100 games played from this position.

What Rybka's Monte Carlo analysis is telling us is that 14.Rxb6 is a crappy move, but that Janowski was practically forced to play it since his alternatives were actually worse (so Janowski essentially painted himself into a corner somewhere along the way prior to this point). But the interesting thing is that Janowski went on to win this game; you can do a database search to find it and play out the rest of the moves if you like.

Back to Rybka: to stop the Monte Carlo analysis, click the "Stop" button (the little red button with the white "x" in the upper left corner of the Monte Carlo analysis screen). You'll see a popup dialogue asking if you wish to save the tree which the analysis generated. If you click "Yes", you'll get the standard Windows dialogue prompting you to pick a folder in which to store it and for you to give it a name. Note that all of the game moves prior to the point at which you started the analysis are also saved as part of the tree; in our example, all of the moves through 14.c3 will be saved along with the "Monte Carlo tree" which begins with Black's 14th move.

If you select "No", you'll simply be returned to the normal game window (board and Notation pane), but you'll notice that any moves you replayed in the tree while the analysis was being generated will be displayed as variations/subvariations in the Notation pane's gamescore.

All of that seems prety verbose, but it's really very simple: Monte Carlo analysis causes Rybka to play a large number of games against itself and then store the results in the form of a statistical tree. That's really not hard to understand at all.

Until next week, have fun!

© 2008, Steven A. Lopez. All rights reserved.