Rybka 3 vs Firebird 1.2 (Long Time Controls)

Anyone who follows computer chess will know that Rybka 3 has been the top computer chess program for a long time now.  Yet Rybka is not without faults in its playing style and move selection.  In short, it remains beatable. What is less well known by the general chess playing public is that many months ago another program was released that could beat Rybka, or so it was claimed.

Most of these “tests” were actually played a Blitz chess speeds and favoured a stripped down chess engine.  There was clearly a need to test both pieces of chess software at a more realistic time control in order to establish a true playing strength.  While Elo ratings for chess engines are somewhat meaningless because they are not calibrated against established human players it is interesting to note that many agree that Rybka is well over a 3k rating and Firebirds developer claims a 3,400 Elo rating for his engine. It seems a good idea for computer chess online to investigate.

The Robbolito Cloning Controversy

The program that could beat Rybka has had many names and forks in its development.  One of the early names the program went by was RobboLito.  Many have accused RobboLito of being a cloned copy of Rybka itself and have refused to test its actual playing strength.  The program remains banned from use on many of the internet chess servers because of these accusations.

At one point the censorship about the program was so severe that it was impossible to make a post on many of the internet chess forums without having the post deleted if it mentioned the name of the software at all.  Thus the program became known as “”The Engine that Shall Not Be Named”.

This kind of Stalinist repression really turned me off frequenting those forums and eventually drove me to set up this website as I feel that the best service of computer chess fans is served by free information exchange and allowing adults to make up their own minds on an issue.

It should be pointed out that, despite the fact that accusations were made against Robbolito and its developers, to date, no case has ever been brought against them.  It is my opinion that if the law has been broken then of course what Robbolito’s developers did was wrong, but as there is presently zero evidence in the public domain to support this and there has never been any prosecution against them I have to assume that they are innocent.  This is a fundamental principle of Anglo-Saxon law and I agree with it – innocence until proven guilty.

Perhaps the only real clone is Dolly the Sheep?

Firebird – Rising From the Ashes

Robbolito’s development was extremely rapid, with a new beta release almost weekly.  Over time this hectic development slowed and later it became apparent to external observers that the software developers had gone on separate paths and the engine’s development had forked.

One of those forks is Firebird, which when I tested it had a public release number of 1.2.

One of the accusations thrown at Robbolito and Firebird both is that they are essentially the same program, but with much of the error checking code stripped out to make them “think” faster.  The argument goes that at short time controls this makes them stronger than Rybka, but at longer time controls Rybka should be equal or maybe stronger as it would make fewer mistakes.

Of course with many in the testing community refusing to test the engine it is impossible to know the answer to these kinds of questions without doing the research yourself.  Thankfully I have a spare dual core Athlon PC and decided to make use of it to try and get to the truth.

Firebird 1.2 versus Rybka 3 Test Results

To test the two programs I gave them both the same narrow book from Aquarium 2010 and set the UI to make the engines play the same opening as black and as white.  My goal here was to remove any book or colour advantage as the object was to test the engines relative to each other and not to find a champion.  The time controls used were 40 moves in 2 hours, recurring.  This means that even this modest list of games took several days to complete.  I would have left it to run until at least one hundred games were completed, but the computer chess world is moving quickly recently and I terminated the test to start another as a different program has recently surfaced that claims to be able to beat Rybka.

The Final Result?

I am sorry to report that the final result was something of an anti-climax.  After 32 games played the result was a dead draw with 16 points apiece, although the games won and lost were different in many cases.

As the sample size is so small it is hard to say anything conclusive about this.  The difference in choice of moves may just be due to small randomising factors within the engines themselves, or it may be evidence that the programs do indeed “think” differently. Whatever the truth, this is my contribution to the debate.

For those interested in the test and the moves played, the PGN of the games is available below.

[download id=”1″]

PGN games download.

Resources

Rybka Homepage

Firebird Homepage

(Firebird has recently been renamed “Fire” and so the original link changed.  To save further confusion the link goes to the homepage of RobboLito and Firebird both.)