ICGA/Rybka controversy: An interview with David Levy (1)

Introduction

In June 2011 it was widely reported in the global media that the International Computer Games Association (ICGA), headed by Dr. David Levy, had found chess programmer Vasik Rajlich in breach of the ICGA’s annual World Computer Chess Championship (WCCC) tournament rule related to program originality. In the ICGA’s accompanying report it was asserted that Rajlich’s chess program Rybka contained “plagiarized” code from Fruit, a program authored by Fabien Letouzey of France.

Rajlich, who cruised to victory in four consecutive WCCC tournaments in 2007, 2008, 2009 and 2010, was retroactively stripped of all titles he had won in competition and was slapped with a lifetime ban. He claimed complete innocence and made it clear that he found the ICGA’s investigatory process and conclusions to be biased and unprofessional, and the charges baseless and unworthy. Rajlich refused to be drawn into a protracted dispute with his accusers or mount a comprehensive defence. This was done by Dr Søren Riis, a computer scientist from London, in a paper we published in four parts. There was vigorous feedback from the chess community, including a rebuttal by the ICGA. Links to the relevant articles are given at the bottom of this page.

In his rebuttal David Levy agreed to do a full one-on-one exclusive interview with ChessBase on the entire ICGA/Rybka scandal. It was conducted by sequential emails, and took a fair amount of time, since David was consulting with experts who live in different time zones around the globe.

Interview with David Levy on the ICGA/Rybka scandal

David Levy: At the outset I wish to make it clear that this is not an issue about verbatim copying. We have proven that Rybka is a derivative, but due to different internal board representations the Fruit code required modification when being copied. But a conversion such as mailbox to bitboard still leaves it a derivative, as would changes such as refactoring the code and tuning the numbers.

IM David Levy, the President of the ICGA

The forums continually mention code copying instead of Rajlich failing to report a derivative in his ICGA entry submissions. It is important to remember that the offence for which Rajlich was convicted and banned by the ICGA was failing to comply with our Tournament Rule 2. The various reports (here and here) show that Rybka 1.0 beta and Rybka 2.3.2a were clearly derivatives of Fruit 2.1. Postings on the forums and Riis stating that merely retyping or converting a program to another format are clearly wrong. Merely retyping and/or converting code from one format to another is still making a derivative, as too are some changes that affect game play (such as evaluation tuning). So yes, the Rybka entries were derivative of the Fruit code in a manner that broke Rule 2. And Rajilch says that "Rybka is original at the level of source code" meaning "the author either typed his own code or typed the code which generated his own code", going so far as to say that Rybka derivatives which he had previously censured are "original at the source code level" if they were typed by hand. If this were the case then, if instead of cutting and pasting a programmer just re-types by hand what he has stolen, then according to Rajlich it is original and does not violate ICGA or copyright rules! Really?

Rybka versions from 1.0 Beta through 2.3.2a were shown to be clear derivatives of Fruit 2.1.

Yes, Rybka 2.3.2a definitely competed in ICGA events. The evaluation of Rybka 2.3.2a clearly was derived from Fruit 2.1. Note that the report states "Rybka 2.3.2 was released Jun 13, 2007, and 2.3.2a about five days later. Rybka competed in and won the ICGA Chess Tournament (Jun 11-18 2007)". ICGA analysis of several versions of Rybka between 1.0 beta and 2.3.2a show they also contained a derivative of the Fruit 2.1 evaluation function. During this period a version of Rybka 2.0 competed in the 2006 ICGA World Computer Chess Championship tournament (May 2006) under the name "Rajlich". The closest commercial release was the MP-only Rybka 2.0 Beta 4 with a release date of May 17 (Beta 4).

During the investigation process one member of the ICGA investigation secretariat asked Rajlich "1. Are you able to provide us with the exact .exes that played in all ICGA events particularly those from 2006 and 2007? If not, is it a fair approximation to use Rybka 2.3.2a for the 2007 event?" On the Rybka Forum, Rajlich said "Yes" to the second question, confirming that this version was used in an ICGA event.

Yes, see above. Rybka 2.0 Beta 4 and Rybka 2.3.2a were sold commercially.

The ICGA has not done extensive analysis of newer versions of Rybka. It was sufficient for the ICGA to determine that Rajlich had broken Tournament Rule 2, which we did. When the ICGA chose to ban Rybka for life due to violating the rules in ICGA events in 2006 and 2007, this required all of Rajlich's programs be removed, since the newer programs would never have been allowed to compete if the violation of earlier versions was discovered sooner and the lifetime ban issued earlier. So while it is possible that newer versions of Rybka also contained Fruit-derived code, it was not essential to prove this once we had proved the case for the 2006 and 2007 ICGA tournaments.

The panel did not do much examination as to what made Rybka stronger, so I would suggest you ask Rajlich directly. But experiments with Strelka, which Vas has said is a clone/derivative of Rybka, show gains due to bitboard speedup, 64 bit code being more efficient than 32 bit code, the addition of a form of futility using verification by a qsearch, and tuning the Fruit evaluation. Harder to test is effect of the the Rybka material imbalance tables, but they could have caused some gain as well.

If you compare the 32 bit version of Rybka 1.0 (Elo 2885) to Fruit 2.1 (Elo 2793), the Elo difference is only 92. The speedup going to 64 bit code (which benefits bitboard programs more than mailbox) is about 34 Elo. In versions of Rybka higher than 1.0, there is speculation that more strength advancement came through additional evaluation tuning, and search changes.

The use of Fruit gave Rajlich a huge head start, as he only had to improve a 2793 Elo program instead of writing one from scratch. To get some idea of what small changes to a strong program can do, consider what an amateur programmer did with Fruit in creating Toga II (an admitted Fruit derivative). He was able to add 135 Elo in strength to the Fruit code base, bringing it to an Elo of 2928. The ratings we quote are from are here.

As we have intimated on previous occasions he should have revealed, when submitting his entries, that Rybka was a derivative of Fruit, in order to allow the ICGA to decide whether it should be allowed entry.

One doesn't normally define things so precisely, for the same reason that the United States Constitution is open for interpretation in various places. If you paint yourself into a corner, you are stuck there, but reasonable people can come to a reasonable decision about "when is it too much to allow." And this is really an issue of functionality, rather than counting characters or lines. For example, if you copy a pawn evaluation procedure, that is copying too much. If you find a line of code that does something in a clever way you did not think of yourself, that probably would not even be noticed.

Your question implies that Rajlich might only have copied a small and insignificant amount of Fruit, while the conclusion of the ICGA investigation was that a lot of code was copied. When considering the extent of copying no one on the panel thought that what Rajlich had done was "borderline" in a quantitative sense. The panel actually discussed how much evidence and which of the evidence points would be necessary and sufficient to demonstrate Rajlich's guilt, and looked closely at evaluation feature overlap because this was seen as the most notable and unarguable from the standpoint of Rule 2 (on program originality).

The essence of the case against Rajlich, as we repeatedly explain, is that when entering the World Computer Chess Championship a programming team is obliged to reveal, in their entry submission, if they have used any derived code. The ICGA can then assess, possibly after questioning the programming team and/or after examining the entrant's source code, whether or not the entry should be accepted. That will depend on the nature of what was copied, permissions received from the author of the original program, and whether another program in the same family has applied to enter the tournament.

I would also like to stress, as a separate issue, that a programmer should always comply with the licence of the engine from which he copies the code, as well as declaring it on the ICGA entry form.

One should also mention here that, even if one were to ignore his copying of Crafty code, the most serious allegations of infringements by Rajlich do not start with his 2006 WCCC entry but with his December 2005 Rybka release and its alleged copyright infringement of Fruit. That release is not, in itself, part of the ICGA's grievance against Rajlich, since copyright infringement does not become an ICGA matter unless and until the infringing software is submitted for entry into the WCCC.

That "one line" would not necessarily have solved the problem. Letouzey would also have had to give his permission as well, in order to be listed as an author. If Rajlich had declared Rybka's Fruit origins on the entry form and complied with the Fruit GPL he might well have been allowed to enter with Letouzey's permission. In that case only one Fruit-based engine would have been allowed – the one which had Letouzey's permission and his name in the entry submission.

For whatever has happened in Rajlich's life due to the ICGA's investigation and verdict, he has only himself to blame. If people do not follow rules or laws then they themselves are responsible for the consequences.

Let us now consider what might have followed if Rajlich's entry form had been filled out correctly in the first place. His entry would certainly have been rejected unless he had also obtained Letouzey's permission and put Letouzey's name in his entry submission. What might have happened then? Perhaps Rajlich would have become frustrated trying to write his own evaluation function, in which case he might have approached Letouzey and joined forces with him in some way, or he might have given up trying to create a world beating program and someone else might have become the world beater for the next four years (either starting from Fruit code, with Letouzey's permission, or otherwise). Or Rajlich might have decided after all to develop his own evaluation function, in which case who knows how strong or weak Rybka might have turned out to be?

Some programmers who enter the WCCC use the Nalimov endgame tablebases and declare as such in their entry submissions. Some programmers use openings books developed by others and similarly declare this in their entry submissions. I do not recall any entry submission that admitted to copying an evaluation function as Rajlich did (particularly in his early versions).

At the time of voting the panel was made up as follows: There were three members of the investigation Secretariat; four members of the ICGA board (who did not vote prior to the completion of the report), and the WCCC Tournament Director (who also did not vote prior to the completion of the report), and 34 members of the panel who were invited to vote if they wished. Here is exactly what we asked the panel members under the topic of "Opinion".

So each panel member who responded to the invitation to vote answered two questions. Question (a) was to determine if they had read all the important evidence, so we could judge if the panel member was qualified to answer (b). Question (b) was to get the opinions of panel members as to whether or not Rajlich violated Rule 2. It was not really handled like a jury trial, and the final determination of guilt was up to the ICGA Executive Committee.

Sixteen panel members voted, all of whom answered "yes" to both questions. The ICGA would not divulge the names of those who voted (other than Ken Thompson) without first asking their permission and we do not intend to do so.

We decided during the investigation process on confidentiality with respect to the names of those who voted. The Secretariat comprised: Mark Lefler, Robert Hyatt and Harvey Williamson. They administered the investigation and they wrote the report. Incidentally, I must re-iterate that the report was written after the panel members voted, whereas Søren Riis incorrectly states on Chessbase.com that "It really goes without saying that the panel members voted on the basis of the findings of the ICGA report." He shouldn't have said that, given that it is false.

The purpose of the voting by the panel was to assess the weight of opinion of those panel members who wanted to make their opinions known, so that the secretariat could take that into account when preparing their report on the investigation and so that the ICGA office bearers could take it into account when deciding on their verdict. After the ICGA office bearers studied the report and the evidence submitted as part of the report, we voted on whether to find Rajlich innocent or guilty of breaking the ICGA's Tournament Rule 2. There was unanimous agreement on a guilty decision.

The punishment was determined by the Executive Committee of the ICGA (excluding the Vice-President for Asia): David Levy, Yngvi Bjornsson, Hiroyuki Iida, Rémi Coulom and Jaap van den Herik, who was the Tournament Director of all the tournaments under discussion. From the outset there was no doubt about disqualification and cancellation of World Champion titles, or on the fact that a ban from participating in future ICGA events was called for. The only matter on which there was any debate was the duration of the ban. Some ICGA office bearers felt that ten years was the appropriate period, while others favoured a life ban. By a majority decision the ICGA decided to make it a life ban.

As to the procedure to decide the degree of punishment – this comes under Article III Section 1 of the ICGA Constitution, which is always published in the ICGA Journal prior to the tri-ennial meeting at which ICGA office bearers are elected. Section 1 states that the Executive Committee of the ICGA is charged with the administrative affairs of the association.

We decided at the start of the process that we would not reveal the names of those who voted, so we did not know at that time who would vote and which way they would vote.

The suggestion that some of those who voted may have had a vested interest in a "guilty" verdict overlooks one very important and indisputable fact. The world of elite chess programmers is a very small one, and these are the people who are best equipped to judge the issues involved in the investigation. So Rajlich's actions were being assessed by his peers. If the ICGA had excluded from the panel anyone who is an elite chess programmer we would have been significantly diminishing the overall quality of the investigation. Remember also that I, as President of the ICGA, and the other Board members who I have listed above, were able to see not only all the evidence and all the votes of the panel, but also we were able to follow the discussion on the closed Wiki that was set up for the benefit of panel members. If any of us had seen any comments that led us to believe that any unfair bias existed within the panel's deliberations, we would have had the right and the duty to say so at the time.

Finally, on this point, I would like to add that I have always enjoyed a courteous relationship with Vasik Raljich, and until the evidence leading to this investigation was presented to me I had absolutely nothing against him. In fact, when I was first told the rumour (as it then was) that he had used code derived from Fruit, I did not believe it, largely because I do not regard the person who told me as a credible source. Furthermore, if I had had a choice of the verdict (in the absence of the evidence) I would very much have preferred it to have been a "Not guilty" one, as the investigation and its aftermath could do absolutely no good for the image of computer chess. The only good that has come out of the whole affair so far as I am concerned, is that programmers who are tempted in the future to break the ICGA's tournament rules in a serious way will now be less likely to do so.

– Part two will follow shortly –
Please do not write in commenting on this interview until part two has been published!

Previous articles on the ICGA/Rybka scandal

Feedback on the ICGA/Rybka disqualification scandal 13.01.2012 – It's a long read, but many of the letters we received in reaction to the defence of the Rybka program by Dr Søren Riis are quite passionate and well thought out in their content. We start with the summary of a long rebuttal of the Riis paper that was sent to us by the ICGA and circulated on the Internet – with links to the full version and ancillary documents. Take a deep breath.

A Gross Miscarriage of Justice in Computer Chess (part four) 05.01.2012 – As proof that Vas Rajlich had copied program code the ICGA presented pages and pages of Fruit and Rybka code side by side. But, according to Dr Søren Riis, what was labeled as Rybka code was actually fabricated to look like Fruit. He criticizes the ICGA's process failures and ruminates on the reasons behind the unprecedented vendetta launched against the star chess programmer.

A Gross Miscarriage of Justice in Computer Chess (part three) 04.01.2012 – A core accusation against Vas Rajlich is that Rybka and Fruit have very similar positional evaluations, and the use of floating point numbers in Rybka’s time management code had to be copied from Fruit. Søren Riis enumerates the ten substantive evaluation differences and shows how the second accusation boils down to a single misplaced keystroke with zero impact on Rybka's play.

A Gross Miscarriage of Justice in Computer Chess (part two) 03.01.2012 – In this part Dr Søren Riis of Queen Mary University in London shows how most programs (legally) profited from Fruit, and subsequently much more so from the (illegally) reverse engineered Rybka. Yet it is Vasik Rajlich who was investigated, found guilty of plagiarism, banned for life, stripped of his titles, and vilified in the international press – for a five-year-old alleged tournament rule violation. Ironic.

A Gross Miscarriage of Justice in Computer Chess (part one) 02.01.2012 – "Biggest Sporting Scandal since Ben Johnson" and "Czech Mate, Mr. Cheat" – these were headlines in newspapers around the world six months ago. The International Computer Games Association had disqualified star programmer Vasik Rajlich for plagiarism, retroactively stripped him of all titles, and banned him for life. Søren Riis, a computer scientist from London, has investigated the scandal.