Can you teach a Large Language Model such as GPT to play chess? That was the core question when Jerome Maag and Lars Schmid, two computer science students, started their project at the Centre for Artificial Intelligence (CAI).

Over one year, the two students worked on machine learning methods to train an "empty" GPT-2 model (i.e., one not pretrained on any language data) with chess data. To this end, they downloaded millions of chess games from an online platform and trained the model to generate the next move, based on the sequence of all previous moves. 

After hundreds of hours of GPU training, the idea paid off: the model learned to generate valid chess moves! This proved that Large Language Models (LLMs) can learn the "language of chess" without having ever seen any rules or explanations of how chess is played.

Together with their supervisors, Mark Cieliebak and Pius von Däniken, the two students submitted their work to the Swiss Text Analytics Conference (SwissText). The submission was accepted as a poster presentation at the conference in June in Chur. But even more, they also received the Best Poster Award of SwissText 2024 for their work! and Lars Schmid, two computer science students, started their project at the Centre for Artificial Intelligence (CAI).