Recognition of a chess position from images during a chess game

VICHOROSKI, HRISTIJAN (author), Žabkar, Jure (mentor)

Chess recognition refers to the process of determining the configuration of chess pieces from an image of a physical chessboard. This task becomes challenging due to variations in camera angles, lighting conditions, and background clutter. These factors make it difficult for traditional computer vision methods to consistently extract an accurate game state. In this thesis, we propose a chess recognition pipeline, addressing the mentioned challenges and creating a robust solution that determines the game state from a player's point of view. The proposed pipeline consists of three main components. First, the YOLOv5 object detection algorithm is used to detect and classify chess pieces, coupled with a color detection algorithm to differentiate between white and black pieces. Second, we created a model to locate the board by detecting coordinates along its sides, using the U-net segmentation architecture as a base. Finally, the outputs of both components are combined to extract FEN notation and Stockfish was integrated to recommend the next move. The results demonstrate the pipeline’s robustness, showcasing strong performance even on previously unseen data. YOLOv5, combined with color detection, effectively identified chess pieces, while the modified U-Net model proved to be a reliable solution for chessboard detection across various challenging scenarios. These findings highlight the effectiveness of the proposed approach in real-world applications of chess game analysis and automation.

• Language: English
• Keywords: chess recognition, computer vision, deep learning, yolo, u-net
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2025
• PID: 20.500.12556/RUL-167472
• COBISS.SI-ID: 227808771
• Publication date in RUL: 24.02.2025

VICHOROSKI, HRISTIJAN, 2025, Recognition of a chess position from images during a chess game [online]. Bachelor’s thesis. [Accessed 19 May 2025]. Retrieved from: https://repozitorij.uni-lj.si/IzpisGradiva.php?lang=eng&id=167472

Secondary language

• Language: Slovenian
• Title: Prepoznavanje šahovske pozicije iz slik med igranjem šaha

Abstract

Prepoznavanje šaha se nanaša na proces določanja postavitve šahovskih figur iz slike fizične šahovnice. Ta naloga postane zahtevna zaradi variacij v kotih kamere, svetlobnih pogojev in nereda v ozadju. Ti dejavniki otežujejo uporabo tradicionalnih metod računalniškega vida za dosledno pridobivanje natančnega stanja igre. V tej diplomski nalogi predlagamo sistem za prepoznavanje, ki naslovi zgoraj omenjene izzive in ustvari robustno rešitev za določanje stanja igre z vidika igralca. Predlagani sistem je sestavljen iz treh glavnih komponent. Algoritem YOLOv5 za detekcijo objektov se uporablja za zaznavo in klasifikacijo šahovskih figur, skupaj z algoritmom za prepoznavanje barve, ki omogoča razlikovanje med belimi in črnimi figurami. Ustvarili smo model za lociranje šahovnice z detekcijo koordinat njenih stranic, pri čemer smo kot osnovo uporabili segmentacijsko arhitekturo U-Net. Rezultati obeh komponent se združijo za tvorjenje FEN notacije, pri čemer je v sistem integriran tudi program Stockfish, ki priporoča naslednjo potezo. Rezultati kažejo na robustnost predlaganega sistema, ki prikazuje dobre rezultate tudi na prej nevidenih podatkih. YOLOv5 v kombinaciji s prepoznavanjem barve učinkovito prepozna šahovske figure, medtem ko se je modificirani model U-Net izkazal za zanesljivo rešitev pri detekciji šahovnice v različnih zahtevnih scenarijih. Ti rezultati poudarjajo učinkovitost predlaganega pristopa v praktičnih aplikacijah analize in avtomatizacije šahovskih iger.

• Keywords: prepoznavanje šaha, računalniški vid, globoko učenje, yolo, u-net