ChatGPT performs poorly at chess because it is designed as a versatile conversational assistant rather than a specialized chess engine, lacking the ability to maintain and verify board states or perform move searches, which leads to errors beyond memorized openings. While it can understand chess concepts and explain strategies, users should rely on dedicated engines like Stockfish for accurate move generation and evaluation, using ChatGPT primarily for analysis and learning support.
ChatGPT’s poor performance at chess is not due to an inherent limitation of neural networks, as smaller specialized models have demonstrated grandmaster-level play without any search. The key issue lies in the purpose and training of ChatGPT: it was optimized to be a versatile conversational assistant rather than a dedicated chess player. While it can flawlessly reproduce common opening moves by recalling frequently seen sequences from its training data, it struggles once the game moves beyond these memorized lines and must generate moves without direct reference, leading to errors and illegal moves.
Despite its shortcomings, ChatGPT does exhibit a surprising level of understanding of the chessboard. Research has shown that language models trained solely on move sequences can internally reconstruct board states with high accuracy, indicating they track positions rather than just autocomplete moves. However, this internal representation is approximate and prone to mistakes, which is critical in chess where a single incorrect piece placement can invalidate subsequent calculations. Unlike specialized chess models trained on positions and engine evaluations, ChatGPT’s broad training objective dilutes its chess-playing capabilities.
Interestingly, older completion-based models like OpenAI’s gpt-3.5-turbo-instruct often play better chess than newer chat-tuned versions. This is because completion models are trained to predict the next text segment, closely matching the format of chess move lists, whereas chat models are fine-tuned for conversational usefulness, which detracts from their raw chess-playing skill. Other general chat models such as Gemini and Claude share similar limitations, as they also lack search capabilities and are optimized for dialogue rather than chess performance.
A fundamental challenge for ChatGPT is that it must reconstruct the entire board state from textual move lists each time it generates a move, without the ability to verify legality or maintain an internal, persistent board. This leads to errors such as suggesting moves for pinned or captured pieces, which a human player would immediately recognize as illegal. The model’s single-pass processing and lack of search mean it cannot double-check or correct these mistakes, unlike dedicated chess engines that evaluate millions of positions per second.
For practical use, ChatGPT is best suited to tasks that involve explaining chess concepts, annotating games after engine analysis, or providing study plans, rather than generating moves or evaluating positions. Users should rely on dedicated chess engines like Stockfish for move legality and evaluation, and use ChatGPT to understand the reasoning behind moves or strategies. By combining the strengths of both—engines for accuracy and language models for explanation—players can gain a deeper and more reliable understanding of chess.