Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 16 maio 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
Value targets in off-policy AlphaZero: a new greedy backup
Policy and value heads are from AlphaGo Zero, not Alpha Zero
Does the neural net of AlphaZero only evaluate the score of a
AlphaGo: How it works technically?, by Jonathan Hui
AlphaZero, a novel Reinforcement Learning Algorithm, in JavaScript
AlphaZero, a novel Reinforcement Learning Algorithm, in JavaScript
LightZero: A Unified Benchmark for Monte Carlo Tree Search in
AlphaZero Explained · On AI
Adaptive Warm-Start MCTS in AlphaZero-Like Deep Reinforcement
AlphaGo Zero – How and Why it Works – Tim Wheeler
Reimagining Chess with AlphaZero, February 2022
Warm-Start AlphaZero Self-play Search Enhancements
AlphaZero Explained · On AI
Win rate of QPlayer vs Random in Tic-Tac-Toe on different board
Recomendado para você
-
Comparison of network architecture of AlphaZero and NoGoZero+ (516 maio 2024
-
AlphaZero learns to solve quantum problems - ΑΙhub16 maio 2024
-
AlphaZero Explained16 maio 2024
-
AlphaZero paper published in journal Science : r/baduk16 maio 2024
-
Is AlphaZero really a scientific breakthrough in AI?, by Jose Camacho Collados16 maio 2024
-
AlphaZero: A General Reinforcement Learning Algorithm that Masters Chess, Shogi and Go through Self-Play16 maio 2024
-
AlphaZero: DeepMind's New Chess AI16 maio 2024
-
MuZero Intuition16 maio 2024
-
How the Artificial Intelligence Program AlphaZero Mastered Its Games16 maio 2024
-
Mastering chess and shogi by self-play with a general reinforcement learning algorithm16 maio 2024
você pode gostar
-
Goose from Untitled Goose Game by Cats, Download free STL model16 maio 2024
-
Display Dinossauro Baby - Decoração Infantil!16 maio 2024
-
Mapas Mentais sobre REGIÕES BRASILEIRAS - Study Maps16 maio 2024
-
Boiadeiro de Berna – Wikipédia, a enciclopédia livre16 maio 2024
-
Assistir Spare Me, Great Lord! – Episódio 01 Online16 maio 2024
-
Deepwoken Modern Celtor by BrookWing on DeviantArt16 maio 2024
-
6 dicas para aproveitar seu Moto 360 Sport ao máximo - TecMundo16 maio 2024
-
Games for Work app on Microsoft Teams - Good Idea or Bad Idea? - Gaming16 maio 2024
-
Read One-Room Hero Chapter 40: Just Curious on Mangakakalot16 maio 2024
-
Top 10+ game online PC miễn phí không thể bỏ lỡ trong 2023!16 maio 2024