Finally! Prophet 5.0 and chess4j 6.1 are out in the wild. The major advancement is the ability to use a neural network (NNUE). This is something I’ve been working towards for a few years! (See Towards NNUE.)
According to my testing, Prophet 5.0 is about 100 ELO stronger than 4.4. However, in all transparency, most of that is due to compiler optimizations. The separation between 5.0 using NNUE and 5.0 using a hand crafted evaluation is only about 20 ELO.
Rank Name Elo + - games score oppo. draws
1 toad-3.0 134 7 7 6000 58% 63 29%
2 luna-2.0.0 120 7 7 6000 56% 63 29%
3 fatalii-0.9.0 119 7 7 6000 56% 63 30%
4 jikchess-0.02 111 8 8 6000 55% 63 23%
5 prophet-5.0-nnue 103 4 4 24000 58% 63 26%
6 prophet-5.0-hce 86 4 4 24000 55% 63 28%
7 admete-1.5.0 82 8 8 6000 51% 63 26%
8 maverick-1.5 58 8 8 6000 47% 63 24%
9 smol-1.61 50 7 7 6000 46% 63 29%
10 tantabus-2.0.0 38 8 8 6000 44% 63 27%
11 sofcheck-0.9.1 22 7 8 6000 42% 63 28%
12 loki-3.5.0 22 8 8 6000 42% 63 26%
13 barbarossa-0.6.0 6 8 8 6000 40% 63 22%
14 prophet-4.4 0 4 4 24000 43% 63 27%
15 fornax-4.0 -4 8 8 6000 38% 63 26%
So, that is a little disappointing, but it’s progress so I’ll take it. There are still lots of ideas to try that will hopefully lead to a series of minor releases in the months to come.
chess4j also has the ability to use a neural network (also NNUE), but without the use of AVX intrinsics that are available in C/C++, it seems the overhead is just too high. I am wondering if the Vector API in Java’s Project Panama may have something to offer here, so for now I’ve left NNUE capability in chess4j until I can investigate.
Important note: HCE is still the default mode in both engines. You have to explicitly enable NNUE using command line parameters. See the README files and sample launch scripts in the release bundles for details.
The network architecture is a 3 layer fully connected network: 2x(768 -> 1536) -> 2. The input encoding is fairly standard in computer chess: a one hot encoding of 2 (players) x 6 (piece types) x 64 (squares) = 768 input bits (mostly a sea of zeros). The input layer is segmented: each position is mirrored vertically with colors flipped, but the weights are shared. This produces two 1536 element outputs, which are concatenated in the hidden layer. The two outputs are (1) the standard evaluation score as would be produced by the hand crafted evaluation, and (2) the probability of winning. These two values are meshed together after inference to produce a single score.
Training data is derived primarily from lichess.org, CCRL, and my collection of Prophet games played against testing partners. Approximately 400 million unique positions have been sampled from those sources and labeled with a depth 5 + quiescence search, as well as the game result. Mate scores are excluded altogether, and all other scores are clamped at +/- 15 pawns.
I want to publicly acknowledge and thank David Carteau for sharing the Cerebrum library, and Dominik Klein for his paper Neural Networks for Chess. I had spent quite some effort understanding neural networks in general and had even written my own trainer, but those two sources in particular helped me to take the next step of understanding NNUE. Also, I had never done any work with AVX intrinsics before, so David’s sample inference code was invaluable. I used it almost verbatim in Prophet.
The immediate plans for Prophet are to continue experimenting with NNUE (architectures, training data), looking for ways to optimize the code to offset the NNUE overhead, and whittle down a rather lengthy list of search related items in my backlog. For chess4j, I’m turning my attention to investigating Java’s Foreign Function & Memory API as a possible alternative to JNI.