2008

October 2017

January 2018

Chess

Go

January 2016

October 2017

December 2017

Leela

Leela Zero

Leela Chess Zero

AlphaGo

AlphaGo Zero

AlphaZero

AlphaZero (?)

Stockfish 10

Stockfish 9

Stockfish 8

Grandmasters

Me

TCEC12 – 32th (last)

TCEC 13
– 21st

CCC 1
– 3rd

CCC 2
– 3rd

TCEC 15
– 1st

CCC 8
– 2nd

TCEC CUP 3
– winner

CCC 7
– 1st

CCC 6
– 2nd

TCEC 14
– 2nd

TCEC CUP 2
– winner

CCC 5
– 2nd

CCC 4
– 2nd

CCC 3
– 2nd

TCEC CUP 1
– semifinals

June 2018

January 2019

June 2019

Chess
Game Tree

10¹²³ nodes

0.64

0.01

-8.93

0.39

Traditional Engines:

Eval function

Search algorithm

Move ordering

1.14

0.28

1.14

0.28

0.64

0.01

-8.93

0.39

Min:

-8.93

Min:

0.01

Min:

0.28

Max:

0.28

– Principal Variation

Traditional Engines:

Eval function

MiniMax (Search algorithm)

Move ordering

Eval function

Search algorithm

Move ordering

-8.93

?.??

Min:

0.28

Traditional Engines:

Eval function

Search algorithm

Move ordering

Max:

0.28

Min: -8.93

Already worse than 0.28,

prune remaining subtrees.

?.??

?.??

α/β pruning

Eval function

Search algorithm

Move ordering

Monte Carlo Tree Search

(kind of)

  • Start with only root node

visits →

  • Route visits to subtrees, prefer more "promising" subtrees
  • Results in very unbalanced trees
  • Best move is the subtree with most visits
  • Most "promising" subtree is combination of:
    • Initial "guess" of what the best move will be
      (when a subtree doesn't yet have many visits)
    • The subtree with the highest average eval of its nodes (when subtree already have many nodes)
    • Underexplored subtrees (much less visits than siblings)
Q+U = \frac{\sum_{subtree}V}{N} + C\times P \times \frac{\sqrt{\sum_{all subtrees}N}}{N + 1}

Q = average value of eval in a subtree

N = number of nodes in subtree

V = eval of a single node

C = a constant to balance Q and U

P = initial guess of move strength

Grows when sibling subtrees get visits, shrinks when own subtree gets visits

U starts high when subtree is small, then gives way to Q when subtree grows

Nxf6

Qg3

Qe3

Qe2

Subtree size (N)

vs Total visits

Subtree eval (Q)

vs Total visits

Subtree eval (Q)
vs Subtree size (N)

Initial guess of move

strength (P)

MiniMax

MCTS

  • Faster
    tens of millions nodes per second
  • Slower
    hundreds of thousands nodes per second
  • Doesn't keep tree in memory
    Searches of 100 000 000 000s nodes are usual
  • Keeps tree in memory
    only searches up to 100 000 000s nodes are realistic
  • Tree is balanced
  • Tree is very unbalanced
    focuses much more on promising variants
  • Final eval is taken from a leaf node
    unstable when eval function is noisy
  • Final eval is average of all nodes
    stable when eval function is noisy
  • Only runs eval function for leaf nodes
    at certain depth
  • Runs eval function for all visited nodes

Eval function

Search algorithm

Move ordering

Move ordering

Q

-1

1

Position eval
(-1 .. 1)

Input Planes
(chess position)

Residual
Neural Network
(40 convolutional layers)

Initial estimation how "promising" move is.

  • For every [from→to] square pair, probability that the move will the best.
    (+some more for promotion moves)

Value head

Policy head

if (Pt == BISHOP)
    {
        // Penalty according to number of pawns on the same color square as the
        // bishop, bigger when the center files are blocked with pawns.
        Bitboard blocked = pos.pieces(Us, PAWN) & shift<Down>(pos.pieces());

        score -= BishopPawns * pos.pawns_on_same_color_squares(Us, s)
                             * (1 + popcount(blocked & CenterFiles));

        // Bonus for bishop on a long diagonal which can "see" both center squares
        if (more_than_one(attacks_bb<BISHOP>(s, pos.pieces(PAWN)) & Center))
            score += LongDiagonalBishop;
    }

    // An important Chess960 pattern: A cornered bishop blocked by a friendly
    // pawn diagonally in front of it is a very serious problem, especially
    // when that pawn is also blocked.
    if (   Pt == BISHOP
        && pos.is_chess960()
        && (s == relative_square(Us, SQ_A1) || s == relative_square(Us, SQ_H1)))
    {
        Direction d = pawn_push(Us) + (file_of(s) == FILE_A ? EAST : WEST);
        if (pos.piece_on(s + d) == make_piece(Us, PAWN))
            score -= !pos.empty(s + d + pawn_push(Us))                ? CorneredBishop * 4
                    : pos.piece_on(s + d + d) == make_piece(Us, PAWN) ? CorneredBishop * 2
                                                                      : CorneredBishop;
    }
}

if (Pt == ROOK)
{
    // Bonus for aligning rook with enemy pawns on the same rank/file
    if (relative_rank(Us, s) >= RANK_5)
        score += RookOnPawn * popcount(pos.pieces(Them, PAWN) & PseudoAttacks[ROOK][s]);

    // Bonus for rook on an open or semi-open file
    if (pos.is_semiopen_file(Us, file_of(s)))
        score += RookOnFile[bool(pos.is_semiopen_file(Them, file_of(s)))];

    // Penalty when trapped by the king, even more if the king cannot castle
    else if (mob <= 3)
    {
        File kf = file_of(pos.square<KING>(Us));
        if ((kf < FILE_E) == (file_of(s) < kf))
            score -= TrappedRook * (1 + !pos.castling_rights(Us));
    }
}

Traditional eval

NN-based eval

  • Requires human chess expertise
  • Can learn all by itself (no humans!)
  • Faster
    typically tens of millions of evals per second
  • Slower
    typically tens of thousands of evals per second
  • Typically superficial
    doesn't recognize patterns, relies on deeper counting
  • Insightful
    sees positional patterns and even basic tactics
  • Works on CPU
  • Benefits from GPU
  • Evaluates one position at a time
  • Needs batches of hundreds of positions to be fast
  • Computes eval in pawns
    e.g. knight=3.1 pawns, bad bishop -0.4 pawns penalty
  • Computes expected outcome
    -1 – certain loss, 1 – win, 0.4 – win is likely

110 input planes

isRep

⇈⇈ Current position ⇈⇈

⇊⇊ 7 previous board positions ⇊⇊

= 0.0

= 1.0

= 0.06

=

=

white can 0-0

white can 0-0-0

black can 0-0

black can 0-0-0

is black's move

Moves without capture

(3/50) = 0.06

Legend

d4c4 c8d6 f2f3 d5a8 e7e4 e2e4 b4b8 g2d2 d4f4 d2a2 c7b8b h3f2 g4d1 b8a6 a1b3 a6b7 c6c8 g7e7 h2f4 b3a2 f1f4 g3g4 d4d7 d6g3 g7a7 g5g3 a3c3 h2d6 a1d4 c8c5 b6c4 g1g6
f7f6 g3h3 a1d1 b7a8 e5e3 b6g1 g4d7 b8b6 c7c6 f5b1 f6h4 g2a8 h4d4 c3c1 e1c2 d5d8 f4h2 f7f8b d4e3 c7b8q f1g1 d6c8 d6e4 e3d3 e7d8r f2e2 b5d5 e3c1 c3h3 h5e5 g3h1 e6d6
a3a1 g5g4 f4h6 f7h5 d7e8q f6b2 c7h2 a7b8 g4g3 g6e7 d3d8 h1h4 c8g4 f8b4 e4g2 a3b1 h7h8b c8e8 h2f3 g5g6 d5d4 a8d5 h3a3 b8f8 f5f3 b6b4 f7d8 d4a7 c5f8 a1h8 h4g4 g2e3
c1a3 b1h7 a2b2 b6d5 e5b5 f2g1 f7d5 h2h7 a7c5 a4d7 c6d6 a6b6 f3d1 g4f2 g7h8 a2d5 a3b5 b6b8 d6d5 d7b5 c3b4 f3g5 g6h6 b6b5 f8e8 d5b7 h8b8 b5e5 g7g1 g7f5 g1g8 d4b3
b7a6 a5h5 e1f2 g2c2 c4b4 h3f5 h2g2 c8a8 c3f6 d7f8 a3g3 h5h6 d4f6 b5d7 d3c2 a8e8 h7b7 d8d2 e4e7 f6f1 g8a8 b6d6 a8a2 c7a5 b5a5 e4c6 h7d7 d1d8 a7b8q h6h4 a7a8q d6d8
g7a1 e4f6 c5c6 a6b5 g6h7 h8e5 f8b8 c8d7 a5b3 h4e1 e8b5 d1f3 c1g5 h6f6 c6a4 g3c3 b2b7 c6b4 a1a3 h4e7 a6a4 d7a7 f3f6 c2b4 b5f1 d1c3 e8g8 a3a6 h7f6 f1e2 e4e6 e4c3
d7b7 h4a4 b3b7 f2g2 c8c2 a5f5 h4h6 c1b3 d8b6 a3b2 g6h8 b7c7 g2f3 h5g3 h7d3 b8a7 f4h4 d3b5 a8g8 a4a8 f7g7 a7h7 e1d1 a4c3 b5h5 b4b1 d5e7 e1g2 e7f5 c7e5 c2d4 f7f1
a1g7 d7d2 h7g7 d7d1 f1b1 b4e1 a5c5 a7f2 a2a8 b2c3 d5d7 a6a7 h3f3 a3c2 d7c7 f4c4 d2d4 a5d2 c5c8 f6f3 h2f1 b2h2 g4e5 e2c3 b3d3 f7b7 d2e4 c3a1 b8g8 h4h8 d8a8 g8c4
d5c7 f7f8r g1h3 a5e1 c2e1 e4b4 b1g6 a4c4 e4e8 f5e6 d8c8 f6g5 f7g8r h4g3 g5e4 e1e7 c2d2 e6g7 c4d5 d3f2 e7b7 d5c6 c8h8 f4e6 b7d8 c7g7 f1g3 d2d1 f4g5 g7g8 e1c1 a4b5
g8f7 c7b5 b1d1 e4c2 b6b7 a8b8 c1c7 h7a7 b5c6 g7g8r f6g7 e1d2 a3a7 c6e6 a7b6 e6a2 b8b5 d8d5 a8h1 g1f3 d4a4 d8c7 f8g6 e3e5 e2d1 f6e7 f8f6 d2b1 c6c7 a7a2 a6a3 c5f5
g7f8r e5c4 b4c2 d8d7 d3e1 g3g7 a6a8 e2e1 e8c8 f1g2 c2c8 g2b2 e3a7 h3h6 h7h2 g1g3 g2f1 h7g5 g7h7 c7d5 a8c8 e7a3 g4g5 g8f8 h4f5 b5e8 g7g3 h2e2 g5d2 b1a2 a8e4 b5b7
g6f6 e6c7 g7g8q a6e2 e4h1 d5e5 h4g6 e2b5 d7c8 b7f7 e2a6 a4h4 e7c6 h2h1 f7b3 d5f3 a7d7 d3f3 d1d2 f7f3 f3f8 a2c1 b7e4 f7d7 d8d3 h1c1 b6d7 g7d7 c4b6 a5b4 h4h1 b3e3
d1b1 c5e4 b7c8 a1a8 c7f4 d5b5 b2a4 g5h5 a3b4 h1f3 e3h3 f2b2 e8d6 d3d6 a6e6 h4f3 b4d4 c3e4 f5h4 a4g4 d3f4 d2h2 c8b8 c2c4 h3g3 h6f5 g6b6 b2d3 b2c2 b4b3 e8a8 b8b1
c3c6 c7a7 e5f7 d8g8 b6b2 h6g4 a4a6 d7d8q d4d6 a5b7 b3d2 e4f2 h1h5 h4h3 b3b1 b8b4 e4g6 c7c8q f4d5 c8e7 g5g8 e5e7 f4f3 g5c1 e5d4 a6c8 f2h1 h2e5 f2d2 f3e5 e2g3 c6a7
h4g2 h6c6 f3d5 h4f6 f3g1 e7f8 a7g7 h4e4 e4a4 d3b1 f4d6 e6f7 c6f3 d6f6 e8f8 d7d8 c4a3 b1e4 f5h7 f7e8b g1e1 b7d6 d7f6 c5d5 f8h8 d4c3 g3h5 e3e6 f6f2 c3a3 a6f6 f2e3
a4b4 a7g1 h7g8b e5d3 e5h8 a4c5 d8f7 f5e7 e8h8 g2f4 e1h1 e8d7 f7f5 e3d2 b6a6 c7b8r e2c4 b7g7 e1a5 g4g6 f5c8 a7a3 h3c3 c6f6 b6b1 g7c3 g3b8 f6g6 e5c6 d4h4 b2b5 f6e8
f5f6 a1c1 d7c5 f4g6 b6e6 a4a3 g3h2 c1g1 h7h4 h1b1 b4a6 e3f5 c1b2 e2f3 d6b8 e7g5 c4c8 b2b6 g4a4 f5f7 c3a4 g6g5 h6h5 c7a6 e4e5 c7d6 e3e1 h5c5 a5a6 b5a6 h2h8 a2f7
h8g6 e5a5 d6d3 h5g7 d5f7 h3e3 b3a3 c2h7 a7b8b f2c2 d7b8 g6f5 d8e8 a1g1 h8f7 d2b4 d7e7 h6g5 e2g2 h6d2 d7d4 h7c2 h5h1 h8g8 h8f8 g8g4 c4b2 d7e8b e8b8 d1d7 a6c7 c2a4
b3d5 e4d6 c5d3 d6f5 c4d3 b4a5 d7c8q b2a3 c1d1 a7b7 c7c2 g3b3 f6e6 d5c3 c6b8 b4b5 d5c4 h5h2 d5b6 g7b7 c5e5 h7h8q c6a6 f6f7 f7g8b a4e8 g7g8b f6e4 d1d6 c5h5 f6a6 d5g5
c4e4 g7h6 b1g1 b6f6 g6g8 b3c5 c4e2 e5e4 a7d4 b8d6 c4d6 h5h3 a1f1 h6a6 g8d5 f2f7 f3a8 c5a5 a2g2 f8h7 e5f4 c4f1 d7e8r a7c8 e7f7 h7e7 g6h5 f8e6 f7c7 h2h4 c5a4 e6c6
g4f4 a4c6 d4c5 d6f7 b4b6 c8c1 d7e8 g5d8 a7a5 a3c4 c6b5 e3c3 a6c4 a5c4 f5b5 d1d4 b7c8b c3a2 g6d6 e2f4 b1b4 e1a1 c2c5 c1e1 e7f8b d2b2 f7c4 f4g4 b4a4 c8g8 b3f3 b7g2
e2g1 b6g6 h2c2 e3e4 e8e6 f6h8 b7b5 g4e3 d8d6 e8e2 b3e6 c6h1 e2a2 d7g7 c3g3 d4d3 d6c5 d2c4 c6e8 e4d4 e3d1 d5a5 e1f1 b5c4 e6h6 c1c8 c8f8 e5a1 g7d4 d3e2 g5f7 f5g6
h1b7 b2g2 f7g8 h1g1 h3d3 g5h3 d8e7 d2f1 c4b3 c4a2 e3f2 e5c3 g6b1 f4b4 h5f4 f3g4 c6e7 e6d5 d3d4 h8c8 g7h5 e2d3 a4e4 b2b8 h7c7 h6g8 c1h1 c1c3 c6d5 f5e3 g5a5 f2f8
e3f4 g5e3 b8b2 b7a8q e8e1 e7c8 d7h3 f6h5 e8e5 a3a2 d6b6 c5c4 h8a1 h1e4 b7a8b f1a1 f8d6 h6h8 b7b3 h2h5 c4a5 f2e1 g1g5 b5e2 f7e8q c7c5 g4b4 c1c5 f6c3 f8f7 c6e4 e3g5
h1c6 e2c1 c3d4 f8a3 h1d1 e7c7 e7f8r g1e3 b5b4 b3c1 h6f7 h2b8 b1b2 f8g7 g6d3 h4h2 b1c3 h7b1 c7d8 f5h3 d5d1 c7b8 d4g4 f7f8 h1e1 b5b8 c3c5 b2b4 c6c4 b8b7 c7c3 d7e6
f4f8 c5c2 f2a2 h4d8 g4h6 g5f6 d3d2 h8g7 h8h2 g8f6 f4g2 f8d8 e7e6 f2h2 g2c6 e3d4 a4a2 b5d3 d7d8r e2f2 f4h3 d4b4 e2e3 g4e6 g6g1 e2d2 g3e1 c3b5 h1g2 b3g3 d3f1 h7h3
h3g4 g6f8 c7c4 e3f3 d3c5 g4c4 d6a3 c6b7 d2g5 e6g6 a5a8 h8h5 c7f7 g1a1 f8f5 b5a7 g7h8r h7g8q e1b4 f5g4 c3f3 b7b6 d6d1 d3d5 c4e5 e1e5 d2e2 f6d4 d8b7 a4f4 d2g2 h8d4
c4c5 e7f8q f3h3 b7e7 b8e5 a8a5 e5d6 h1h8 e4f3 d2b3 f4f5 b1f1 h2g1 b4h4 h6f8 d4f3 d1h1 h8c3 e6d7 d1f2 a3d3 d5d3 b1d2 e5c5 g8g3 a7b8r d5e4 b4a2 h6c1 f8e7 d4h8 e5h2
a3c1 d5a2 h3h8 e2b2 g4e2 d5c5 f5f8 c6d4 f7h7 c1f4 e7g8 c6g2 g4g7 a2a4 g2f2 f6d6 e3g2 e6c5 d6d4 b7a7 c5e3 c8d8 e3e8 d6c7 f1f7 e7d7 e3g4 c5b6 e6b6 b3g8 b3b8 g5g7
c6c2 b1b5 b1b3 b2e2 b8c7 d1e1 e4b1 g8b8 a6c5 c1b1 d1d5 h5f7 h5e8 b2f2 e6f8 e5g3 h7f7 b2b3 d4e2 f2h4 h3d7 f4d2 e1d3 g5f5 f2f4 g2g8 c6g6 h2b2 a8f3 b4g4 g7e6 e2e6
b2d1 c2c6 e6e8 g4f5 c2e4 g2g7 b4a3 d6e6 d7f5 c8a6 g3g5 f3f7 e7b4 c1c4 h3h4 e3b6 a4a1 b7b2 e7e8q a8g2 b1c1 c7e7 e6e2 a3h3 h7g8r g1c1 e4e3 d8a5 b6c7 h8h1 e7h4 b1h1
f3h1 f4f6 h6f4 a8a1 g6c6 b6c6 b3b2 c6d7 e1e4 b3h3 f8c8 g4g8 g5e5 h1g3 a2e6 c7e6 c7g3 c7h7 a3a4 e5g5 d2f2 d3a3 a4b6 a1f6 g3f2 a2c3 h3h5 h5g5 b7c6 f1h1 e6e4 f1d2

Neural Network output

1859 numbers:

  • Position value (expected game outcome) (-1.0 .. 1.0)
  • For each of the 1858 moves, probability that it will end up being chosen (0.0 .. 1.0)

Initially NN parameters are random

Contributors generate
Lc0 vs Lc0
games

NN training:

Old network + training data = 

new network

Neural
Network

NN parameters

Training data

new NN
parameters

latest NN parameters

old NN
parameters

Training
data
100 000 positions per minute

Training

Training data

Move Visits
e2e4 312
d2d4 193
g1f3 108
c2c4 79
g2g3 42
b2b3 38
f2f4 29
b1c3 10
Move Visits
d7d5 218
g8f6 205
c7c5 142
g7g6 102
d7d5 89
b2b3 29
f2f4 9
b1c3 5
Move Visits
e1f2 293
e2e4 183
g2g3 112
g1h3 89
e2e3 53
f2f4 45
d2d3 14
c2c3 9
Move Visits
d8h4 717
d7d6 34
g7g5 21
f7f5 12
d7d5 6
b7b5 3
h7h5 2
f8b4 1

checkmate

0-1
black wins

V=1

V=-1

V=1

V=-1

P

P

P

P

Self play, 800 visits (≈25ms) per move

NN doesn't know rules of chess

Move Visits
d5e7 657
c2c3 12
c2c4 9
... ...
Move Visits
... 0
48 9
... 0
452 657
... 0
1126 12

NN doesn't know:

  • Meaning of move indices
    (e.g. that move "452" is related to "d5" or "e7")
  • How pieces move
    (never sees result of move)
  • Which moves are legal
  • Goal of the game
    (i.e. role of king, checkmate)

NN is trained:

  • whether position is winning
  • move probabilities per index

V=1

V=1

Training Server

  • Noisy GPUs
  • Slow Internet

Web server

lzero.org

  • Silent
  • Fast Internet

Data backup server (data.lczero.org)

  • Large disk

Contributors

  • Generate 100 000 chess moves per minute

- training data (positions with P and V)

- neural network weights

Jul 2018

Sep 2018

Nov 2018

Jan 2019

Mar 2019

May 2019

1.0M

0.5M

1.5M

2.0M

2.5M

Generated games per day

 - test10

 - test20

 - test30

 - test40

https://lc0.org/contributions

Openings preference (test30)

https://lc0.org/openings

Neural Network

  • Larger network

  • Trying ideas from research papers

  • Better opening variety
    (opening book, chess960)

  • Different network versions play vs each other

Search

What's next?

  • Support transpositions

  • Improve scalability

  • Analysis support

  • Fix known weaknesses:

    • ​Time management

    • Sudden blunders

    • Better endgame

    • Non-standard openings

https://lczero.org/

– project page

https://lc0.org/start
https://lc0.org/play
https://lc0.org/watch

https://lc0.org/github
https://lc0.org/chat
https://lc0.org/forum
https://lc0.org/blog
https://lc0.org/downloads

https://lc0.org/slides

 – how to start with Lc0
– where to play Lc0 online
   – where to watch Lc0 playing

   – our code at Github
– our Discord chat
   – our forum
– our blog
            – download Lc0

  – these slides

image/svg+xml

Lc0

By Alexander Lyashuk

Lc0

Talk at Big Techday 12, organized by TNG Technology Consulting GmbH. Happened on June 7th, 2019.

  • 1,585
Loading comments...