# Available games

![](_static/green_circ10.png "green circle"): thoroughly-tested. In many cases,
we verified against known values and/or reproduced results from papers.

<font color="orange"><b>~</b></font>: implemented but lightly tested.

<font color="red"><b>X</b></font>: known issues (see code for details).

Status                                       | Game
-------------------------------------------- | ----
![](_static/green_circ10.png "green circle") | [Backgammon](#backgammon)
<font color="orange"><b>~</b></font>         | [Battleship](#battleship)
<font color="orange"><b>~</b></font>         | [Blackjack](#blackjack)
![](_static/green_circ10.png "green circle") | [Breakthrough](#breakthrough)
![](_static/green_circ10.png "green circle") | [Bridge](#bridge)
![](_static/green_circ10.png "green circle") | [(Uncontested) Bridge bidding](#uncontested-bridge-bidding)
<font color="orange"><b>~</b></font>         | [Catch](#catch)
<font color="orange"><b>~</b></font>         | [Cliff Walking](#cliff-walking)
<font color="orange"><b>~</b></font>         | [Clobber](#clobber)
<font color="orange"><b>~</b></font>         | [Coin Game](#coin-game)
![](_static/green_circ10.png "green circle") | [Connect Four](#connect-four)
<font color="orange"><b>~</b></font>         | [Cooperative Box-Pushing](#cooperative-box-pushing)
![](_static/green_circ10.png "green circle") | [Chess](#chess)
<font color="orange"><b>~</b></font>         | [Deep Sea](#deep-sea)
![](_static/green_circ10.png "green circle") | [First-price Sealed-Bid Auction](#first-price-sealed-bid-auction)
![](_static/green_circ10.png "green circle") | [Gin Rummy](#gin-rummy)
![](_static/green_circ10.png "green circle") | [Go](#go)
![](_static/green_circ10.png "green circle") | [Goofspiel](#goofspiel)
![](_static/green_circ10.png "green circle") | [Hanabi](#hanabi)
![](_static/green_circ10.png "green circle") | [Havannah](#havannah)
<font color="orange"><b>~</b></font>         | [Hearts](#hearts)
<font color="orange"><b>~</b></font>         | [Hex](#hex)
![](_static/green_circ10.png "green circle") | [Kuhn poker](#kuhn-poker)
<font color="orange"><b>~</b></font>         | [Laser Tag](#laser-tag)
![](_static/green_circ10.png "green circle") | [Leduc poker](#leduc-poker)
<font color="orange"><b>~</b></font>         | [Lewis Signaling](#lewis-signaling)
![](_static/green_circ10.png "green circle") | [Liar's Dice](#liars-dice)
<font color="orange"><b>~</b></font>         | [Markov Soccer](#markov-soccer)
![](_static/green_circ10.png "green circle") | [Matching Pennies (Three-player)](#matching-pennies-three-player)
![](_static/green_circ10.png "green circle") | [Negotiation](#negotiation)
<font color="orange"><b>X</b></font>         | [Oh Hell](#oh-hell)
![](_static/green_circ10.png "green circle") | [Oshi-Zumo](#oshi-zumo)
![](_static/green_circ10.png "green circle") | [Oware](#oware)
![](_static/green_circ10.png "green circle") | [Pentago](#pentago)
<font color="orange"><b>~</b></font>         | [Phantom Tic-Tac-Toe](#phantom-tic-tac-toe)
![](_static/green_circ10.png "green circle") | [Pig](#pig)
<font color="orange"><b>~</b></font>         | [Poker (Hold 'em)](#poker-hold-em)
![](_static/green_circ10.png "green circle") | [Quoridor](#quoridor)
<font color="orange"><b>~</b></font>         | [Sheriff](#sheriff)
<font color="orange"><b>~</b></font>         | [Slovenian Tarok](#slovenian-tarok)
<font color="orange"><b>~</b></font>         | [Skat (simplified bidding)](#skat-simplified-bidding)
<font color="orange"><b>~</b></font>         | [Solitaire (K+)](#solitaire-k)
![](_static/green_circ10.png "green circle") | [Tic-Tac-Toe](#tic-tac-toe)
![](_static/green_circ10.png "green circle") | [Tiny Bridge](#tiny-bridge)
![](_static/green_circ10.png "green circle") | [Tiny Hanabi](#tiny-hanabi)
![](_static/green_circ10.png "green circle") | [Trade Comm](#trade-comm)
![](_static/green_circ10.png "green circle") | [Y](#y)

## Details

### Backgammon

*   Players move their pieces through the board based on the rolls of dice.
*   Idiosyncratic format.
*   Traditional game.
*   Non-deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Backgammon)

### Battleship

*   Players place ships and shoot at each other in turns.
*   Pieces on a board.
*   Traditional game.
*   Deterministic.
*   Imperfect information.
*   2 players.
*   Good for correlated equilibria.
*   [Farina et al. '19, Correlation in Extensive-Form Games: Saddle-Point
    Formulation and
    Benchmarks](https://papers.nips.cc/paper/9122-correlation-in-extensive-form-games-saddle-point-formulation-and-benchmarks.pdf).
    Based on the original game
    [(wikipedia)](https://en.wikipedia.org/wiki/Battleship_\(game\))

### Blackjack

*   Simplified version of blackjack, with only HIT/STAND moves.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   1 player.
*   [Wikipedia](https://en.wikipedia.org/wiki/Blackjack)

### Breakthrough

*   Simplified chess using only pawns.
*   Pieces on a grid.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Breakthrough_\(board_game\))

### Bridge

*   A card game where players compete in pairs.
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   4 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Contract_bridge)

### (Uncontested) Bridge bidding

*   Players score points by forming specific sets with the cards in their hands.
*   Card game.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Contract_bridge)

### Catch

*   Agent must move horizontally to 'catch' a descending ball. Designed to test
    basic learning.
*   Agent on a grid.
*   Research game.
*   Non-deterministic.
*   Perfect information.
*   1 players.
*   [Mnih et al. 2014, Recurrent Models of Visual Attention](https://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention.pdf),<br>[Osband et al '19, Behaviour Suite for Reinforcement Learning, Appendix A](https://arxiv.org/abs/1908.03568)

### Cliff Walking

*   Agent must find goal without falling off a cliff. Designed to demonstrate
    exploration-with-danger.
*   Agent on a grid.
*   Research game.
*   Deterministic.
*   Perfect information.
*   1 players.
*   [Sutton et al. '18, page 132](http://www.incompleteideas.net/book/bookdraft2018mar21.pdf)

### Clobber

*   Simplified checkers, where tokens can capture neighbouring tokens. Designed
    to be amenable to combinatorial analysis.
*   Pieces on a grid.
*   Research game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Clobber)

### Coin Game

*   Agents must collect their and their collaborator's tokens while avoiding a
    third kind of token. Designed to test divining of collaborator's intentions
*   Agents on a grid.
*   Research game.
*   Non-deterministic.
*   Perfect, incomplete information.
*   2 players.
*   [Raileanu et al. '18, Modeling Others using Oneself in Multi-Agent
    Reinforcement Learning](https://arxiv.org/abs/1802.09640)

### Connect Four

*   Players drop tokens into columns to try and form a pattern.
*   Tokens on a grid.
*   Traditional game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Connect_Four)

### Cooperative Box-Pushing

*   Agents must collaborate to push a box into the goal. Designed to test
    collaboration.
*   Agents on a grid.
*   Research game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Seuken & Zilberstein '12, Improved Memory-Bounded Dynamic Programming for
    Decentralized POMDPs](https://arxiv.org/abs/1206.5295)

### Chess

*   Players move pieces around the board with the goal of eliminating the
    opposing pieces.
*   Pieces on a grid.
*   Traditional game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Chess)

### Deep Sea

*   Agent must explore to find reward (first version) or penalty (second
    version). Designed to test exploration.
*   Agent on a grid.
*   Research game.
*   Deterministic.
*   Perfect information.
*   1 players.
*   [Osband et al. '17, Deep Exploration via Randomized Value Functions](https://arxiv.org/abs/1703.07608)

### First-price Sealed-Bid Auction

*   Agents submit bids simultaneously; highest bid wins, and that's the price
    paid.
*   Idiosyncratic format.
*   Research game.
*   Non-deterministic.
*   Imperfect, incomplete information.
*   2-10 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/First-price_sealed-bid_auction)

### Gin Rummy

*   Players score points by forming specific sets with the cards in their hands.
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Gin_rummy)

### Go

*   Players place tokens on the board with the goal of encircling territory.
*   Tokens on a grid.
*   Traditional game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Go_\(game\))

### Goofspiel

*   Players bid with their cards to win other cards.
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   2-10 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Goofspiel)

### Hanabi

*   Players can see only other player's pieces, and everyone must cooperate to
    win.
*   Idiosyncratic format.
*   Modern game.
*   Non-deterministic.
*   Imperfect information.
*   2-5 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Hanabi_\(card_game\)) and
    [Bard et al. '19, The Hanabi Challenge: A New Frontier for AI Research](https://arxiv.org/abs/1902.00506)
*   Implemented via
    [Hanabi Learning Environment](https://github.com/deepmind/hanabi-learning-environment)

### Havannah

*   Players add tokens to a hex grid to try and form a winning structure.
*   Tokens on a hex grid.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Havannah)

### Hearts

*   A card game where players try to avoid playing the highest card in each
    round.
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   3-6 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Hearts_\(card_game\))

### Hex

*   Players add tokens to a hex grid to try and link opposite sides of the
    board.
*   Uses tokens on a hex grid.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Hex_\(board_game\))
*   [Hex, the full story by Ryan Hayward and Bjarne Toft](https://webdocs.cs.ualberta.ca/~hayward/hexbook/hex.html)

### Kuhn poker

*   Simplified poker amenable to game-theoretic analysis.
*   Cards with bidding.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Kuhn_poker)

### Laser Tag

*   Agents see a local part of the grid, and attempt to tag eachother with
    beams.
*   Agents on a grid.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Leibo et al. '17](https://arxiv.org/abs/1702.03037),
    [Lanctot et al. '17](https://arxiv.org/abs/1711.00832)

### Leduc poker

*   Simplified poker amenable to game-theoretic analysis.
*   Cards with bidding.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Southey et al. '05, Bayes’ bluff: Opponent modelling in poker](https://arxiv.org/abs/1207.1411)

### Lewis Signaling

*   Receiver must choose an action dependent on the sender's hidden state.
    Designed to demonstrate the use of conventions.
*   Idiosyncratic format.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Lewis_signaling_game)

### Liar's Dice

*   Players bid and bluff on the state of all the dice together, given only the
    state of their dice.
*   Dice with bidding.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Liar%27s_dice)

### Markov Soccer

*   Agents must take the ball to their goal, and can 'tackle' the opponent by
    predicting their next move.
*   Agents on a grid.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Littman '94, Markov games as a framework for multi-agent reinforcement learning](https://www2.cs.duke.edu/courses/spring07/cps296.3/littman94markov.pdf),<br>[He et al. '16, Opponent Modeling in Deep Reinforcement Learning](https://arxiv.org/abs/1609.05559)

### Matching Pennies (Three-player)

*   Players must predict and match/oppose another player. Designed to have an
    unstable Nash equilibrium.
*   Idiosyncratic format.
*   Research game.
*   Deterministic.
*   Imperfect information.
*   3 players.
*   "Three problems in learning mixed-strategy Nash equilibria"

### Negotiation

*   Agents with different utilities must negotiate an allocation of resources.
*   Idiosyncratic format.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   [Lewis et al. '17](https://arxiv.org/abs/1706.05125),
    [Cao et al. '18](https://arxiv.org/abs/1804.03980)

### Oh Hell

*   A card game where players try to win exactly a declared number of tricks. 
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   3-7 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Oh_Hell)

### Oshi-Zumo

*   Players must repeatedly bid to push a token off the other side of the board.
*   Idiosyncratic format.
*   Traditional game.
*   Deterministic.
*   Imperfect information.
*   2 players.
*   [Buro, 2004. Solving the oshi-zumo game](https://link.springer.com/chapter/10.1007/978-0-387-35706-5_23) <br> [Bosansky et al. '16, Algorithms for Computing Strategies in Two-Player Simultaneous Move Games](http://mlanctot.info/files/papers/aij-2psimmove.pdf)

### Oware

*   Players redistribute tokens from their half of the board to capture tokens
    in the opponent's part of the board.
*   Idiosyncratic format.
*   Traditional game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Oware)

### Pentago

*   Players place tokens on the board, then rotate part of the board to a new
    orientation.
*   Uses tokens on a grid.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Pentago)

### Phantom Tic-Tac-Toe

*   Tic-tac-toe, except the opponent's tokens are hidden. Designed as a simple,
    imperfect-information game.
*   Uses tokens on a grid.
*   Research game.
*   Deterministic.
*   Imperfect information.
*   2 players.
*   [Auger '11, Multiple Tree for Partially Observable Monte-Carlo Tree Search](https://hal.archives-ouvertes.fr/hal-00563480v2/document),<br>[Lisy '14, Alternative Selection Functions for Information Set Monte Carlo Tree Search](https://core.ac.uk/download/pdf/81646968.pdf), <br>[Lanctot '13](http://mlanctot.info/files/papers/PhD_Thesis_MarcLanctot.pdf)

### Pig

*   Each player rolls a dice until they get a 1 or they 'hold'; the rolled total
    is added to their score.
*   Dice game.
*   Traditional game.
*   Non-deterministic.
*   Perfect information.
*   2-10 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Pig_\(dice_game\))

### Poker (Hold 'em)

*   Players bet on whether their hand of cards plus some communal cards will
    form a special set.
*   Cards with bidding.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   2-10 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Texas_hold_%27em)
*   Implemented via [ACPC](http://www.computerpokercompetition.org/).

### Quoridor

*   Each turn, players can either move their agent or add a small wall to the
    board.
*   Idiosyncratic format.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2-4 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Quoridor)

### Sheriff

*   Bargaining game.
*   Deterministic.
*   Imperfect information.
*   2 players.
*   Good for correlated equilibria.
*   [Farina et al. '19, Correlation in Extensive-Form Games: Saddle-Point
    Formulation and
    Benchmarks](https://papers.nips.cc/paper/9122-correlation-in-extensive-form-games-saddle-point-formulation-and-benchmarks.pdf).
*   Based on the board game "Sheriff of Nottingham"
    [(bbg)](https://boardgamegeek.com/boardgame/157969/sheriff-nottingham)

### Slovenian Tarok

*   Trick-based card game with bidding.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   3-4 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/K%C3%B6nigrufen#Slovenia)
*   [Luštrek et al. 2003, A program for playing Tarok](https://pdfs.semanticscholar.org/a920/70fe11f75f58c27ed907c4688747259cae15.pdf)

### Skat (simplified bidding)

*   Each turn, players bid to compete against the other two players.
*   Cards with bidding.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   3 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Skat_\(card_game\))

### Solitaire (K+)

*   A single-player card game.
*   Card game.
*   Traditional game.
*   Non-deterministic.
*   Imperfect information.
*   1 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Klondike_\(solitaire\)) and
    [Bjarnason et al. '07, Searching solitaire in real time](http://web.engr.oregonstate.edu/~afern/papers/solitaire.pdf)

### Tic-Tac-Toe

*   Players place tokens to try and form a pattern.
*   Uses tokens on a grid.
*   Traditional game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Tic-tac-toe)

### Tiny Bridge

*   Simplified Bridge with fewer cards and tricks.
*   Cards with bidding.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2, 4 players.
*   See implementation for details.

### Tiny Hanabi

*   Simplified Hanabi with just two turns.
*   Idiosyncratic format.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2-10 players.
*   [Foerster et al 2018, Bayesian Action Decoder for Deep Multi-Agent
    Reinforcement Learning](https://arxiv.org/abs/1811.01458)

### Trade Comm

*   Players with different utilities and items communicate and then trade.
*   Idiosyncratic format.
*   Research game.
*   Non-deterministic.
*   Imperfect information.
*   2 players.
*   A simple emergent communication game based on trading.

### Y

*   Players place tokens to try and connect sides of a triangular board.
*   Tokens on hex grid.
*   Modern game.
*   Deterministic.
*   Perfect information.
*   2 players.
*   [Wikipedia](https://en.wikipedia.org/wiki/Y_\(game\))
