Commit ec39ddb8 authored by Vivien Kraus's avatar Vivien Kraus
Browse files

Clean up the MCTS API

parent 534ef706
Pipeline #179550 passed with stages
in 37 minutes and 35 seconds
......@@ -153,5 +153,4 @@
\ No newline at end of file
\ No newline at end of file
......@@ -60,7 +60,6 @@ AM_CFLAGS += \
BUILT_SOURCES += tuneconfig.h
......@@ -71,8 +70,7 @@ PRIO_1 = lib/
PRIO_2 = releases.xml
PRIO_3 = src/libtarot/cards/tarot/cards_defs.h \
src/libtarot/cards/tarot/cards_private_impl.h \
$(dist_model_DATA:.model=.h) \
PRIO_4 = src/libtarot/semantic-api/api.xml
PRIO_5 = $(bin_PROGRAMS)
This diff is collapsed.
......@@ -19,25 +19,14 @@ TANGLED = \
$(srcdir)/%reldir%/events.c \
$(srcdir)/%reldir%/petit-sec.c \
$(srcdir)/%reldir%/xml.c \
$(srcdir)/%reldir%/duplicate-a-posteriori.c \
$(srcdir)/%reldir%/tarot-tune \
$(srcdir)/tuneconfig.h \
$(srcdir)/%reldir%/tarot-figures/convergence-parameter.eps \
$(srcdir)/%reldir%/tarot-figures/convergence-parameter.png \
$(srcdir)/%reldir%/tarot-figures/convergence-agreed.eps \
$(srcdir)/%reldir%/tarot-figures/convergence-agreed.png \
$(srcdir)/%reldir%/tarot-figures/convergence-random.eps \
dist_noinst_SCRIPTS += %reldir%/tarot-tune
$(srcdir)/%reldir%/tarot.texi \
$(srcdir)/%reldir%/ \
$(srcdir)/%reldir%/setup.el \
$(srcdir)/%reldir%/org-stamp \
$(srcdir)/%reldir%/tarot.texi \
......@@ -46,11 +35,7 @@ MAINTAINERCLEANFILES += \
.PHONY: tune
tune: %reldir%/tarot-tune
$(MAKE) $(AM_MAKEFLAGS) src/tarot-app/tarot-status src/tarot-app/tarot-deal src/tarot-app/tarot-mcts
./pre-inst-env $(srcdir)/%reldir%/tarot-tune | tee $(srcdir)/%reldir%/convergence.csv
$(srcdir)/%reldir%/org-stamp: $(srcdir)/%reldir%/ $(srcdir)/%reldir%/setup.el $(srcdir)/%reldir%/convergence.csv
$(srcdir)/%reldir%/org-stamp: $(srcdir)/%reldir%/ $(srcdir)/%reldir%/setup.el
$(org_verbose) cd $(srcdir) && $(EMACS) --batch \
-l %reldir%/setup.el \
--eval "(defun package-version () \"$(VERSION)\")" \
......@@ -1752,59 +1752,6 @@ The best number of iterations is returned by
Get the best candidate in /mcts/, or =NULL= if there are no successful
#+attr_texinfo: :options {struct} TarotMctsNode
An inside node of the MCTS tree. It has a parent, and multiple
children. If not set, these relations are set to =NULL=.
#+attr_texinfo: :options {const TarotMctsNode *} tarot_mcts_root (const TarotMcts *@var{tree})
#+texinfo: @deftypefunx {const TarotMctsNode *} tarot_mcts_parent (const TarotMctsNode *@var{node})
#+texinfo: @deftypefunx {const TarotMctsNode *} tarot_mcts_first_child (const TarotMctsNode *@var{node})
#+texinfo: @deftypefunx {const TarotMctsNode *} tarot_mcts_next_sibling (const TarotMctsNode *@var{node})
#+texinfo: @deftypefunx {const TarotMctsNode *} tarot_mcts_previous_sibling (const TarotMctsNode *@var{node})
Move inside the MCTS /tree/.
#+attr_texinfo: :options {const TarotGameEvent *} tarot_mcts_get (const TarotMctsNode *@var{node}, size_t *@var{simulations}, int *@var{sum})
Get the data associated /node/: the total number of /simulations/ for
this node and its children, the /sum/ of all scores of those
simulations, and return the associated event.
The dog events are not part of the tree.
The hyperparameters are:
#+attr_texinfo: :options {void} tarot_mcts_set_parameter (TarotMcts *@var{mcts}, double @var{value})
#+texinfo: @deftypefunx {double} tarot_mcts_get_parameter (const TarotMcts *@var{mcts})
This is the only parameter in the general-purpose MCTS algorithm.
#+attr_texinfo: :options {void} tarot_mcts_set_simulation_random (TarotMcts *@var{mcts}, double @var{value})
#+texinfo: @deftypefunx {double} tarot_mcts_get_simulation_random (const TarotMcts *@var{mcts})
#+texinfo: @deftypefunx {void} tarot_mcts_set_simulation_agreed (TarotMcts *@var{mcts}, double @var{value})
#+texinfo: @deftypefunx {double} tarot_mcts_get_simulation_agreed (const TarotMcts *@var{mcts})
These parameters define how the simulation is performed by /mcts/.
For a high /value/ of the random simulation, /mcts/ will more
frequently simulate the tricks as if all players played at random.
For a low /value/ of the random simulation, /mcts/ will more
frequently simulate the tricks as if all players would decide which
card to play together before actually playing. So, the trick taker is
first designated, then {{{per}}} partners will give {{{per}}} more
card points and {{{per}}} opponents will play low-valued cards.
#+attr_texinfo: :options {size_t} tarot_mcts_default_iterations ()
Return the least number of iterations to achieve state-of-the-art card
playing with MCTS.
** The libtarot test suite
In this section we present the test suite to show different aspects of
......@@ -1902,422 +1849,6 @@ A method is composed of:
- an optional =instance= parameter (otherwise, this is a top-level
function, or a static method);
- some =parameter='s.
* Tuning the parameters
The MCTS algorithm comes with a bunch of parameters. We have:
- the main MCTS parameter, used to choose between a shallow
exploration of many moves and a deep exploration of a few moves;
- the tendency for the algorithm to simulate using random moves;
- the tendency for the algorithm to simulate if everyone agrees on
their cards;
- the number of iterations.
More parameters will come later.
In order to find the optimal values for all these parameters, we have
to /tune/ them. The current strategy is the following: start with a
population of players with different parameter values. Then, run the
following selection procedure:
1. Select 3 players from the population. Run 10 random 3-player
games, and sum the scores. There is one winner, one loser and more
than one non-winner;
2. Update the genes for the loser: get the values from the winner;
3. Mutate the loser: each parameter adds a normal value, multiplied by
the standard deviation between all non-winners;
4. Repeat with a 4-player variant, then 5-player with call.
The key phase is defining the genes of a player. I propose that all
genes are numbers between 0 and 1. Then, the MCTS parameter is
computed as 10 to the power of the scaled gene between -4 and 4, the
number of iterations as a power between 2 and 4 of 10, and the random
vs agreed as a simplex.
We don't really know when to stop. So we will have to investigate
** Running a match between MCTS players
First, we write the main function. It takes an R dataframe (in fact,
a tibble[fn::Tidyverse <3]) of parameter values, a random seed for the
deal, and augment the dataframe it with the score after a match. How
does it work? It simply calls the =tarot= program with the
appropriate flags, and parses the status to know which player should
play next. We should not forget to run the program with =LANG=C= so
that we can parse the output.
First, let's make a function to find who's next. The idea is simple:
first, set the locale to =C=. Then, call =tarot --status=. Feed in
the game. Take the output. Look at the last line. It should be a
player number, starting with a capital P. Discard that P and parse
the number. Congratulations, you have found who's next!
Let's just do that.
#+name: r-function-get-next
#+BEGIN_SRC R :eval no
get_next <- function (game) {
at_end <- suppressWarnings ({
system2 ("tarot-status",
args = c ("at-end"),
stdout = TRUE,
input = game,
env = c ("LANG=C"))
if (is.null (attr (at_end, "status"))) {
} else {
status <- system2 ("tarot-status",
args = c ("next"),
stdout = TRUE,
input = game,
env = c ("LANG=C"))
stopifnot (length (status) == 1)
last_line <- status[length (status)]
1 + as.integer (last_line)
This was easy. Now, a function to create a game.
#+name: r-function-new-game
#+BEGIN_SRC R :eval no
new_game <- function (n_player, with_call, seed) {
args <- c ("--n-players", n_player, "--seed", shQuote (seed))
if (with_call) {
args <- c (args, "--call-allowed")
ret <- system2 ("tarot-deal",
args = args,
stdout = TRUE,
input = c (),
env = c ("LANG=C"))
if (length (attr (ret, "status")) != 0
&& attr (ret, "status") != 0) {
} else {
We also need to know the score, if we don't have a next player.
#+name: r-function-score
#+BEGIN_SRC R :eval no
score <- function (game) {
status <- system2 ("tarot-status",
args = c ("score"),
stdout = TRUE,
input = game,
env = c ("LANG=C"))
sapply (status, as.integer)
The last function we need adds an MCTS move to the current game.
#+name: r-function-mcts
#+BEGIN_SRC R :eval no
mcts <- function (game,
simulation_random) {
args <- c ("--n-iterations", n_iterations,
"--mcts-parameter", parameter,
"--parameter-agreed", simulation_agreed,
"--parameter-random", simulation_random)
ret <- system2 ("tarot-mcts",
args = args,
stdout = TRUE,
input = game,
env = c ("LANG=C"))
if (length (attr (ret, "status")) != 0
&& attr (ret, "status") != 0) {
} else {
So, let's bind everything together. The next function will run a
match against 3, 4 or 5 players.
#+name: r-function-mcts-match-aux
#+BEGIN_SRC R :eval no
run_match_aux <- function (parameters, seed) {
run_player <- function (game, i) {
n_iterations <- parameters$n_iterations[i]
parameter <- parameters$parameter[i]
simulation_agreed <- parameters$simulation_agreed[i]
simulation_random <- parameters$simulation_random[i]
mcts (game, n_iterations, parameter, simulation_agreed, simulation_random)
run_next <- function (game) {
n <- get_next (game)
if ( (n)) {
score (game)
} else {
run_next (run_player (game, n))
np <- 4
wc <- FALSE
np <- nrow (parameters)
if (np == 5) {
wc <- TRUE
game <- new_game (np, wc, seed)
if (length (game) == 1 && (game)) {
parameters$score <- NA
} else {
score <- run_next (game)
if (length (score) == nrow (parameters)
&& all (is.finite (score))) {
parameters$score <- score
} else {
parameters$score <- NA
But since the first player would always be the same, it would be
better to start with the next player each time. The trick is in fact
to shuffle the parameters and shuffle them back along with the score.
#+name: r-function-mcts-match
#+BEGIN_SRC R :eval no
run_match <- function (parameters, seed, n) {
aux <- function (i) {
indices <- (seq_len (nrow (parameters)) + i - 1) %% nrow (parameters) + 1
parameters_shuffled <- parameters[indices,,drop = FALSE]
score_shuffled <- run_match_aux (parameters_shuffled, sprintf ("%s.%d", seed, i))$score
score <- array (0, nrow (parameters))
score[indices] <- score_shuffled
all_scores <- sapply (seq_len (n), aux)
mean_scores <- rowMeans (all_scores, na.rm = TRUE)
sd_scores <- apply (all_scores, 1, sd, na.rm = TRUE)
parameters$mean <- mean_scores
parameters$sd <- sd_scores
Now let's implement the genetic tuning. The first thing to do is map
the genes to the parameters. For now, we will simply ignore the gene
for the number of iterations. Otherwise, there is a small chance that
the tuning would lead to games with 1 iteration only, or too many
#+name: r-function-map-genes-to-parameters
#+BEGIN_SRC R :eval no
parameters_of_genes <- function (genes) {
get_n_iterations <- function (i) {
get_parameter <- function (i) {
raw <- genes[i, 2]
10 ^ (raw * 8 - 2)
get_simulation_agreed <- function (i) {
raw <- genes[i, 3]
if (raw <= 0) {
raw <- 1e-8
if (raw >= 1) {
raw <- 1
get_simulation_random <- function (i) {
raw <- genes[i, 4]
if (raw <= 0) {
raw <- 1e-8
if (raw >= 1) {
raw <- 1
n_iterations <- sapply (seq_len (nrow (genes)), get_n_iterations)
parameter <- sapply (seq_len (nrow (genes)), get_parameter)
simulation_agreed <- sapply (seq_len (nrow (genes)), get_simulation_agreed)
simulation_random <- sapply (seq_len (nrow (genes)), get_simulation_random)
data.frame (n_iterations = n_iterations,
parameter = parameter,
simulation_agreed = simulation_agreed,
simulation_random = simulation_random)
This function is boring. Let's evolve a population.
#+name: r-function-evolve
#+BEGIN_SRC R :eval no
evolve <- function (genes) {
n_players <- sample (c (3, 4, 5), 1)
selection <- sample (seq_len (nrow (genes)), n_players)
parameters <- parameters_of_genes (genes[selection,])
seed <- as.character (runif (1))
parameters <- run_match (parameters, seed, 10)
which.winner <- which.max (parameters$mean)
which.loser <- which.min (parameters$mean)
which.nonloser <- which (seq_len (n_players) != which.loser)
new_genes <- sapply (seq_len (ncol (genes)), function (j) {
m <- genes[selection[which.winner], j]
s <- sd (genes[selection[which.nonloser], j])
rnorm (1, m, s / 3)
genes[selection[which.loser],] <- new_genes
Finally, we can make a complete program to evolve the whole population
until convergence. The nice thing is that it prints the data to plot
it later.
#+BEGIN_SRC R :tangle tarot-tune :shebang "#!/usr/bin/env Rscript" :noweb yes :eval no
options (warn = 1)
convergence <- function (genes) {
by_parameter <- sapply (seq_len (ncol (genes)), function (j) {
mini <- min (genes[, j])
maxi <- max (genes[, j])
(maxi - mini < 1e-3)
all (by_parameter)
n <- 10
genes <- matrix (runif (n * 4), n, 4)
parameters <- parameters_of_genes (genes)
cat (sprintf ("%s\n",
paste (sapply (colnames (parameters), function (param) {
sprintf ("min_%s,max_%s", param, param)
}), collapse = ",")))
while (!convergence (genes)) {
genes <- evolve (genes)
parameters <- parameters_of_genes (genes)
cat (sprintf ("%s\n",
paste (sapply (seq_len (ncol (parameters)), function (j) {
sprintf ("%f,%f", min (parameters[, j]), max (parameters[, j]))
}), collapse = ",")))
The tuning has been done by the maintainer. The final parameter
values are saved in the file =tuneconfig.h= so that it is the default
for libtarot. The following plots show the convergence: figure
[[fig-convergence-parameter]] for the main parameter value, figure
[[fig-convergence-agreed]] for the usefulness of a smart simulation,
figure [[fig-convergence-random]] for the usefulness of a random
#+name: fig-convergence-parameter
#+caption: Convergence of the non-losing parameter values
#+name: fig-convergence-agreed
#+caption: Convergence of the non-losing probability to agree during the simulations
#+name: fig-convergence-random
#+caption: Convergence of the non-losing probbility to simulate at raandom
#+name: gen-tuneconfig-h
#+BEGIN_SRC R :results output :exports none
data <- read.csv ("convergence.csv")
n_iterations <- (data$min_n_iterations + data$max_n_iterations) / 2
parameter <- (data$min_parameter + data$max_parameter) / 2
agreed <- (data$min_simulation_agreed + data$max_simulation_agreed) / 2
random <- (data$min_simulation_random + data$max_simulation_random) / 2
cat (sprintf ("#ifndef %s\n", guard))
cat (sprintf ("#define %s\n", guard))
cat (sprintf ("#define TUNE_ITERATIONS %d\n",
as.integer (floor (n_iterations[length (parameter)]))))
cat (sprintf ("#define TUNE_PARAMETER %f\n", parameter[length (parameter)]))
cat (sprintf ("#define TUNE_SIMULATION_RANDOM %f\n", random[length (parameter)]))
cat (sprintf ("#define TUNE_SIMULATION_AGREED %f\n", agreed[length (parameter)]))
cat (sprintf ("#endif /* not %s */\n", guard))
cat ("/* ")
dir.create ("tarot-figures", recursive = TRUE)
library ("ggplot2")
data$iteration <- seq_len (nrow (data))
(ggplot (data, aes (x = iteration,
ymin = min_parameter,
ymax = max_parameter))
+ geom_ribbon (alpha = .3)
+ scale_y_log10 ()
+ xlab ("Iteration")
+ ylab ("Non-losing parameter value")
+ ggtitle ("Convergence of the main MCTS parameter value"))
ggsave ("tarot-figures/convergence-parameter.png")
ggsave ("tarot-figures/convergence-parameter.eps")
(ggplot (data, aes (x = iteration,
ymin = min_simulation_agreed,
ymax = max_simulation_agreed))
+ geom_ribbon (alpha = .3)
+ xlab ("Iteration")
+ ylab ("Non-losing parameter value")
+ ggtitle ("Convergence of the tendency to agree during simulations"))
ggsave ("tarot-figures/convergence-agreed.png")
ggsave ("tarot-figures/convergence-agreed.eps")
(ggplot (data, aes (x = iteration,
ymin = min_simulation_random,
ymax = max_simulation_random))
+ geom_ribbon (alpha = .3)
+ xlab ("Iteration")
+ ylab ("Non-losing parameter value")
+ ggtitle ("Convergence of the tendency to play at random during simulations"))
ggsave ("tarot-figures/convergence-random.png")
ggsave ("tarot-figures/convergence-random.eps")
cat (" */\n")
#+BEGIN_SRC C :eval no :tangle ../tuneconfig.h :noweb yes
,* file tuneconfig.h
,* Copyright (C) 2019 Vivien Kraus <>
,* This program is free software: you can redistribute it and/or
,* modify it under the terms of the GNU General Public License as
,* published by the Free Software Foundation, version 3 of the
,* License.
,* This program is distributed in the hope that it will be useful, but
,* WITHOUT ANY WARRANTY; without even the implied warranty of
,* General Public License for more details.
,* You should have received a copy of the GNU General Public License
,* along with this program. If not, see
,* <>.
* Machine learning with tarot
There are support for different paradigms of machine learning in
tarot. The library provides a set of hand-crafted high-level
......@@ -15,13 +15,6 @@
info_TEXINFOS += %reldir%/tarot.texi
tarotfiguresdir = $(infodir)/tarot-figures
dist_tarotfigures_DATA = \
%reldir%/tarot-figures/convergence-parameter.png \
%reldir%/tarot-figures/convergence-agreed.png \
%reldir%/tarot-figures/convergence-random.png \
%reldir%/tarot-figures/convergence-parameter.eps \
%reldir%/tarot-figures/convergence-agreed.eps \
%canon_reldir%_tarot_TEXINFOS = \
......@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: tarot\n"
"POT-Creation-Date: 2019-10-21 23:12+0200\n"
"POT-Creation-Date: 2019-10-26 19:22+0200\n"
"PO-Revision-Date: 2019-10-12 11:34+0200\n"
"Last-Translator: Vivien Kraus <>\n"
"Language-Team: French\n"
......@@ -18,7 +18,7 @@ msgstr ""
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"
#: src/tarot-app/main.vala:64
#: src/tarot-app/main.vala:52
#, c-format
msgid ""
"ERROR: environment variable '%s' is not set. Try reinstalling the program.\n"
......@@ -26,35 +26,35 @@ msgstr ""
"ERREUR : la variable d'environnement '%s' n'est pas définie. Essayez de "
"réinstaller le programme.\n"
#: src/tarot-app/main.vala:79
#: src/tarot-app/main.vala:67
msgid "Error: XDG_DATA_HOME nor HOME are set.\n"
msgstr "Erreur : ni XDG_DATA_HOME ni HOME ne sont définiies.\n"
#: src/tarot-app/main.vala:84
#: src/tarot-app/main.vala:72
#, c-format
msgid "Error: environment variable '%s' is not set.\n"
msgstr "Erreur : la variable d'environnement '%s' n'est pas définie.\n"
#: src/tarot-app/main.vala:98 src/tarot-app/program_commons.c:45
#: src/tarot-app/main.vala:86 src/tarot-app/program_commons.c:45
#, c-format
msgid "ERROR: could not initialize libtarot.\n"
msgstr "ERREUR : impossible d'initialiser libtarot.\n"
#: src/tarot-app/main.vala:110
#: src/tarot-app/main.vala:98
msgid "Print version information and exit"
msgstr "Affiche l'information de version et quitte"
#: src/tarot-app/main.vala:112 src/tarot-app/main.vala:122
#: src/tarot-app/tarot_deal.c:66 src/tarot-app/tarot_mcts.c:96
#: src/tarot-app/tarot_stacking.c:82 src/tarot-app/tarot_features.c:62
#: src/tarot-app/tarot_cnn_features.c:59
#: src/tarot-app/main.vala:100 src/tarot-app/main.vala:110
#: src/tarot-app/tarot_deal.c:66 src/tarot-app/tarot_mcts.c:89
#: src/tarot-app/tarot_stacking.c:81 src/tarot-app/tarot_features.c:61
#: src/tarot-app/tarot_cnn_features.c:58
msgid "version"
msgstr "version"
#: src/tarot-app/main.vala:129 src/tarot-app/tarot_status.c:57
#: src/tarot-app/tarot_deal.c:106 src/tarot-app/tarot_mcts.c:147
#: src/tarot-app/tarot_stacking.c:112 src/tarot-app/tarot_features.c:91
#: src/tarot-app/tarot_cnn_features.c:87
#: src/tarot-app/main.vala:117 src/tarot-app/tarot_status.c:57
#: src/tarot-app/tarot_deal.c:106 src/tarot-app/tarot_mcts.c:135
#: src/tarot-app/tarot_stacking.c:111 src/tarot-app/tarot_features.c:90
#: src/tarot-app/tarot_cnn_features.c:86
#, c-format
msgid ""
"%s (libtarot %s)\n"
......@@ -1170,7 +1170,7 @@ msgstr "fini"
msgid "score"
msgstr "score"
#: src/tarot-app/tarot_status.c:99 src/tarot-app/tarot_features.c:43
#: src/tarot-app/tarot_status.c:99 src/tarot-app/tarot_features.c:42
#, c-format
msgid "Error: this game is not finished yet.\n"
msgstr "ERREUR : cette partie n'est pas terminée.\n"
......@@ -1200,9 +1200,9 @@ msgstr ""
"Erreur : impossible d'utiliser cette graine aléatoire, « %s », pour la\n"
"distribution. Veuillez réessayer avec une graine différente.\n"
#: src/tarot-app/tarot_deal.c:65 src/tarot-app/tarot_mcts.c:95
#: src/tarot-app/tarot_stacking.c:81 src/tarot-app/tarot_features.c:61
#: src/tarot-app/tarot_cnn_features.c:58
#: src/tarot-app/tarot_deal.c:65 src/tarot-app/tarot_mcts.c:88
#: src/tarot-app/tarot_stacking.c:80 src/tarot-app/tarot_features.c:60
#: src/tarot-app/tarot_cnn_features.c:57
msgid "help"
msgstr "aide"
......@@ -1218,7 +1218,7 @@ msgstr "appel-autorisé"