AIs calibrated to kyu/dan strength with easier to understand settings

[sanderland]

Current options are rather mathematical, calibrating some settings -> kyu/dan rank and a slider that sets them would improve usability.

[bale-go]

Katrain is an amazing piece of software. Having weaker opponents is a major selling point in my opinion.
I wanted to try out how the weaker AIs fare against GnuGo 3.8, which is 8 kyu.
ScoreLoss uses visits, which makes it slower and its strength depends on the max_visits. I figured it would be better to have a policy based method in this kyu range.
I opted for P:Pick since it seemed to be the most straightforward way to adjust the strength.

I set up a GnuGo AI with level 10 strength (8 kyu) as the opponent. My goal was to find settings where the game is even between the two AIs from the initial play to the endgame.
With the default P:Pick settings (pick_override: 0.95, pick_n=5, pick_frac=0.33) katago was still too strong.
I ran several games. In the beginning P:Pick always gained a huge advantage. However in the endgame P:Pick made obvious blunders that a DDK would clearly spot.

GnuGo 3.8 was black, default P:Pick was white, komi = 6.5

Interestingly, in the beginning pick_override of 0.8 was enough to get rid of the obvious blunders, but in the endgame 0.6 value was needed.
To account for the phenomenon, I changed pick_override to 0.8 and changed line 56 to decrease it over the game: (ai.py)

    elif policy_moves[0][0] > (ai_settings["pick_override"]*(1-(361-len(legal_policy_moves))/361.*.5)):

This needed an earlier definition of legal_policy_moves (I put it it line 46).
After the patch, no more obvious (meaning DDK) blunders were seen. However P:Pick still seemed to be stronger in the beginning, then in the endgame. So I decided against decreasing the number of moves considered. (Originally it gets smaller by the decrease in legal_policy_moves)

I wrote a little script to iteratively find the parameter (total number of moves seen by katago) that gives even games with different strengths of GnuGo. At least 10 games were played to estimate the parameter.
GnuGo 3.8 at level 8 (ca. 10 kyu): 24
GnuGo 3.8 at level 10 (ca. 8 kyu): 30
GnuGo 3.8 at level 10 with 4 handicap stones and 0.5 komi (ca. 4 kyu): 49

A simple linear regression gives: (total number of moves seen by katago) = -4.25 kyu + 66
This equation - with the changing pick_override setting - might be used to make the AI strength slider.

With above listed changes I could get an even game with GnuGo at various strengths.
I also tried it at my level, and the games were very enjoyable. GnuGo even when given handicap stones is too defensive. Modified P:Pick was much more creative.

Black: GnuGo 3.8 level 10
White: Modified PPick
(elif policy_moves[0][0] > (0.8*(1-(361-len(legal_policy_moves))/361.*.5)))
total number of moves seen by katago: 30

[sanderland]

@bale-go amazing work!
If you were setting up these matches manually, you may be interested in https://github.com/sanderland/katrain-bots for the self-play scripts and gtp-esque connectors.

One issue I have with formulas like this is deciding which parameters are exposed to users and how to expose or explain them to users -- or hide them alltogether.

[bale-go]

Thank you for the suggestion about katrain-bots. I used PyAutoGUI to automate the games.
I wanted to test the modified P:Pick for stronger bots. I opted for the open source Pachi.
"pachi -t =5000 --nodcnn" is 3k
"pachi -t =5000:15000" is 3d currently at KGS.

More than 10 games were run for each bot. After finding iteratively the correct parameter (total number of moves seen by katago), the games were quite balanced, without any serious blunder. None of the bots had an extra advantage at the beginning, middle or the end of the game.

Black: pachi -t =5000:15000 (rank=3d)
White: Modified PPick
(elif policy_moves[0][0] > (0.8*(1-(361-len(legal_policy_moves))/361.*.5)))
total number of moves seen by katago: 115

Even games with different bots at different total number of moves seen by katago.

GnuGo 3.8 at level 8 (ca. 10 kyu): 24
GnuGo 3.8 at level 10 (ca. 8 kyu): 30
GnuGo 3.8 at level 10 with 4 handicap stones and 0.5 komi (ca. 4 kyu): 49
pachi -t =5000 --nodcnn (3 kyu): 66
pachi -t =5000:15000 (3 dan): 115

Linear regression did not give a good fit in this wider rank range. Theoretically it makes more sense to use the log10 of the total number of moves seen. That way it is not possible to have negative number of seen moves.

The equation: (total number of moves seen by katago) = int(round(10**(-0.05737*kyu + 1.9482)))

The equation works for ranks from 12 kyu to 3 dan, which covers more than 90% of active players.
It should be noted that since there is no 0 kyu/dan, 3 dan = -2 kyu
This equation - with the changing pick_override setting to (0.8*(1-(361-len(legal_policy_moves))/361.*.5)) - might be used to make the AI strength slider for 90% of players.

The equation has an other nice feature. Extrapolating the line gives ca. 10 dan for perfect play, where total number of moves seen is the size of the go board (361).

One issue I have with formulas like this is deciding which parameters are exposed to users and how to expose or explain them to users -- or hide them alltogether.

I think it would be nice to have a simple view, where one could set the kyu level directly.
Maybe a tournament mode could be added later, where one starts at a user set rank. When human players win, they gain a rank, when they lose they lose a rank.

[sanderland]

any way you could have a look at this for p:weighted and scoreloss? I think they're my prefered AIs and curious how they perform on blunders in early vs end game.

[bale-go]

The reason I did not use scoreloss, is that it heavily depends on the max_visits and it is much slower.

Theoretically, I find the approach of P:Pick better. The value of the NN_policy seems rather arbitrary in many cases. One can see this by comparing the values of the score_loss and the NN_policy for a given position. The absolute value of NN_policy does not directly reflect the score_loss. For example, NN_polciy(best) = 0.71 and score_loss(best) = 0 points; NN_polciy(second_best) = 0.21 and score_loss(second_best) = 1 points. However, I found that the order of the moves from the best to the worst is very similar for score_loss and NN_Policy. P:Weighted relies on the absolute value of NN_policy. P:Pick relies on the order of the moves of NN_policy.
The latter is more robust.

[sanderland]

The compute cost and visits conflation is definitely an issue. However, a major weakness of pick over weighted is being blind to 'all' good moves on a regular basis and playing some policy <<1% move, at which point the ordering is quite random.

[bale-go]

I guess what I try to argue here is that having a policy value less than 1% is not a problem per se.

If you check amateur human vs. human games, there are multiple less than 1% or even less than 0.1% moves. The obvious blunders can be removed by using a shifting pick_override setting (80% initially to 50% endgame).
I looked in the logs of modified P:Pick with gnugo. total number of moves seen was 30, a seemingly low value. But P:Pick did not make clearly bad moves (that gnugo could take advantage of). Only 8% of the moves were below 0.1% and all of them were in the first 25 moves, where the decrease of NN_policy from best to worst is steep. Like placing a stone on the 4th line instead of the 3rd line could result in a NN_poicy << 1%. Only one third of the modified P:Pick moves were less than NN_policy=1%.

In the end the user experience is the most important. The runs with different bots show that the modified P:Pick policy makes a balanced opponent for a wide range of ranks.
I guess you might add a condition to remove NN_policy < 0.1%, but I think humans around 10 kyu make those too from time to time.

[sanderland]

<<1% is more like 0.1%, which more often is problematic (first line moves and such).
Anwyay, could you PR this as a new AI option into the v1.1.1 branch? If we have them side-by-side we can see where it leads.

[bale-go]

This is the first time I use GitHub (I only registered to participate in this fascinating project).
I will try my best.

[sanderland]

refactored a bit after the merge and added tests since it was turning into quite the spaghetti. It went all the way to losing by 80 points to near jigo against p:weighted and looks nice -- what bounds do you think there are on the rank?
Will see about running a couple of ogs bots on this and see where their ranks end up.

[bale-go]

The upper limit currently is the strength of the policy network, around 4d.
I played with it at 20k to check if everything is working at lower strengths. It seemed to play like a beginner as expected. But I do not know of any bots that play in that range to test the balanced play from opening to endgame, like I did with the 3d - 12k range bots.
Running ogs bots at different kyu settings (maybe? 8k, 5k, 2k, 2d) is a great idea. Let's see some real life data.

[sanderland]

got them working on ogs -- seems to work nicely, but it really shows how bad local is after adding an endgame setting to it!

[sanderland]

 * ai:p:rank(kyu_rank=2): ELO 1326.7 WINS 249 LOSSES 39 DRAWS 0
 * ai:p:territory(): ELO 1302.0 WINS 185 LOSSES 65 DRAWS 3
 * ai:p:tenuki(): ELO 1276.5 WINS 192 LOSSES 60 DRAWS 0
 * ai:p:weighted(): ELO 1156.7 WINS 206 LOSSES 81 DRAWS 1
 * ai:p:local(): ELO 1106.5 WINS 130 LOSSES 162 DRAWS 0
 * ai:p:pick(): ELO 1044.6 WINS 125 LOSSES 127 DRAWS 2
 * ai:p:rank(kyu_rank=6): ELO 1026.3 WINS 167 LOSSES 120 DRAWS 2
 * ai:p:rank(kyu_rank=10): ELO 761.2 WINS 86 LOSSES 202 DRAWS 0
 * ai:p:rank(kyu_rank=14): ELO 582.2 WINS 45 LOSSES 247 DRAWS 0
 * ai:p:rank(kyu_rank=18): ELO 417.3 WINS 3 LOSSES 289 DRAWS 0