Index: doc/gtp.texi =================================================================== RCS file: /cvsroot/gnugo/gnugo/doc/gtp.texi,v retrieving revision 1.11 diff -u -r1.11 gtp.texi --- doc/gtp.texi 20 Aug 2004 10:30:12 -0000 1.11 +++ doc/gtp.texi 26 Oct 2004 02:31:37 -0000 @@ -1,327 +1,178 @@ @menu * The Go Text Protocol:: The Go Text Protocol -* Protocol applications:: Protocol applications -* Protocol conventions:: Protocol conventions -* Regression with GTP:: Regression testing with GTP +* Running in GTP mode:: Running GNU Go in GTP mode +* GTP applications:: GTP applications +* Adding new GTP commands:: Adding new GTP commands * GTP command reference:: Details on every GTP command @end menu @node The Go Text Protocol -@section The GNU Go Text Protocol +@section The Go Text Protocol -GNU Go 3.0 introduced a new interface, the Go Text Protocol -(GTP). The intention is to make an interface that is better suited -for machine-machine communication than the ascii interface and -simpler, more powerful, and more flexible than the Go Modem -Protocol. - -There are two versions of the protocol. Version 1 was used with -GNU Go 3.0 and 3.2. GNU Go 3.4 and later versions uses version 2. -The specification -is available at @url{http:http://www.lysator.liu.se/~gunnar/gtp/}. - -The GTP has two principal current applications: it is used -in the test suite (@pxref{Regression}) and it is used to -communicate with @command{gnugoclient}, which is not part -of the GNU Go distribution, but has been used to run GNU -Go on NNGS. Other potential uses might be any of the current -uses of the GMP, for which the GTP might serve as a replacement. -This would likely entail extension and standardization of -the protocol. +GNU Go 3.0 introduced a new interface, the Go Text Protocol, abbreviated +GTP. The intention was to make an interface that is better suited for +machine-machine communication than the ascii interface and simpler, more +powerful, and more flexible than the Go Modem Protocol. + +There are two versions of the protocol. Version 1 was used with GNU Go +3.0 and 3.2. GNU Go 3.4 and later versions use protocol version 2. The +specification of GTP version 2 is available at +@url{http://www.lysator.liu.se/~gunnar/gtp/}. GNU Go 3.4 is the +reference implementation for GTP version 2, but all but the most common +commands are to be regarded as private extensions of the protocol. + +The GTP has a variety of applications. For GNU Go the first use was in +regression testing (@pxref{Regression}), followed by communication with +the NNGS go server and for automated test games against itself and other +programs. Now there are also many graphical user interfaces available +supporting GTP, as well as bridges to other Go servers than NNGS. + +@node Running in GTP mode +@section Running GNU Go in GTP mode + +To start GNU Go in GTP mode, simply invoke it with the option +@option{--mode gtp}. You will not get a prompt or any other output to +start with but GNU Go is silently waiting for GTP commands. A sample GTP session may look as follows: @example - hannah 2289% ./interface/gnugo --quiet --mode gtp - 1 loadsgf regression/games/incident156.sgf 249 - =1 - - 2 countlib C3 - =2 4 - - 3 findlib C3 - =3 C4 B3 D3 B2 - - 5 attack C3 - =5 0 - - owl_attack C3 - = 1 B4 - - 3 owl_defend C3 - =3 1 B5 - - owl_attack A2 - ? vertex must not be empty - - quit - = -@end example - -By specifying @option{--mode gtp} GNU Go starts in the GTP -interface. No prompt is used, just start giving commands. The -commands have the common syntax - -@example -[id] command_name [arguments] -@end example - -The command is exactly one line long, i.e. it ends as soon as a -newline appears. It's not possible to give multiple commands on the -same line. Before the command name an optional identity number can be -specified. If present it must be an integer between 0 and 2^31-1. The -id numbers may come in any order or be reused. The rest of the line -after the command name is assumed to be arguments for the command. -Empty lines are ignored, as is everything following a hash sign up to -the end of the line. - -If the command is successful, the response is of the form +virihaure 462% ./gnugo --mode gtp +1 boardsize 7 +=1 + +2 clear_board +=2 + +3 play black D5 +=3 + +4 genmove white +=4 C3 + +5 play black C3 +?5 illegal move + +6 play black E3 +=6 + +7 showboard +=7 + A B C D E F G + 7 . . . . . . . 7 + 6 . . . . . . . 6 + 5 . . + X + . . 5 + 4 . . . + . . . 4 + 3 . . O . X . . 3 + 2 . . . . . . . 2 WHITE (O) has captured 0 stones + 1 . . . . . . . 1 BLACK (X) has captured 0 stones + A B C D E F G -@example -=[id] result -@end example - -Here @samp{=} indicates success and @code{id} is the identity -number given in the command or the empty string if the id was -omitted. This is followed by the result, which is a text string -ending with two consecutive newlines. - -If the command fails for some reason, the response takes the form +8 quit +=8 -@example -?[id] error_message @end example -Here @samp{?} indicates failure, @code{id} is as before, and -@code{error_message} gives an explanation for the failure. This -string also ends with two consecutive newlines. +Commands are given on a single line, starting by an optional identity +number, followed by the command name and its arguments. -The available commands may always be listed using the single command -@command{help}. Currently this gives the list below. +If the command is successful, the response starts by an equals sign +(@samp{=}), followed by the identity number of the command (if any) and +then the result. In this example all results were empty strings except +for command 4 where the answer was the white move at C3, and command 7 +where the result was a diagram of the current board position. The +response ends by two consecutive newlines. + +Failing commands are signified by a question mark (@samp{?}) instead of +an equals sign, as in the response to command 5. + +The detailed specification of the protocol can be found at +@url{http://www.lysator.liu.se/~gunnar/gtp/}. The available commands in +GNU Go may always be listed using the command @command{list_commands}. +They are also documented in @xref{GTP command reference}. + + +@node GTP applications +@section GTP applications + +GTP is an asymmetric protocol involving two parties which we call +controller and engine. The controller sends all commands and the engine +only responds to these commands. GNU Go implements the engine end of the +protocol. + +With the source code of GNU Go is also distributed a number of +applications implementing the controller end. The most interesting of +these are: -@example -aa_confirm_safety -all_legal -analyze_eyegraph -attack -attack_either -black -block_off -break_in -boardsize -captures -clear_board -clear_cache -color -combination_attack -connect -countlib -cputime -decrease_depths -defend -defend_both -disconnect -dragon_data -dragon_status -dragon_stones -dump_stack -echo -echo_err -estimate_score -accuratelib -accurate_approxlib -experimental_score -eval_eye -final_score -final_status -final_status_list -findlib -finish_sgftrace -fixed_handicap -get_handicap -get_random_seed -set_random_seed -genmove -genmove_black -genmove_white -get_connection_node_counter -get_life_node_counter -get_owl_node_counter -get_reading_node_counter -get_trymove_counter -gg_genmove -gg-undo -help -increase_depths -influence -is_legal -known_command -komi -get_komi -ladder_attack -level -set_search_diamond -reset_search_mask -limit_search -set_search_limit -draw_search_area -limit_search -list_commands -list_stones -loadsgf -name -new_score -analyze_semeai -analyze_semeai_after_move -tactical_analyze_semeai -owl_attack -owl_connection_defends -owl_defend -owl_does_attack -owl_does_defend -owl_substantial -owl_threaten_attack -owl_threaten_defense -play -popgo -printsgf -orientation -place_free_handicap -protocol_version -query_boardsize -query_orientation -quit -restricted_genmove -reg_genmove -report_uncertainty -reset_connection_node_counter -reset_life_node_counter -reset_owl_node_counter -reset_reading_node_counter -reset_trymove_counter -same_dragon -set_free_handicap -showboard -is_surrounded -does_surround -surround_map -start_sgftrace -test_eyeshape -time_left -time_settings -top_moves -top_moves_black -top_moves_white -trymove -tryko -tune_move_ordering -undo -version -white -worm_cutstone -worm_data -worm_stones -@end example - -For exact specification of their arguments and results, -@xref{GTP command reference}. - - -@node Protocol applications -@section Protocol applications - -The protocol is asymmetric and involves two parties, which we may -call @samp{A} and @samp{B}. @samp{A} is typically some kind of arbiter or -relay and @samp{B} is a go engine. All communication is initiated by @samp{A} -in form of commands, to which @samp{B} responds. - -Potential setups include: - -@enumerate -@item Regression testing. -@display -A (regression script) -- B (engine). -@end display +@itemize +@item @file{regression/regress.awk} @quotation -A sets up a board position and asks B to e.g. generate a move or -find an attack on a specific string. -@end quotation -@item Human vs program. -@display -A (GUI) -- B (engine) -@end display +Script to run regressions. The script sends GTP commands to set up and +evaluate positions to the engine and then analyzes the responses from +the engine. More information about GTP based regression testing can be +found in the regression chapter (@pxref{Regression}). +@end quotation +@item @file{regression/regress.pl} @quotation -The GUI relays moves between the human and the engine and asks the -engine to generate moves. Optionally the GUI may also use GTP to -ask the engine which moves are legal or give a score when the game -is finished. +Perl script to run regressions, giving output which together with the +CGI script @file{regression/regress.plx} generates HTML views of the +regressions. @end quotation -@item Program vs program with arbiter. -@display -B1 (engine 1) -- A (arbiter) -- B2 (engine 2) -@end display +@item @file{regression/regress.pike} @quotation -A relays moves between the two engines and alternately asks the -engines to generate moves. This involves two different GTP -channels, the first between A and B1, and the second between A and -B2. There is no direct communication between B1 and B2. The -arbiter dictates board size, komi, rules, etc. -@end quotation -@item Program vs program without arbiter. +Pike script to run regressions. More feature-rich and powerful than +@file{regress.awk}. +@end quotation +@item @file{regression/view.pike} @quotation -The same as above except that B1 includes the arbiter -functionality and the first GTP link is shortcut. +Pike script to examine a single regression testcase through a graphical +board. This gives an easy way to inspect many of the GNU Go internals. @end quotation -@item Connection between go server and program. -@display -Go server -- A (relay) -- B (engine) -@end display +@item @file{interface/gtp_examples/twogtp} @quotation -A talks with a go server using whatever protocol is needed and -listens for match requests. When one arrives it accepts it, starts -the go engine and issues GTP commands to set up board size, komi, -etc. and if a game is restarted it also sets up the position. Then -it relays moves between the server and the engine and asks the -engine to generate new moves when it is in turn. +Perl script to play two engines against each other. The script +essentially sets up both engines with desired boardsize, handicap, and +komi, then relays moves back and forth between the engines. @end quotation -@end enumerate +@item @file{interface/gtp_examples/twogtp-a} +@quotation +An alternative Perl implementation of twogtp. +@end quotation +@item @file{interface/gtp_examples/twogtp.py} +@quotation +Implementation of twogtp in Python. Has more features than the Perl +variants. +@end quotation +@item @file{interface/gtp_examples/twogtp.pike} +@quotation +Implementation of twogtp in Pike. Has even more features than the Python +variant. +@end quotation +@item @file{interface/gtp_examples/2ptkgo.pl} +@quotation +Variation of twogtp which includes a graphical board. +@end quotation +@end itemize + +More GTP applications, including bridges to go servers and graphical +user interfaces, are listed at +@url{http://www.lysator.liu.se/~gunnar/gtp/}. + +@node Adding new GTP commands +@section Adding new GTP commands + +The implementation of GTP in GNU Go is distributed over three files, +@file{interface/gtp.h}, @file{interface/gtp.c}, and +@file{interface/play_gtp.c}. The first two implement a small library of +helper functions which can be used also by other programs. In the +interest of promoting the GTP they are licensed with minimal +restrictions (@pxref{GTP License}). The actual GTP commands are +implemented in @file{play_gtp.c}, which has knowledge about the engine +internals. -Setups 1 and 5 are in active and regular use with GNU Go. Programs -implementing setup 3 are also distributed with GNU Go (the files -@file{interface/gtp_examples/twogtp} and -@file{interface/gtp_examples/2ptkgo.pl}). - -@node Protocol conventions -@section Protocol conventions - -The GTP is currently unfinished and unstandardized. It is hoped that it -will grow to fill the needs currently served by the GMP and perhaps -other functions. As it is yet unstandardized, this section gives some -general remarks which we hope will constrain its development. We also -discuss how the GTP is implemented in GNU Go, for the benefit of anyone -wishing to add new commands. Notice that the current set of GTP commands -is a mix of generally useful ones and highly GNU Go specific ones. Only -the former should be part of a standardized protocol while the latter -should be private extensions. - -The purpose of the protocol is machine-machine communication. It may be -tempting to modify the protocol so that it becomes more comfortable for -the human user, for example with an automatic showboard after every -move. @strong{This is absolutely not the purpose of the protocol!} -Use the ascii interface instead if you're inclined to make such a -change. - -Newlines are implemented differently on different operating -systems. On Unix, a newline is just a line feed LF (ascii \012). -On DOS/Windows it is CRLF (\013\012). Thus whether GNU Go -sends a carriage return with the line feed depends on which -platform it is compiled for. The arbiter should silently -discard carriage returns. - -Applications using GTP should never have to guess about the basic -structure of the responses, defined above. The basic construction for -coding a GTP command can be found in @code{gtp_countlib()}: +To see how a simple but fairly typical command is implemented we look at +@code{gtp_countlib()} (a GNU Go private extension command): @example static int @@ -331,104 +182,101 @@ if (!gtp_decode_coord(s, &i, &j)) return gtp_failure("invalid coordinate"); - if (p[i][j] == EMPTY) + if (BOARD(i, j) == EMPTY) return gtp_failure("vertex must not be empty"); return gtp_success("%d", countlib(POS(i, j))); @} @end example -The functions @code{gtp_failure()} and @code{gtp_success()} -automatically ensures the specified response format, assuming the -strings they are printing do not end with a newline. +The arguments to the command are passed in the string @code{s}. In this +case we expect a vertex as argument and thus try to read it with +@code{gtp_decode_coord()} from @file{gtp.c}. + +A correctly formatted response should start with either @samp{=} or +@samp{?}, followed by the identity number (if one was sent), the actual +result, and finally two consecutive newlines. It is important to get +this formatting correct since the controller in the other end relies on +it. Naturally the result itself cannot contain two consecutive newlines +but it may be split over several lines by single newlines. + +The easiest way to generate a correctly formatted response is with one +of the functions @code{gtp_failure()} and @code{gtp_success()}, assuming +that their formatted output does not end with a newline. Sometimes the output is too complex for use with gtp_success, e.g. if -we want to print vertices, which gtp_success() doesn't +we want to print vertices, which gtp_success() does not support. Then we have to fall back to the construction in e.g. -@code{gtp_genmove_white()}: +@code{gtp_genmove()}: @example static int -gtp_genmove_white(char *s) +gtp_genmove(char *s) @{ - int i, j; - UNUSED(s); - if (genmove(&i, &j, WHITE) >= 0) - play_move(POS(i, j), WHITE); - + [...] gtp_start_response(GTP_SUCCESS); gtp_print_vertex(i, j); return gtp_finish_response(); @} @end example -Here @code{gtp_start_response()} writes the equal sign and the request id while -@code{gtp_finish_response()} adds the final two newlines. The next example -is from @code{gtp_influence()}: +Here @code{gtp_start_response()} writes the equal sign and the identity +number while @code{gtp_finish_response()} adds the final two newlines. +The next example is from @code{gtp_list_commands()}: @example +static int +gtp_list_commands(char *s) +@{ + int k; + UNUSED(s); + gtp_start_response(GTP_SUCCESS); - get_initial_influence(color, 1, white_influence, - black_influence, influence_regions); - print_influence(white_influence, black_influence, influence_regions); - /* We already have one newline, thus can't use gtp_finish_response(). */ + + for (k = 0; commands[k].name != NULL; k++) + gtp_printf("%s\n", commands[k].name); + gtp_printf("\n"); return GTP_OK; +@} @end example As we have said, the response should be finished with two newlines. Here we have to finish up the response ourselves since we already have -one newline in place. - -One problem that can be expected to be common is that an engine -happens to finish its response with three (or more) rather than -two consecutive newlines. This is an error by the engine that -the controller can easily detect and ignore. Thus a well -behaved engine should not send stray newlines, but should they -appear the controller should ignore them. The opposite problem -of an engine failing to properly finish its response with two -newlines will result in deadlock. Don't do this mistake! - -Although it doesn't suffice in more complex cases, gtp_success() is by -far the most convenient construction when it does. For example, -the function @code{gtp_report_uncertainty} takes a single argument -which is expected to be "on" or "off", after which it sets the -value of @code{report_uncertainty}, a variable which affects -the form of future GTP responses, reports success, and exits. The -function is coded thus: +one newline in place from the last command printed in the loop. -@example -static int -gtp_report_uncertainty(char *s) -@{ - if (!strncmp(s, "on", 2)) @{ - report_uncertainty = 1; - return gtp_success(""); - @} - if (!strncmp(s, "off", 3)) @{ - report_uncertainty = 0; - return gtp_success(""); - @} - return gtp_failure("invalid argument"); -@} -@end example +In order to add a new GTP command to GNU Go, the following pieces of +code need to be inserted in @file{play_gtp.c}: +@enumerate +@item A function declaration using the @code{DECLARE} macro in the list +starting at line 68. +@item An entry in the @code{commands[]} array starting at line 200. +@item An implementation of the function handling the command. +@end enumerate -@node Regression with GTP -@section Regression testing with GTP +Useful helper functions in @file{gtp.c}/@file{gtp.h} are: +@itemize +@item @code{gtp_printf()} for basic formatted printing. +@item @code{gtp_mprintf()} for printing with special format codes for +vertices and colors. +@item @code{gtp_success()} and @code{gtp_failure()} for simple responses. +@item @code{gtp_start_response()} and @code{gtp_end_response()} for more +complex responses. +@item @code{gtp_print_vertex()} and @code{gtp_print_vertices()} for +printing one or multiple vertices. +@item @code{gtp_decode_color()} to read in a color from the command arguments. +@item @code{gtp_decode_coord()} to read in a vertex from the command arguments. +@item @code{gtp_decode_move()} to read in a move, i.e. color plus +vertex, from the command arguments. +@end itemize -GNU Go uses GTP for regression testing. These tests are implemented as -files with GTP commands, which are fed to GNU Go simply by redirecting -stdin to read from a file. The output is filtered so that equal signs -and responses from commands without id numbers are removed. These -results are then compared with expected results encoded in GTP comments -in the file, using matching with regular expressions. More information -can be found in the regression chapter (@pxref{Regression}). @node GTP command reference @section GTP command reference -This section lists the GTP command set along with some information about each -command. Each entry in the list has the following fields: +This section lists the GTP commands implemented in GNU Go along with +some information about each command. Each entry in the list has the +following fields: @itemize @bullet @item Function: What this command does. @@ -439,13 +287,10 @@ @item Returns: What is displayed after the @dfn{=} and before the two newlines. Typical values include @dfn{nothing} or @dfn{a move coordinate} or some status string (there are others). -@item Status: How this command fits in with the GTP API. +@item Status: How this command relates to the standard GTP version 2 +commands. If nothing else is specified it is a GNU Go private extension. @end itemize -For suggestions on how to make effective use of the GTP command set, -@xref{Protocol applications}. Some of these commands are standard -commands in the protocol, others are GNU Go private commands. - Without further ado, here is the big list (in no particular order): @itemize @bullet Index: doc/regression.texi =================================================================== RCS file: /cvsroot/gnugo/gnugo/doc/regression.texi,v retrieving revision 1.6 diff -u -r1.6 regression.texi --- doc/regression.texi 6 Jul 2003 14:31:55 -0000 1.6 +++ doc/regression.texi 26 Oct 2004 02:31:37 -0000 @@ -19,7 +19,8 @@ @menu * Regression Testing:: Regression Testing in GNU Go * Test Suites:: Test Suites -* Performance Testing:: Performance Testing +* Running the Regressions:: Running the Regression Tests +* Running regress.pike:: Running regress.pike * HTML Views:: HTML Views @end menu @@ -31,8 +32,10 @@ suffix @file{.tst}, with corresponding correct results and expected pass/fail status encoded in GTP comments following the test. To run a test suite the shell scripts @file{test.sh}, @file{eval.sh}, and -@code{regress.sh} can be used. There are also Makefile targets to do -this. If you @command{make all_batches} most of the tests are run. +@file{regress.sh} can be used. There are also Makefile targets to do +this. If you @command{make all_batches} most of the tests are run. The +Pike script @file{regress.pike} can also be used to run all tests or a +subset of the tests. Game records used by the regression tests are stored in the directory @file{regression/games/} and its subdirectories. @@ -85,8 +88,8 @@ primarily used to report statistics, e.g. how many tactical reading nodes were spent while running the test suite. -@node Performance Testing -@section Performing tests +@node Running the Regressions +@section Running the Regression Tests @code{./test.sh blunder.tst} runs the tests in @file{blunder.tst} and prints the results of the commands on numbered lines, which may look @@ -172,8 +175,49 @@ For convenience the tests are also available as makefile targets. For example, @code{make blunder} runs the tests in the blunder test suite by -executing @code{eval.sh blunder.tst}. @code{make test} runs all test -suites in a sequence using the @code{regress.sh} script. +executing @code{eval.sh blunder.tst}. @code{make all_batches} runs all +test suites in a sequence using the @code{regress.sh} script. + +@node Running regress.pike +@section Running regress.pike + +A more powerful way to run regressions is with the script +@file{regress.pike}. This requires that you have Pike +(@url{http://pike.ida.liu.se}) installed. + +Executing @code{./regress.pike} without arguments will run all +testsuites that @code{make all_batches} would run. The difference is +that unexpected results are reported immediately when they have been +found (instead of after the whole file has been run) and that statistics +of time consumption and node usage is presented for each test file and +in total. + +To run a single test suite do e.g. @code{./regress.pike nicklas3.tst} or +@code{./regress.pike nicklas3}. The result may look like: +@example +nicklas3 2.96 614772 3322 469 +Total nodes: 614772 3322 469 +Total time: 2.96 (3.22) +Total uncertainty: 0.00 +@end example +The numbers here mean that the test suite took 2.96 seconds of processor +time and 3.22 seconds of real time. The consumption of reading nodes was +614772 for tactical reading, 3322 for owl reading, and 469 for +connection reading. The last line relates to the variability of the +generated moves in the test suite, and 0 means that none was decided by +the randomness contribution to the move valuation. Multiple testsuites +can be run by e.g. @code{./regress.pike owl ld_owl owl1}. + +It is also possible to run a single testcase, e.g. @code{./regress.pike +strategy:6}, a number of testcases, e.g. @code{./regress.pike +strategy:6,23,45}, a range of testcases, e.g. @code{./regress.pike +strategy:13-15} or more complex combinations e.g. @code{./regress.pike +strategy:6,13-15,23,45 nicklas3:602,1403}. + +There are also command line options to choose what engine to run, what +options to send to the engine, to turn on verbose output, and to use a +file to specify which testcases to run. Run @code{./regress.pike --help} +for a complete and up to date list of options. @node HTML Views @section HTML Regression Views