a) Suitability for whom. Some of these can be adjusted downward in playing strength so as to be useful for beginners. Others cannot, or there is a minimum strength level possible.

b) How best used? You can learn bad habits from those programs that have habits that are bad (or modes in which that is true) but not necessarily under all conditions of use. A program that might make a poor opponent weakened so that you can play it even (because not like a human) might be perfectly OK if set stronger enough so that you must take several handicap stones.

c) Cost (not all are free). And that should include what you get for that price (are future upgrades free or reduced price?). Try to keep the political rants (free software vs sold software) within reasonable limits. I suggest the original vision of the free software movement for that, not free as in "free beer". In other words, "free" means "for no more than the reasonable/customary cost when mailed to you on standard medium" (nobody is obligated to provide you with a free download, actually nobody is obligated to provide you with a run exec already compiled for your computer, just free source code*) and "expensive" means "more than the price of a book" (where book means a typical college text book simce that's what the folks who used the phrase would think of as a book).

d) Remember, hardware does matter. In describing the strength of these programs we should include "when running on what". By and large the crunch power of the hardware and the time allowed the computer per move are inversely proportional for a given level of strength.

Feel free to add to this list of what we should include.

* In other words, a "free software" project could (without violating the rules) charge a small amount by only providing on CD or DVD and for providing a ready to run executable for your OS. They could not get away with charging very much since anybody else who wished to could make these available for free download as redistribution of "free software" is allowed if following the rules of the license. AFAIK no current free software projects are unavailable for free download but I know of at least on that will also provide on CD or DVD for $10/ disk and various downloading services will do this for a price.

Your 4 points make an excellent framework for reviews. I've just gotten started - recently enough that I haven't lost my first 100 games yet. I've been playing on my iPad, so I haven't spent much money on software, but the difference among programs, even just at the beginner level, is noticeable. For example, I can beat Champion Go at the lowest level, most times, but with Smartgo Kifu I'll be happy to begin losing by fewer than 10 points (with a 5 stone handicap!)

So if I were buying PC software like MFOG or Smartgo, at a much greater cost, I'd probably be angry and discouraged. I'm not, thanks to what I've learned on this forum, but the 4 points above would be a helpful buyers guide.

And you have just suggested an additional point and an observation

point: When discussing a program make sure to indicate what hardware would be required for the specifed strength. Some of these programs will play on less hardware but weaker than described.

observation: Unlike a human only somewhat stronger than you are, with some of these programs, especially the ones using some form of MCTS for move evaluation, the margin of your loss may be a poor indication of the difference in strength. Thus taking a four stone handicap and always losing by 10 does not mean that five stones would be enough. You might still lose by ten taking five stones and perhaps the same with six stones! Not unlike when a very strong player is helping you learn the game. If the program sees it is safely ahead it will move so as to remain safely ahead. Sort of like a match race between two sailboats. If one boat is ahead and the boat behind veers off to one side of the direct course to the next mark the leading boat will also go off course so as to stay between the trailing boat and the mark. While that doesn't maximize the margin of victory it ensures remaining ahead no matter how the wind might shift.

I decided to experiment with Smartgo Kifu, which I believe is strictly MCTS. I started out with a 6 stone handicap. The game assigned me a huge lead because I covered a lot of territory. I've lost by a lot with that head start before, but this time I spread out my stones, and then as soon as I took one prisoner the game resigned! When I took this same approach yesterday, with a 5 stone handicap, the game continued to the end but brought my margin of loss down quite a bit. While it was fun to 'overpower' my adversary, the games I play with Champion Go, win or lose, are more interesting. I believe Champion Go, like Igowin, uses MCTS plus a knowledge tree.

Next experiment is to see if Smartgo Kifu will behave differently if I play without a handicap.

If you see something like that, report it. When a program using some form of MCTS evaluator is playing a game with handicap stones it is necessary that there be dynamic adjustment of a pseudo-komi to get the algorithm to work correctly. The sort of thing that might have a bug for some particular number of handicap stones. We who work in the cypher mines (in my case used to) need problems reported if we are to fix them.

OK --- I'll put up a first review to try to show what I think they should be like (feel free to disagree.

Many Faces of Go

Platform: A PC with at least a 2 core 2 GHz processor and a GByte of memory. The prgram may run on less but probably not as strong. Operating systems MS Windows only (claimed that it will run under a Windows emulator).

Cost: About $90 which is on the high side. Note however that this is for a "household site" license so may be a good deal in comparison to alternatives sold with a one computer license if you have several go players in the household or want it on all your machines. Oonce you have bought a version of MFOG future versions cost about half and interim updates are free. Some people find it annoying that you have to obtain a separate key for each user on each machine.

Incldues:
a) A "how to play go" tutorial for beginners.
b) Joseki tutor.
c) Fuseki tutor.
d) Go problems in three levels. These are sort of a "flashcard" system where the frequency with which you see problems (in all color revelrsals and rotations) is based upon your record of getting them right or wrong.
e) Game editor.
f) Internet client for playing on the servers.
g) Go player.
This has seven levels, 1 dan, 3 kyu (these top two use a MCTS evaluator), 6 kyu, 9 kyu, 12 kyu, 15 kyu, and 18 kyu (these lower five use a "go knowledge based" AI evaluator). You can set an option to rnadomaize moves (don't always play the same in the same situation) which applies just to the lower five levels as a MCTS evaluator always does that. You can specify non-standard placement of handicap stones and komi (but not reverse komi). You can ask it to indicate why it made certain moves (what "go reasons" behind them) and that is true even when the MCTS evaluator is being used.

In common with most AI evaluators those levels will play somewhat passively and "tight". In common with most MCTS evaluators those levels will play very agressively and loose. That is a limitation in terms of learning to play against human opponents since you will meet all styles of players. Also, the AI levels may have weakensses you could learn to exploit.

Verification of strength:
I haven't tried the lower levels. I took the program on one of my laptops to the local go club where two very strong players (7 dan and 5 dan) played it at the appropriate handicaps. They agreed that it was a legitimate 1 dan (both thought weaker in strategy but significantly stronger in tactical fighting; both got surprised by its counter-attack)

PS -- since this is the first review, please say if you think I left anything important out.

I think time for thinking/goban size is important for strength evaluation.
I want to know from other users about their opinions how strong is actual MFGO version on PC( similar to single proc 4core 3.7GHz/18GB RAM hardware) in case of 1h and 3h or more thinking time for computer on 19/19.

As you may know I've been playing with Fuego some, an open-source program using a similar algorithm to Many Faces. There's an ongoing Malkovich and a finished one, which may give you some idea of how the strength scales with thinking time (30 min / move).


viewtopic.php?f=37&t=4356

lifein19x19.com/forum/viewtopic.php?f=37&t=3960

Do not assume that programs will be stronger on stronger computers. I think in many cases level 10 on one machine equals level 10 on another regardless of processing power, the difference is how fast it will make a move.

Are you familiar enough with feugo to add the sort of review I just did. But don't give as an example of its playing strength how it would perform on a 120 core super cluster! (playing at a more reasonable speed of perhaps 30 sec per move --- see, I just multiplied your number of cores by 60 changing minutes to seconds).

Amount of RAM can have an impact also, and can become a limiting factor when you get faster machines and/or longer time limits.

Maybe.

I haven't cheated and looked at the code of any of these (and in any case, it's a higher level design than the code which would matter). But it's not obvious that a MCTS type algorithm would necessarily require a great deal of storage for the tree being built. I doubt that all the data of the games played out is being kept, just the node being added to the tree*.

Algorithms can be limited by time or space or both. I think this one is time limited.

In any case, I intended this as a review for "users", not us techies who make (or used to make) our living in the cypher mines (a few hundred thousand lines of code in my day, but not game software).

* a naive approach might think you'd need all the games to be able to make sure no exact duplicates. But that'd scarcely be worth the time required to check against the small probablity of it happening and small damage if it does.

http://dvandva.org/pipermail/computer-g ... 00326.html


You are right that these programs do not save the playouts in memory. But the tree search portion can still grow very big very fast. 3GB in 30 seconds!

ETA: I don't really know how much impact it would have if you don't have enough memory. I think most of these programs won't actually die when you run out, they will just prune the tree. I would guess that as long as you have a modern system with 4GB or 8GB of RAM you would be ok. But if you're running on an old system with 1GB or less, just letting it run longer to compensate for being a slow computer might not help you a lot.

If we're talking about the software package, and not just the go-playing engine, then I think the interface is worth mentioning. For example, while I find MFOG's interface to be dated and somewhat ugly, the board is freely resizeable and its usability is overall pretty good. Crazy Stone is "prettier" and simpler (of course, it also has far few functions than MFOG), but it oddly limits the window size to three choices, which I find too small on my laptop (without my external monitor).

Try to keep this "user"

Do include in your reviews of specific programs "uses lots of memory" but don't assume this would be true for other programs. Emeraldemon is running fuego for a Malkovich giving it 30 minutes per move! (that's an awful lot of playouts) He doesn't seem to be having any memory problem. As a pro I would ask the Pachi crew what sort of an abstract data structure they are using for the tree. There obviously can't be more nodes than playouts if each playout adds a node and I wouldn't think it should take all that many bytes to store the data of a node.

Judicata, do you feel familiar enough with using Crazy Stone on your laptop to reveiw it?

And Emeraldemon, feugo?

I really haven't played it enough yet, but I'll be happy to review when I have.

The window size issue is really annoying. It offers, in theory, three "Window Sizes": Small (800x600), Medium (1024x768), and Large (1280x1024). I say "in theory" because the Large option is disabled when using my 1280x1024 external monitor. And the medium size is disabled on my 1366x768 laptop monitor (Thinkpad X220, FYI), which leaves me with a tiny board and actually affects my ability to focus. If the window is actually larger and it disables the higher resolutions to protect me from myself (i.e. having the window extend beyond the viewable screen), then shame on the program designers.
[EDIT: See update and work-around here. ]

As for the actual substance of the program: I've won and lost on both level 6 and level 7 (out of 10), I haven't tried 8-10 yet, and haven't played many games. It also seems to seek control of the center, at almost any cost (perhaps by design to capitalize on its fighting ability).

EDIT: Although I haven't provided much info yet, here is some relevant system info: Intel Core i5 2.5ghz, Windows 7 64-bit, 4gb Ram.

I believe Feugo and most all modern MCTS programs, when they run out of memory start pruning the tree. So yes the program will continue to run, but there could be an impact to the playing strength of the program. I don't have much experience with Fuego, but I saw some instructions for compiling it that gave a switch to allow you to use ~3.5GB of memory instead of "only" 2GB. http://fuego.sourceforge.net/fuego-doc- ... mingw.html

I guess my point is that reviews should include how much memory the machine has and/or how much they gave to the program. For that matter it should include all settings relevant to playing strength.

ETA: I should have posed all these memory issues as questions. I haven't used these programs much myself, I'm just an interested observer. I saw a few references saying they can use a lot of memory, but I have no idea how much difference it makes if they are limited to less.

For future references --- in a case like this were there is a very large range of crunch power within a processor family perhaps better to specify in terms of number of cores and speed of yours rather than just a family designation (the most powerful i5's and i7's are several times more powerful than the weaker ones).

Again, let's try to keep this "user" and not speculate on details of implementations. There are usually possible tradeoffs between speed and space and a wise choice of structures can possible greatly reduce space and whether structures no longer in use are collected or abandoned (and again that is a tradeoff). Leave those things for us "pros" to discuss.

I'm not entirely sure what you mean, but I'll speculate on anything I think may be useful for people reading a review. Are you referring to my speculation that Crazy Stone is designed to seek center influence to capitalize on its fighting ability? If so, I respectfully disagree--a layperson is in a position to offer such speculation. I try to make it clear that I'm speculating, and comments are always offered simply for what they're worth.

Of course people shouldn't pretend to be an expert on something they're not, but anyone from novices to hobbyists to professionals can comment on what they observe and speculate. Also, I have a propensity for anti-elitism, so your comment may have hit a nerve .

I added my system info as an afterthought because people seem interested in that. I don't know how it affects the program. Here's some additional info: Intel Core i5 2520m 2.5GHz, 2-core.

Ok, my Fuego review:

Availability: Fuego is free and open source, with semi-active development: (http://fuego.sourceforge.net/). On the other hand, it currently requires building from source, which I guess puts it out of reach for many. If the phrase "./configure; make; make install" has meaning to you, you should be fine, although you will also have to install boost if you don't have it already.

GUI: Fuego uses Go Text Protocol (GTP), so it can be used with any GUI that knows that protocol, but they suggest downloading the also free GoGui program (http://gogui.sourceforge.net/). GoGui is a java app that should just run without problems for most users. If you use GoGui, Fuego can take advantage of some special features like setting Fuego's options through menus & buttons rather than command line, and showing the variations it considers in real-time, which I find interesting. Otherwise the UI is nice but unremarkable, I'd say.

Hardware: there is an option to set the amount of memory fuego uses, and Fuego will use up everything you give it before it starts pruning. If your computer grinds to a halt, try lowering the memory allocation. I haven't done any tests, but since more memory means a larger search tree, it's safe to assume the program will be weaker the less memory it gets. It is multi-threaded, so it can take advantage of your 2 or 4 core machine to compute faster.

Like many (most? all?) computer programs, Fuego only understands area scoring. It also only understands Canadian byo-yomi. Fuego will use the time you give it, so you can adjust playing strength somewhat by increasing or decreasing the time settings. If you don't set time, Fuego defaults to 10 sec/move. There isn't a "difficulty" parameter, as some programs have.

Now what people really care about: how it plays. It is much stronger on smaller boards than 19x19: if I had to guess I would say something like 5k-1k on 19x19, 1d-4d on 9x9. It's a bit difficult to judge strength, since to my knowledge there isn't any rated Fuego bot anywhere. I set up an unrated bot to play some on KGS, and 4k-5k seemed about right for the medium-fast settings I was using (25/5:00 Canadian byo-yomi).

A few words about the style: it only cares about winning and losing, not point differential. If Fuego thinks it is far behind, it will make increasingly desperate moves and then resign. This may be frustrating for beginners who want to play a game to the end. Similarly when it is ahead it will play protective moves that reduce its lead but ensure the win, so losing by 0.5 or 1.5 against Fuego is pretty standard.

As for usage, I primarily play it on 9x9 with fast-ish settings when I want to kill some time & practice my reading. I typically take black and a reverse komi and it's a good challenge for me.