When chess computers for the mass public first came out, I bought one that was highly rated (in both senses) and thoroughly enjoyed it at first, as it gave me a good game. Then I discovered its quirks, and soon it became possible to demolish the machine easily in every single game and even give it a large handicap. In go terms, if it was 1-dan when I bought it, it was 6-kyu once I got used to it.

I later went on to work on a shogi computer with some of the best brains in the computer chess business. I learned that my experience was normal, and indeed I learned further tricks so that my chess computer was more like 9-kyu now.

When our shogi computer was finished and we took it to Japan, claiming tongue in cheek that it was close to 1-dan, we even beat a pro on 4 pieces and got a front-page coup. But we knew the pro was being very nice to us - even we knew how to beat the machine on even bigger handicaps.

Back in the chess world, huge advances were later made, and of course Kasparov lost a famous match. But Garry complained bitterly at the time and ever after that the Deep Blue team refused to let him see examples of its previous games. The assumption is that the team knew that they could be beaten once its quirks were known.

I expect the same pattern to apply to computer go. I don't accept the MoGo and Zen ratings except as short-term indicators (and, yes, impressive ones at that). Once they go commercial I expect their grades to drift down far and swiftly, just like my first copy of MFOG.

Looking at the chess experience, though, should we now expect vast improvement? I'm not sure about that. Computers are good at storing vast opening lines. Great for chess, but not a major factor in go, perhaps? Computers are good at analysing fights. This is where they really dominate at chess nowadays, but in go you can often avoid fights, and the board is big enough to allow a lost fight to become a sacrifice with compensation elsewhere. (Calculating ability might, hwoever, work well in go for the endgame, where most of us are weak.)

On the whole, therefore, I would expect that people who buy a commercial copy of Zen or Mogo would be disappointed with their purchase in a matter of weeks, and I don't think we are going to see a genuine challenge to even a strong amateur for a very long time. I actually hope I'm wrong, but experience seems to count for something.

We may get a clearer view in London this Christmas. There is to be a ?ten-game match between John Tromp (a 1 or 2-dan, I think) and the best computer program as chosen by a computer person - for money! I think John Tromp's money is safe, although he may get bitten in the bum in the first game or two if he hasn't seen that program before.

I thought that I discussed that already by referring to the two stats. Though I did so only by name (and not by rating). ManyFaces1 (at 10 seconds/move) has a somewhat higher rating than ManyFaces (30 minutes for the moves). A strong 1 dan vs a weaker 1 dan. The program can't benefit from "more time to think" as well as a human can. The algorithm does do better with more time but not proportionally so (it doesn't "scale" with time). Related to "sample size" in probability where the lay person often asks "why didn't they use a larger sample?" (they used a size such that it would take a gigantic increase in the size of the sample to increase the certainty of the result by a meaningful amount -- the point iof diminishing returns).

BUT (a very big but) it seems that from the point of view of the creators/vendors of such software optimizing for relatively quick play is crucial. The customers don't seem to want slow playing. In direct answer to your question, it would play weaker compared to the human if the human made good use of the additional available time (able to recognize should stop and think at this point -- one of my current problems).

In regard to the broader question, can software like this be used as a learning tool I think how used may be crucial. If grossly ahead will play overly safely. If grossly behind overly recklessly. So you want to establish initial conditions where the program will be slightly ahead (take a slightly inadequate handicap if weaker or give is a slightly excessive handicap if you are stronger).

What John just posted is relevant but more a matter of whether critical deficiencies can be remedied as fast as discovered or not. While game playing software not my "line of country" I have been watching the rate of improvement over time and it has been impressive. The issue isn't whether a release quickly jumps up a level and then begins declining as some loophole or other is discovered by its opponents as much as whether the developer analyzing games of this period of decline can identify the deficiency and find some "fix" for it. We humans "learn" from our mistakes; the program needs to be altered to do so (although not applicable here, that could be automatic if an AI of the "neural net" sort which can "learn" and "forget").

Sounds like the bots are Korean.

This is correct.


Is this a question?

_________________
- Bantari
______________________________________________
WARNING: This post might contain Opinions!!

Not exactly. Here comes the corresponding question:
How do your claim and the non-existence of "Bantari"
in the KGS archives fit?

I am just wondering why a KGS participant (like you)
writes long postings with questions about the strength
of a bot instead of simply playing this bot on KGS.

Ingo.

Please, Ingo, or whatever your real name is - do not make this thread about me.
I will try to answer your questions this time, out of politeness, but I do not see why do you expect me to justify myself to you. I don't know you, and you come across as being slightly too aggressive in your attempts to redirect the topic of this thread and focus on me rather than the actual subject. Actually, I do have some suspicions as to who you are... funny, really.



I am not sure what archives you are referring to.
I have an account Bantari on KGS, as anybody can verify by just checking the user info.
I do not play with that account, its just for socializing.
Sad precaution from my IGS days, you understand.

Not sure what else you are fishing for.



The bot I have seen only plays 10sec/move games, which I am not very interested in.
If you read my posts carefully, you would have realized that my questions was how this speed-play rating translates into slower game rank. This question is mostly targeted at dan players owning MFoG. Do you belong to that group?

_________________
- Bantari
______________________________________________
WARNING: This post might contain Opinions!!

Hello Bantari,



My handle is my name from real life (except for o-Umlaut).
Now, I have also given the address of my website
in my profile here.



Thanks.



Perception is subjective, on all sides.

I got a feeling of aggressiveness on your side, when
I read your first posting in this thread. Here is
the passage which increased my blood pressure.

Bantari wrote:


To give you my observation on the strength of Many Faces:
* It is KGS 1-dan at the 10sec/move level.

* It is slightly lower (perhaps by half a degree)
at slower levels like 30sec/move.

* It would be slightly stronger (perhaps also by half a degree)
without the ladder crap it produces from time to time.
When playing against ManyFaces you will realize that
it is not so easy to provoke "strange ladder" situations.



I meant
http://www.gokgs.com/gameArchives.jsp



At slower times Many Faces is slightly weaker, as seen
from other MF accounts where it plays (sometimes) at
30 sec/move or 30 min/game.



No. But over the years I have collected a good
amount of experience, concerning strengths and
weaknesses of go bots.

Ingo.

I do not belong to that group (I'm not a dan player). But I will try (again) to explain some things about the behavior of MFOG and other programs using the same basic algorithm.

You question is too specific to how you want to use the program and too specific to your particular situation for answers to be general statements about the usefulness of this sort of go playing software. The performance of algorithms of this class varies* with the "state of the game". Will do very well if the state of the game favors the computer (it's chances are >50%) and poorly otherwise. I think that is what makes it less useful to you, but wouldn't necessarily make it unuseful to people willing to use it under the proper conditions.

So when you ask "is it really 1 d" you mean specifically "will it usually play as well as a 1 d if playing against a human 1 d opponent even?" That is a different question from whether it will play as well as a 1 d human would against an opponent needing 3-4 stones agaist a 1 d or whether it will play as well as a human 1d would when taking a handicap against a stronger human opponent. Especially if in the former case the handicap is just a tad too little or in the latter a tad too much. And I stress that usually (for the "playing even" situation). Some of the games would be satisfactory (in the way you want) but others not.

You also ask about the effect of different time controls. Compared to the human it will do better at uniformly short time provided that's enough to get the "sample size" adequate. It isn't playing objectively weaker when given more time but can't benefit as much as the human can and so comparatively weaker. Not by a huge amount.

So to the general question "is this sort of software useful?" I'd say depends on who you are and how are you trying to use it. I think it's very useful for a player weaker than 1 d as a learning tool, strong enough to punish mistakes. ROFLOL but how it plays provides me with a fairly reliable indication of the state of the game. If I see it begin to make some unnecessary safety plays then I know that perhaps contrary to my own evaluation, it is safely ahead and will win the game. If I see it making senseless overplays the reverse (but sometimes I've missed something vital and those aren't actually overplays!). So I find it useful, but I'm not a 1 d player (need 4 stones against MFOG12 to have any chance, 5 to have good chances, crush it at 6) --- btw, that's playing at 1 hour for all the moves using a 2 core 2+ gHz machine.

* That is the way in which most different from a human player whose performance level isn't going to be so drasticly dependent on "the state of the game".

I got a registration key for MFOG 12 as a prize for winning a local tournament. I've only played a handful of games with it, time settings 15 minutes, 5/30. In the one even game, at first it seemed to be a reasonable facsimile of AGA 2-3 kyu, but somewhere in the middle game (neither side having a significant advantage in my view), it started playing bizarre responses to my moves, and completely collapsed. The other games I gave it 9 stones, just to see if I could win (I couldn't), and 3 stones, where I might have won if I hadn't been playing late at night and just messing around.

I wonder whether they'd be more fun to play against if there were a "play to maximize expected points" option (instead of "play to maximize expected winning probability"). Should that be easy for developers to implement?

It might even make them stronger in general: waiting for an opponent to make a mistake can be better than making ridiculous overplays, at least for sufficiently ridiculous values of ridiculous.

That's what the previous generation of bots did and it only got them so far. The monte carlo algorithm changed that by focusing on winning probability and suddenly beat all the old bots. It's not possible to change MC to maximize the win (and perform better than last generation).

Btw, what would be the biggest win? Killing all enemy stones of course. Try that yourself in your next 10 games and watch your rank plummet

_________________
FusekiLibrary, an opening library.
SGF converter tools: Wbaduk NGF to SGF | 440 go problems | Fuseki made easy | Tesuji made easy | Elementary training & Dan level testing | Dan Tutor Shortcut To Dan

Thanks, this is interesting. Maybe there's some intermediate though?

For example: What if you use the current "winning probability" method, but each turn, adjust komi to the value such that you think your current winning probability is roughly 50%, and then make the move that has the highest winning probability.

Variants:

-Only let the komi float in one direction (e.g. still play stodgily when you're ahead as current bots do, but let the komi float if you're behind).

-Put a cap on how far you'll let the komi float. (Maybe different caps in the two directions, generalizing the previous point.)

Does this do anything?

Edit: More variants:

- You probably don't want to lose the edge computers have at the endgame (over similarly ranked humans), so e.g. let komi float less as you get to the endgame, or only start the komi floating if your expected win percentage gets rather low (i.e. the range where computers do strange things).

Lindentree --- What version (update) are you running? And what are you running the program on? (machine power). If the version of MFOG12 you downloaded was the original may need some of the fixes. I think the Feb 14th build is the latest version (you don't need to redo the key to do this uninstall/reinstall). Also machine power is critically connected to the time setting for the machine if machine power isn't above some threshhold. That's probably why you are allowed to set seperate time controls for the players. For the program to play as strongly on this old desktop (far weaker than "standard" for this program) as on one of my laptops (which are "standard" for the program) I'd have to quadruple its time allowance.

I have the latest version, 12.020, running it on a laptop with a dual-core 2.0 Ghz processor. I probably should test it on different time settings/handicap, but honestly I have better things to do. I'm glad I didn't pay for it; the extra features aren't worth it, although I'm going through the (substandard) problem collection as a supplement to my daily L&D regimen.

Zen19N (the one that plays 30+5x:30) is on KGS now, played and won 2 games as a 1d so far.

ETA: Looks like it is playing 20+5x:30 now. Still very reasonable time controls for online games.

Try a time setting of 30 minutes, 5/30 and see what happens. Or 60 minutes (for all).

This sort of algorithm is very non-linear with time (because the reliability of statistics relative to sample size is very non-linear). Your computer power is "standard" for the program and you have the current version so those aren't the problem. But most of these go playing programs look at how much time they are being allowed and adjust their sample size accordingly. Not quite enough time and performance will deteriorate rapidly (but above a certain threshhold improvement with more time slight).

You should be seeing better than ~2-3k AGA. I'm running MFOG12 on a comparable machine and I certainly would be noticing if not doing significantly better that that (I need the same handicap against MFOG 12 that I do against a human 1d AGA).

Kasparov played very badly in his match against Deeper Blue, but he, or any other world champion, would certainly be annihilated in a match now against the top computer chess programs in the world..At the time, I don't think IBM chose not to agree to a rematch because they were afraid of losing, it was more likely that they had nothing to gain by a rematch.

I don't think UCT based engines are as easy to exploit as the old generation of computer programs. Zen is a commercial available program already, I haven't see anyone on KGS systematically trashing it. I would back the computer over Tromp myself. Since I won't be able to watch the match properly, I doubt I will have a bet on it though.

Speed game is not computer advantage, it's human fault.
Computer can play 1sec/move or less while human can't.
If you think speed game is not fair to human then give computer infinite time.
Of course those matches are not 'enjoyable' by your taste but in computer there's no 'enjoyable' or 'unenjoyable' time.

ROFLOL --- Well not infinite time. The whole point of the problem not being exactly solvable is that the solution has to be within some finite amount of time.

I think we need to pay some attention to the particular time settings that the developers of this sort of software choose to have their bots play at. They probably know where, given the speed of the machine used, the points of diminishing returns lie. Also of course related to the willingness of human users to agree to playing at those time controls.

The machines used make a big difference which you can roughly translate into the time the machine is allowed (only roughly since there is overhead combining the results from multiple processors). On one hand I do have interest on how well these programs can perform on massive machines but more interest in their practical strength (running on a machine of reasonable power that an end user could be expected to possess).

Thus if the overhead is ~20%, the the program running on a thousand core supercomputer at 10 seconds/move might equate to the same program running on a 2 core machine at an hour per move. Which should be an indication how severe the diminishing returns are (how small the benefit from additional time above some critical amount).