It is currently Thu May 01, 2025 5:19 am

All times are UTC - 8 hours [ DST ]




Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4

How long until a bot reaches consistent 6d on KGS?
Poll ended at Fri Jul 01, 2011 2:33 pm
<3 months 0%  0%  [ 0 ]
<6 months 13%  13%  [ 4 ]
<1 year 50%  50%  [ 15 ]
<2 years 13%  13%  [ 4 ]
<3 years 10%  10%  [ 3 ]
<5 years 0%  0%  [ 0 ]
never 7%  7%  [ 2 ]
The Terminator Skynet takes over the world first. 7%  7%  [ 2 ]
Total votes : 30
Author Message
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #61 Posted: Fri Jun 03, 2011 6:44 am 
Gosei
User avatar

Posts: 2116
Location: Silicon Valley
Liked others: 152
Was liked: 330
Rank: 2d AGA
GD Posts: 1193
KGS: lavalamp
Tygem: imapenguin
IGS: lavalamp
OGS: daniel_the_smith
iazzi wrote:
daniel_the_smith wrote:
Even when chess programs say something like "the rook on the 7th makes this a winning position", they say it because they've done all the reading. A rook on the 7th in a slightly different position might *not* be winning.


However this is false. Computer programs are not able to do all the reading even when there are about ten pieces (including kings) on the board. They use the above heuristic to actually say that the position is good enough and needs no more evaluating, exactly like a human would do. Would you lose a queen blindly? No. You would consider it a lost position and not read the variation any longer. So does a computer, it uses heuristics to avoid reading.


But that's not what I meant. By "all the reading", I do not mean "every line, all the way to the end of the game"-- that would be silly and impossible given modern hardware and the known physical limits on computation. I mean "every reasonable line, far enough to see which is best with a high probability". I know how alpha-beta searches work. :) (Although modern chess engines don't use alpha-beta any more, but the idea is the same.)

(On a side note, computers don't, strictly speaking, do any reading when there are < a few pieces left, they use pre-calculated endgame tables. The effect is the same, though.)

(Even when solving games, they'll stop reading a line if they can prove that it's worse than another line.)

_________________
That which can be destroyed by the truth should be.
--
My (sadly neglected, but not forgotten) project: http://dailyjoseki.com

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #62 Posted: Fri Jun 03, 2011 6:51 am 
Honinbo

Posts: 10905
Liked others: 3651
Was liked: 3374
Mike Novack wrote:
Fotland claims MFOG 12.022 running on a 2 core ~2 GHz machine at a time control of 60 minutes for all moves should be about 1 dan against humans. Now I'm not personally strong enough to test that but at our local go club a 7 dan gave it 6 stones and a 5 dan 4 stones and both lost to the computer and they judged, "yes, 1 dan". Interesting to note they felt an unbalanced 1 dan, weaker than that strategically but stronger tactically; not unlike what is typical of a go prodigy when a young child.


Are you sure about what is typical of young future pros?

Not that I know, but Sakata said that his teacher, Masubuchi, would set up whole board positions and ask him where he would play. He would answer and she would smile and pat him on the head. It sounds like he had a pretty good strategic sense. :)

_________________
The Adkins Principle:
At some point, doesn't thinking have to go on?
— Winona Adkins

Visualize whirled peas.

Everything with love. Stay safe.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #63 Posted: Fri Jun 03, 2011 7:52 am 
Gosei
User avatar

Posts: 1744
Liked others: 704
Was liked: 288
KGS: greendemon
Tygem: greendemon
DGS: smaragdaemon
OGS: emeraldemon
hyperpape wrote:
emeraldemon wrote:
Someone mentioned trying to do a Malkovich style with a bot... I don't have Zen19, But Fuego is pretty decent and will produce playouts of its thought process (actually w/ GoGui you can watch it consider moves in real time, it's pretty interesting). If someone was interested we could try to set up a game where I just let fuego think about the move for 1 hour or so, then put its "thoughts" in hide tags. Playing short time games on my hardware, I've found it to be a bit weaker than me, but not much, maybe 4k or 3k. I don't know what would happen if there's a longer time limit.

Is anyone interested in challenging a bot to a Malkovich?
This is a nice idea, but how far is Fuego from the top bots?


I'm not really sure, like I said I think its probably around 3k on my machine, so it's definitely not 5d. But it might still be interesting just to see the style and what things cause it trouble. I don't know about Zen or Many Faces, but Fuego loves to play random center moves in the opening (like 8-9 point) then connect everything and form a super moyo. Its fighting is pretty strong, but sometimes it seems to be confused by ko. It loves tennuki and loves to take sente. If it's winning it will play safety moves, if it's losing it will attack aggressively. It can be a bit annoying sometimes because in a lost game it may play 5-10 desperate invasion type moves, like it's hoping you'll miss an atari :-?

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #64 Posted: Fri Jun 03, 2011 8:03 am 
Gosei
User avatar

Posts: 2060
Location: Texas
Liked others: 546
Was liked: 173
Rank: KGS 3k
GD Posts: 264
KGS: Chew
emeraldemon wrote:
hyperpape wrote:
emeraldemon wrote:
Someone mentioned trying to do a Malkovich style with a bot... I don't have Zen19, But Fuego is pretty decent and will produce playouts of its thought process (actually w/ GoGui you can watch it consider moves in real time, it's pretty interesting). If someone was interested we could try to set up a game where I just let fuego think about the move for 1 hour or so, then put its "thoughts" in hide tags. Playing short time games on my hardware, I've found it to be a bit weaker than me, but not much, maybe 4k or 3k. I don't know what would happen if there's a longer time limit.

Is anyone interested in challenging a bot to a Malkovich?
This is a nice idea, but how far is Fuego from the top bots?


I'm not really sure, like I said I think its probably around 3k on my machine, so it's definitely not 5d. But it might still be interesting just to see the style and what things cause it trouble. I don't know about Zen or Many Faces, but Fuego loves to play random center moves in the opening (like 8-9 point) then connect everything and form a super moyo. Its fighting is pretty strong, but sometimes it seems to be confused by ko. It loves tennuki and loves to take sente. If it's winning it will play safety moves, if it's losing it will attack aggressively. It can be a bit annoying sometimes because in a lost game it may play 5-10 desperate invasion type moves, like it's hoping you'll miss an atari :-?


If it's interesting to you enough to do, I'll take on you and your robot army! I'd be curious to see it's 'thought processes'.

Here to take my stand against Skynet.

_________________
Someday I want to be strong enough to earn KGS[-].

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #65 Posted: Fri Jun 03, 2011 9:12 am 
Gosei
User avatar

Posts: 1744
Liked others: 704
Was liked: 288
KGS: greendemon
Tygem: greendemon
DGS: smaragdaemon
OGS: emeraldemon
Alright lets do it! Do you want me to make a thread?

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #66 Posted: Sat Jun 04, 2011 12:08 am 
Gosei
User avatar

Posts: 1639
Location: Ponte Vedra
Liked others: 642
Was liked: 490
Universal go server handle: Bantari
snorri wrote:
Bantari wrote:
Seriously...
I have not played against the Zen 5d program, but I did a few games against some other program claiming to be 1d/2d and few months back and let me tell you - this programs were nowhere near dan! I think the trick is to play very fast games and capitalize of humans making really silly mistakes, especially kyu players and players not used to fast games. When I observe the Zen program play right now, it looks like it makes nothing more than average moves, but it makes them consistently, without major blunders - within the time limits. I would be very interesting to see how this software would fare against a reasonable dan player, in even game, with longer time controls.


So what? Can't the exact same comment be made about some human players? How many KGS 5ds hold those ratings entirely through blitz and would not be able to do so in OTB longer games? If so, it seems a little unfair to blame the people running these bots for adopting a strategy that's been used for years by humans. :D

As for winning while playing only average-looking moves, this is also what was said of Lee Changho. It can hardly be considered an insult.


Who said anything about 'blaming'? Or 'insult'?
You take this too personally, dude.

Of course - you are right, there are 5d players out there who are only 5d because of specific trickiness they adopt, like super fast games, or only playing weaker players and then bullying them with trick moves, and so on... But this does not change the fact that there are also honest-to-god real 5d human players out there. And better than 5d as well. So saying 'Humans have reached 5d and better' would be justified.

On the other hand - as far as I can tell, computer programs have ALL reached their dan level ratings by using such 'special strategies'. And this goes especially for the bots claiming to be high dans. So I do not think it is justified to start yelling euphorically 'YAY! COMPUTERS HAVE REACHED 5D!' This has nothing to do with placing blame or with diminishing the cummulative achievements all the various programmers.

Oh... and I did not say 'average looking' moves. I said *average* moves. There is a difference when you think about it a little. Or even a lot, if you need.

_________________
- Bantari
______________________________________________
WARNING: This post might contain Opinions!!

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #67 Posted: Sat Jun 04, 2011 6:52 am 
Lives in gote

Posts: 335
Location: Germany
Liked others: 41
Was liked: 97
GD Posts: 351
Bantari wrote:
On the other hand - as far as I can tell, computer programs have ALL reached their dan level ratings by using such 'special strategies'. And this goes especially for the bots claiming to be high dans.


It's partly off-set by players who only win against the bots by knowing and exploiting the specific weaknesses of the bots. Zen19D would probably play at 6D KGS if it wasn't for this.

There is also a Zen19 version (Zen19N) that plays at 20 minutes main time, 5x30s, and it reached 3D on KGS at those settings, which contradicts the claim that bots only succeed at blitz settings.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #68 Posted: Sat Jun 04, 2011 10:33 am 
Lives in sente

Posts: 1045
Liked others: 0
Was liked: 182
Mivo wrote:
It's partly off-set by players who only win against the bots by knowing and exploiting the specific weaknesses of the bots. Zen19D would probably play at 6D KGS if it wasn't for this.

There is also a Zen19 version (Zen19N) that plays at 20 minutes main time, 5x30s, and it reached 3D on KGS at those settings, which contradicts the claim that bots only succeed at blitz settings.


On the second statement -- those arguing "bots only succeed at blitz settings" are folks who jump from "relatively stronger against the human at blitz" to "much stronger (relatively) at blitz than more normal play speed" without looking at the empirical evidence. As you have pointed out, if we look at that evidence the difference is between 1 and 2 stones for the bots for which we have any evidence.

On the first, is it at all clear that all bots have the sort of weaknesses which can be played against? Having a weakness doesn't necessarily mean an easily exploitable weakness or a different problem than when playing against a human opponent who has that style of play.


This post by Mike Novack was liked by: Li Kao
Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #69 Posted: Sat Jun 04, 2011 5:45 pm 
Lives in gote
User avatar

Posts: 324
Location: Dresden
Liked others: 26
Was liked: 22
Rank: KGS 7 kyu
KGS: Mnemonic, dude13
Mike Novack wrote:
On the first, is it at all clear that all bots have the sort of weaknesses which can be played against? Having a weakness doesn't necessarily mean an easily exploitable weakness or a different problem than when playing against a human opponent who has that style of play.

As much as I might agree with you on your other posts, I believe that bots have easily exploitable weaknesses than humans. Bots are easily confused by non standard play because they rely more on databases/josekis/standard sequences than humans. At a given level their reading is usually 1 to 2 stones weaker than that of a human and so their responses to unusual play are suboptimal. Example.

_________________
While I was teaching the game to a friend of mine, my mother from the other room:
"Cutting? Killing? Poking out eyes? What the hell are you playing?"

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #70 Posted: Sun Jun 05, 2011 4:21 am 
Lives in sente

Posts: 1045
Liked others: 0
Was liked: 182
Mnemonic, that is a perfect example of the problem when we say "bots are" or "bots do"

gnugo is a program using both an AI move generator and an AI evaluator (pick the best of the lot and make that move). At the current state of the art, none of the all AI programs are even close to the strength of the programs using some variant of MCTS as the evaluator (I'm not going to discuss how they select the set of plausible moves to evaluate).

What we observe with regard to one bot cannot be blindly attributed to other bots. When we do we say silly things.

Please, look for yourself (try out one of the MCTS programs) and see if you still see an easily exploitable weakness. Do they have peculiarities of play? Yes, but that doesn't mean easily exploitable. For example, if the peculiar style of an opponent is "leave positions unfinished, keep the game complicated and "busy" how would you exploit that? To be sure, try to simplify. But note that were you playing a handicap game against a stronger (human) opponent you'd be faced with the same problem.

Reading? The MCTS programs are doing nothing of the sort. Not in the sense that humans "read out" a position. In both experiments where we tried MFOG with the appropriate handicap for a 1 dan against high dan humans at my local go club in both cases it was precisely by ingeneous counter attack based on over the whole board aji that the program surprised them when they thought they were safely ahead.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #71 Posted: Sun Jun 05, 2011 5:59 am 
Lives in gote

Posts: 335
Location: Germany
Liked others: 41
Was liked: 97
GD Posts: 351
Mike Novack wrote:
Mnemonic, that is a perfect example of the problem when we say "bots are" or "bots do"


His example nevertheless illustrates his (and my) point. In case of Zen19D, you can look at the recent KGS games by Liisa, who (as 1d) wins against Zen19d because of the free placement of the handicap stones in a way that would be unlikely to work against a human 5d player (certainly not more than once), because it's pretty "bad". However, it works well against Zen19d because of the software's high center moyo priority. Software is simply more predictable than a human opponent. Or rather, it tends to be far more consistent and, at least currently, doesn't actually learn from mistakes or inefficient responses to specific, unusual situations. Humans can't compete with a machine's computing power, but they excel at adapting to changing and new conditions.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #72 Posted: Sun Jun 05, 2011 10:18 am 
Lives in sente

Posts: 1045
Liked others: 0
Was liked: 182
Mivo wrote:
His example nevertheless illustrates his (and my) point. In case of Zen19D, you can look at the recent KGS games by Liisa, who (as 1d) wins against Zen19d because of the free placement of the handicap stones in a way that would be unlikely to work against a human 5d player (certainly not more than once), because it's pretty "bad". However, it works well against Zen19d because of the software's high center moyo priority............


Sorry, but what are you seeing in the results of those few games between Zen19d and Liisa that is inconsistent with their relative ranks? If we suppose Lissa to have been a high 1 dan (a reasonable assumption since currently 2 dan) and Zen19d a lowish 5 dan then we would expect Liisa to win more than 50% at a four stone handicap.

But if allowed, free placement of handicap stones might be a useful trick. Would force the bot to play without its fuseki book and that probably would weaken it somewhat (the MCTS type algorithms don't work at their best till some number of stones on the board). How much weaker? Again that would be an empirical question but I doubt as much as one stone. Easy for the Zen folks (or the team of any other book using MCTS program) to measure (just try it "book empty").

BTW -- if you think the MCTS programs have a "high center mojo strategy" or for that matter any strategy then you don't understand what this algorithm is doing (or not doing). If these programs usually play as if they had a "high mojo strategy" all that means is that usually playing that sort of move from the current position is the most probable road to victory. If, given the current position, a tight territorial move is the most probable road to victory, then that's the sort of move the program would make.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #73 Posted: Tue Jun 07, 2011 10:19 am 
Lives in gote
User avatar

Posts: 476
Liked others: 193
Was liked: 83
Rank: Dutch 2 dan
GD Posts: 56
KGS: hopjesvla
Mike N., how do you know these things about Zen? As this particular bot is so much stronger than any other bot I have ever seen, it would not surprise me in the slightest if its makers included some novel tricks in the program... I have not seen any specifics posted about Zen except that it's apparently based on MCTS, but I do not find it unthinkable that its playouts have a built-in preference for moves with thick center influence.

_________________
My name is Gijs, from Utrecht, NL.

When in doubt, play the most aggressive move

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #74 Posted: Tue Jun 07, 2011 1:42 pm 
Lives in sente

Posts: 1045
Liked others: 0
Was liked: 182
Because of finite time, the fundamental algorithm gets cut off before "finished", so yes indeed, you can certainly bias the result (move chosen) by biasing in favor of some moves or other at the start of the process. To turn that around, what strength penalty is MFOG for its strong bias toward "human like" play*.

But why do you say "so much stronger"? How much do we assume a factor of several times as much computing power is worth? How much have we already determined a difference in time control is worth? (I know of at leaset one clear example with the same program, same machine, different time controls).

Net difference appears to be a stone or two between the top programs.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #75 Posted: Mon Jun 13, 2011 9:21 am 
Lives in gote

Posts: 653
Location: Austin, Texas, USA
Liked others: 54
Was liked: 216
yoyoma wrote:
Zen19D - 5d - playing with 9x:15
Zen19N - 3d - playing with 20minute main + 5x:30

Also Zen19N was playing on a less powerful machine, and it hasn't been around since April.

Update:
Zen19S - 4d - playing with 20minute main + 5x:30

Zen19S is the same software and hardware as Zen19D, only difference is time controls.

Zen19S wrote:
Q: What is Zen19S?
A: Zen19S is a version of Zen19, runs on a mini-cluster of 6 pcs (a 6-core Xeon W5680/4 GHz, two 4-core i7 920/3.2 GHz, and three 4-core Core2Quad/3 GHz) connected via a GbE LAN. Although Zen19 is a prototype of a commercial product, Zen19S is being developed for academic research by team DeepZen, a joint project of ZenAuthor and me. #Same as Zen19D except time setting.

Q: What does S stand for?
A: Slow.

Q: How strong is it?
A: Estimated 5d for blitz and 4d for longer games.

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #76 Posted: Fri Jun 17, 2011 3:50 am 
Lives in gote

Posts: 355
Liked others: 52
Was liked: 43
Rank: AGA 2d
IGS: ethanb
The other interesting thing (to me) is that when you push the blitz settings past a certain value, the bots lose again. Intuition counts for a LOT in this game.

Now, I don't have 24 cores available at home, but I know that on my computer shrinking the time per move below 10 seconds gives me huge advantages against MoGo or GnuGo. Probably more parallelization would push the threshold down to a level where the advantage of intuition is outweighed by inability of the human to click fast enough, so this won't be universally applicable. :)

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #77 Posted: Fri Jun 17, 2011 3:55 am 
Tengen

Posts: 4382
Location: Caldas da Rainha, Portugal
Liked others: 499
Was liked: 733
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
Interesting tidbit: when Myungwan Kim played Mogo, giving it a 9 stone handicap, he used only 11 minutes of his time, and said that he would not have benefited from using more.

_________________
Occupy Babel!

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #78 Posted: Fri Jun 17, 2011 4:03 am 
Honinbo

Posts: 9552
Liked others: 1602
Was liked: 1712
KGS: Kirby
Tygem: 커비라고해
hyperpape wrote:
Interesting tidbit: when Myungwan Kim played Mogo, giving it a 9 stone handicap, he used only 11 minutes of his time, and said that he would not have benefited from using more.


That is very interesting. I wonder why. Maybe further time spent reading ahead would be spent reading variations that Mogo wouldn't likely come up with? I wonder what he meant.

_________________
be immersed

Top
 Profile  
 
Offline
 Post subject: Re: Computers reach 5d on KGS
Post #79 Posted: Sat Jun 18, 2011 6:39 pm 
Tengen

Posts: 4382
Location: Caldas da Rainha, Portugal
Liked others: 499
Was liked: 733
Rank: AGA 3k
GD Posts: 65
OGS: Hyperpape 4k
It's ambiguous: Mogo uses Kim's thinking time. So perhaps he reasoned that it would help the bot too much to have more time, even if he could play slightly better with more time.

But there was also a professional who said that when he played an amateurs, only his fingers got tired not his brain. That would imply that professional basic instinct is what defeats amateurs, in which case more time is almost irrelevant.

I suspect both factors matter a bit.

_________________
Occupy Babel!

Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 79 posts ]  Go to page Previous  1, 2, 3, 4

All times are UTC - 8 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group