Webcam + Go

[SebastianR]

Hi,
im currently developing a program that captures a live stream from a webcam and saves the recognized game as sgf file.
Anyone want to give it a try? Its not finished yet but i want to know if its working with other setups (cams, boards, light, etc) than mine.

thats what it looks like
http://www.youtube.com/watch?v=j-_l_dpzQNI

get it here and let me know if its working/crap/lacking features (sorry for the freehoster)
http://www.megaupload.com/?d=YO5ZSWH5

its written in c# so it requires windows (or wine - havent tried that) + .NET 2.0 Framework
also dont pay attention to the move i make, im not a go player myself - i barely know the rules

what you need to do
1. run the program
2. set up the camera. The board should fill the preview window (so dont place board and cam too far apart).
the camera must have a good look at the whole board, so dont place it too low (the angle between camera and pane of the board should be at least 45°).
3. shadows are a bit of a problem. If u use directed light the source should be on the same side as the camera.
But it probably works anyways if the shadows cast are not too sharp/dark
4. Place 4 black stones directly on the corners and wait for the program to make a latching sound.
5. youre free to play. the program makes a sound when it detects a new stone. if it dont make sure you place the stone directly on the line intersection


basti

[EdLee]

Sebastian, very nice! But at the end it says "this will only work for the first 100 moves or so"
Any chance you can improve it so that it records the entire game (sometimes 300+ moves)?
And have you tested "ko" situations and captures? Thanks!

the angle between camera and pane of the board should be at least 45°

What if the camera is set up directly overhead the board,
i.e. the camera is exactly orthogonal to the plane of the board? That'd be ideal, wouldn't it?
Like they do in Japanese, Korean, and Chinese TV live broadcast of professional games. For example:
www.youtube.com/watch?v=w3qphZ4wzpg

[Chew Terr]

Yeah, I'm really interested in this. When you said only the first 100 moves work, do you mean for recording, or do you mean that turning the board can only be re-recognized for the first 100 moves or so?

[SebastianR]

I see...that was ambiguous. I meant the automatic board recognition after turning will only work if there are enough of the lines to see (no too many stones placed)
Recording the game should work no matter how many stones are placed. Version 0.9 however will NOT SAVE MORE THAN 50 (or so - I forgot) MOVES.
This is because i havent yet figured out whether to sell it or not. Dont hate me for that

@EdLee
Yes that setup would be optimal. But i didn't expect everyone to be willing to have a cam dangeling over his head while playing. Also if its not fixed there somehow it will not work later on because sooner or later an airblow will move it - causing it to loose the board.

basti

[EdLee]

i didn't expect everyone to be willing to have a cam dangeling over his head while playing. Also if its not fixed

Yes, it would take some rigging equipment like a tripod
plus something else to firmly fix the camera overhead, and the result would be very nice.

[Chew Terr]

The easiest thing might be a playstation eye, since it comes with a little mounting platform that would give you a nice angle, and it's USB compatible. Assuming the camera can handle it, but I bet it can. It's essentially designed for that card game that does similar line recognition.

[Jordus]

But i didn't expect everyone to be willing to have a cam dangeling over his head while playing.

Answer: Duck Tape camera to forehead. No dangling over the head problem.....

[Phelan]

There was a thread about this type of thing in GD, I'll find it tomorrow.

[Chew Terr]

It's here. http://www.godiscussions.com/forum/show ... ght=camera

This seems more complete and usable, though.

Edit: downloaded it to check it out. Either my webcam wasn't being recognized or I do not know how to properly start recording in the program. My netbook does not have an available resolution setting that will show the whole program, so if there are buttons on the bottom, I cannot select them. Any suggestions for what I need to do?

[SebastianR]

@Chew Terr
Ahh thx for the link. Nothing new for me but that GregA outlined the problem pretty well. Although these things are easier said that done
Actually i consider my Program almost finished. But im eager to know if it works for other people.

[Phelan]

I'll hook up my webcam and try it when I have some time.

Edit: By the way, does it deal well with wobbling cameras? Imagine someone hitting the table the camera or the board is on, just enough to make it wobble, but not enough to actually dislodge stones.

[SebastianR]

Not sure if wobbling is any different than the rotation i showed in the youtube video.
as long as the program "sees" something that matches the game it has observed before (no known stone may be missing from the view)
nothing is going to happen. if the camera is shaken or fell down or the table moved the program will search (and usually find if the game has not advanced too far)
the board. just make sure the setup parameters are still met.

[Phelan]

Cool! I really have to try it then.

[GregA]

Very impressive! I've put my project on a hold a bit as my work has heated up lately, but I plan to put my code up on an opensource site like google code and do some more work on it. I've mostly focused on the problem of automatically detecting a board from a single image, regardless of how many stones are on it - and at as low of an angle as possible. As I'm sure you've noticed, it's not an easy problem. With lots of stones on the board, you have to rely on the edges of the board, and it's easy to find false edges. And at low angle, you have to be very accurate or it is very difficult to accurately find which stones are on which intersection.

[SebastianR]

Thx greg,
were you successful with that? Right now, I don't have any idea on how to recognize a board full of stones other than brute force.
And because the picture of the board is a "function" of 4 (important) variables (2 rotations and 2 translations) that would take forever.
And additionally, in very low angles half the lines - the horizontal - become useless because they have nothing to do with where the stone is seen. And
in resolutions for realtime image processing they are barely seeable even when the board ist empty. Also the stones become ellipses and when covered by other stones
"in front" of them their form is something even more irregular. I tried my program and it would work with angles as low as 25 to 30 degrees but only when properly set up (with stones in the corner).
But on the other hand, I have only experience with that one approach of detection...
Maybe your good with only taking the two most dominant lines, guessing the camera parameter and interpolating the rest of the grid