I'm using OpenCV, which gives a lot of useful computer vision algorithms. First I convert to gray, then dilate (to erase the lines on the board, actually), then canny to get the edges, then houghlines to find dominant lines. If I then pick the right 4 lines at this point, my code usually does fine, but sometimes there are a lot of extra lines that confuse things. My next step, actually, is to get a bunch of sample images and tune my algorithm for choosing the board boundary lines.SebastianR wrote:Thx greg,
were you successful with that? Right now, I don't have any idea on how to recognize a board full of stones other than brute force.
And because the picture of the board is a "function" of 4 (important) variables (2 rotations and 2 translations) that would take forever.
And additionally, in very low angles half the lines - the horizontal - become useless because they have nothing to do with where the stone is seen. And
in resolutions for realtime image processing they are barely seeable even when the board ist empty. Also the stones become ellipses and when covered by other stones
"in front" of them their form is something even more irregular. I tried my program and it would work with angles as low as 25 to 30 degrees but only when properly set up (with stones in the corner).
But on the other hand, I have only experience with that one approach of detection...
Maybe your good with only taking the two most dominant lines, guessing the camera parameter and interpolating the rest of the grid
Once I have the lines, I get the corners, pass that to GetPerspectiveTransform to get my transform, and WarpPerspective to give me a square image to check for stones.
There's a trick from one of the papers I mentioned in one of the threads on GD, where once you find the corners of the board, you then adjust them to get the corners of a rectangle floating roughly half a stone height above the board. There's still the issue of overlapping stones as you say - in the view warped to a square, stones stretch out and overlap quite a lot. Clearly at some angle between 45 and 0 degrees everything breaks down and difficulty of finding stones jumps to impossible.
At the moment, I ignore the lines entirely, and I try to ignore the stones too, when finding the board corners. This seems to me to be throwing out too much information. It seems to me an ideal algorithm would try to use all available information. I've been working on selecting lines using the knowledge that boards are roughly square, likely to be in the center of the image, and at an angle greater than 20 degrees or so (because I can't do anything with really low angle images, anyway). But it would be nice to also take into account the fact that boards tend to be roughly a specific color, with a black grid filling some portion of it. And the portions that don't have a grid should have filled ellipses that are either black or white. And the shape of the ellipses is tied to the angle, which, in turn, is tied to the specific boundary lines being considered - all in an interconnected feedback loop.
It would be nice to be able to combine this information somehow in a board probability map immage, where pixels that seem like part of a board are given a high score. This could then be processed with some mathematical morphology tricks to get areas that are likely to contain board edges. This would then be fed to the hough line selection code.
And then, after finding a best choice of boundary lines, the corners could be micro-adjusted. A given set of corners implies a grid floating above the board where stones could be found, and it implies a grid on the board where gridlines could be found. By shifting the corners around, you could explore the state space and try to improve the grid and stone detection until you reach the best fit.
At the moment, I do some brain-dead simple stone detection that is easily fooled by glare on the board - in the warped image, go to the expecte intersection point and search within a small circle, classifying pixels as white/black/board, and then pick the highest count. I'd like to make a stone detector that knows the camera angle and therefore the expected stone shape - it should also be able to know where the gridlines should be expected if there is no stone. Then there is whole-board knowledge, which would be nice to take into account - if there are some problematic intersections that I can't decide if they're white stones or open board, I should be able to compare with areas that I more easily detected either white stones or open board. There are a bunch of tricks to adjust for lighting artifacts, but unfortunately they involve looking at average brightnesses of larger areas. That seems like a big problem - white stones surrounded by black stones or white stones or board would have very different average regional brightnesses.
Anyway, I've been very busy with work, so I haven't been able to touch the code for a while, but I look forward to doing some more work on it - and putting it all up on some opensource hosting site for others to play with.
I think you stopped your project so maybe they can reborn with my help.