Hi ZeroKun,
I travelled around Japan two years ago, and I visited quite a lot of go salons. I've included a list with short descriptions and photographs below:
TokyoNihon KiinMapQuite a few people have mentioned this already, it's easy to find, and there's always a ton of people to play with. Registration was 1000 JPY for one day. Most people playing here were - as is usually the case with go in Japan - elderly. The ranks at the Nihon Kiin are also legendary in their inflation. I played at around KGS 6-kyu level when I visited here, but an employee of the Nihon Kiin overseas division urged me to register as 1-dan. I only played two games against other 1-dan opponents, but I won both fairly easily. So keep this in mind when registering.

Takadanobaba Go ClubMapThis go club is run by Mr. Urasoe from the Nihon Kiin overseas division. He is a very friendly person, and very strong at go. (I think he takes two stones playing against a top pro. At any rate, I'm not strong enough to assess how strong he is.) Perhaps it's best to check at the Nihon Kiin first, as the club is rather hard to find (it's not too far from Takadanobaba JR station). I think they meet on Saturdays. This is a rare example of a vibrant go club with a lot of younger members. They're super-friendly, and I love the atmosphere there.

Ueno Go ClubThere used to be a large go club right next to ueno station. In fact, you can see the people playing go from the platform closest to the road between the station and Ueno park. I haven't got any pictures from here, but I loved playing there. Nice for combining go with a trip to one of the many museums in Ueno park.
OsakaKansai KiinMapWorth a visit if you're in Osaka. The general playing level seemed to be slightly higher here than in the Nihon Kiin. I entered the Kiin right as a class by a local pro came to an end, and got matched with the "weakest" player in the room. He still comfortably slaughtered me time and again. There's a little salon at the back with a comfy sofa and a television showing non-stop go.

Sekitoba Go SalonMapThis was my favourite place to play in Osaka. The go club is hidden on the top floor of a building in a busy shopping street. The owner was really friendly: he waived the entry fee as I was a traveling foreigner, and even presented me with a small book published by the salon, which had a detailed commentary on a game played by Takemiya Masakai, as well as some articles on the club and go in general.

HiroshimaMapI would definitely recommend playing go in Hiroshima. I was there in winter, but that didn't stop the locals from playing their daily games outside in the freezing cold. Firmly covered in multiple layers of clothing, these old men went at it with a vigor many of their young fellow countrymen would not be able to muster. It was so cold I could barely hold the stones half-way during the middle game, yet they played game after game. They tended to play very fast, and therefore sometimes made some silly mistakes for their general level. I loved the atmosphere. You can find the Hiroshima locals playing go in the park in front of the atomic bomb museum, near the atomic bomb dome.

NagasakiMapVery close to Dejima (the old artificial island and the place where the Dutch VOC continued their trade with Japan during the Edo Period) you can find this salon:


I didn't have time to play there, but the people were friendly.
KochiMapThere's a small branch of the Nihon Kiin in Kochi. When I went there, it was deserted (yet unlocked, so I sneaked in and took some pictures):

InnoshimaMapWhile not technically a go salon, you can play go at the Shusaku Memorial Hall on Innoshima. I loved the museum, and the trek up the cemetery to see Shusaku's grave. You might want to bring a go stone from your own bowl at home, which you can leave as an offer at his gravesite. It's a simple gesture, yet one I wish I would have thought of before I arrived there empty-handed and saw the small tin box with black and white stones in front of the grave.


I hope you have a great time in Japan!
Cheers,
Hush