Hi all,
I've uploaded
WWW::KGS::GameArchives to CPAN.
This module enables you to send a query to the archives, and also parses the result into Perl data structure.
Although this module itself is not harmful, if you abuse this module, it can be harmful to the archives server.
See also
http://www.gokgs.com/robots.txt.
If you feel inconvenience to this module, please let me know. I'll improve it.
I don't intend to violate KGS's policy.
By the way, I think gameArchives.jsp should send the Last-Modified response header.
It would be useful for caching.
Enjoy!
Ryo
-----
EDIT:
WWW::KGS::GameArchives was renamed to
WWW::GoKGS::Scraper::GameArchives.
The
WWW::GoKGS distribution also provides scrapers which can scrape KGS Tournament pages.
The scrapers are tested by
Travis CI once a day.
EDIT2:
As of 0.12, WWW::GoKGS#user_agent defaults to
LWP::RobotUA which consults /robots.txt
before sending HTTP requests, and also set a proper delay between requests.
To use this module, the users must provide an email address which is used
to generate the From request header, while they can still set their own
user agent whenever they want.
NOTE: LWP::RobotUA fails to read /robots.txt on KGS since the web server
doesn't return the Content-Type response header. This module can not
solve this problem.