Life In 19x19 :: EGD machine friendly access

For testing, at least, you will want to be using YOUR OWN COPY of this database, or perhaps they can point you to a test copy of theirs. You DON'T test against "production", at least not in the early stages of development.

BTW -- the sort of goof using "direct access" when the volume meant you should have used "batch access" is the sort of thing "but it worked fine in testing" <<because in testing, your volume was tiny>> Emergency rewriting of a new application because the developers made this goof and when they went into production brought the database to it's knees (because of a huge increase in access transactions) one of the things I used to do.

Author:	tiger314 [ Sun Oct 04, 2015 12:34 am ]
Post subject:	EGD machine friendly access
Hi, I am a programming newbie so sorry if I don't get the obvious etc. I would like to play with the European database data a bit but cannot find a good way to extract tournament wall lists. Is there some way I can querry the database in the fashion "List tournament codes of all tournaments which ..." and "Send just the clean tournament wall list for the tournament code xxxxxxxx"? Thanks in advance

Author:	HermanHiddema [ Sun Oct 04, 2015 2:32 am ]
Post subject:	Re: EGD machine friendly access
Have you tried emailing the EGD developers? They have an email address for questions listed at the bottom of the website.

Author:	Mike Novack [ Sun Oct 04, 2015 7:15 am ]
Post subject:	Re: EGD machine friendly access
tiger314 wrote: Hi, I am a programming newbie so sorry if I don't get the obvious etc. I would like to play with the European database data a bit but cannot find a good way to extract tournament wall lists. Is there some way I can querry the database in the fashion "List tournament codes of all tournaments which ..." and "Send just the clean tournament wall list for the tournament code xxxxxxxx"? Thanks in advance Since you say you are a "newbie" let me point something out. You have to think about the behavior of the application you are trying to design. If it will be making only a few queries against the database, go after it normally. But if you think it might be making a lot of them (more than a couple percent of the database) it could be MUCH faster to first get a (flat file) copy of the backup of the database and extract your data en masse. CONTACT THOSE IN CHARGE OF THIS DATABASE. For testing, at least, you will want to be using YOUR OWN COPY of this database, or perhaps they can point you to a test copy of theirs. You DON'T test against "production", at least not in the early stages of development. BTW -- the sort of goof using "direct access" when the volume meant you should have used "batch access" is the sort of thing "but it worked fine in testing" <<because in testing, your volume was tiny>> Emergency rewriting of a new application because the developers made this goof and when they went into production brought the database to it's knees (because of a huge increase in access transactions) one of the things I used to do.

Author:	hyperpape [ Sun Oct 04, 2015 4:51 pm ]
Post subject:	Re: EGD machine friendly access
Mike Novack wrote: For testing, at least, you will want to be using YOUR OWN COPY of this database, or perhaps they can point you to a test copy of theirs. You DON'T test against "production", at least not in the early stages of development. BTW -- the sort of goof using "direct access" when the volume meant you should have used "batch access" is the sort of thing "but it worked fine in testing" <<because in testing, your volume was tiny>> Emergency rewriting of a new application because the developers made this goof and when they went into production brought the database to it's knees (because of a huge increase in access transactions) one of the things I used to do. Also, make sure to discuss the changes with all stakeholders, get risk assessments, six sigma certification, and dedicated test personnel. Create rollback plans, in case deployment goes badly, and hire several consultants. Jiminy cricket. Seriously, though: 1. Ask for a dump of the database, ideally in a textual format (good suggestion, Mike). 2. Probably don't get it. 3. Write some python to scrape the html (I'm not sure what people use for scraping, but I know some people swear by beautiful soup for parsing HTML). Be diligent about rate limiting your access! So to start: 1. Install python. 2. Install pip. 3. Install beautiful soup. 4. Starting figuring out how to use beautiful soup to get the data you need.

Life In 19x19 http://www.lifein19x19.com/

EGD machine friendly access http://www.lifein19x19.com/viewtopic.php?f=18&t=12323	Page 1 of 1

Page 1 of 1	All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group http://www.phpbb.com/