godiscussions.com archive

tapir · #1

I am aware that l19x19 has the archive of godiscussions.com, however I am not aware whether it is accessible as it was announced it would be. Did I miss something or is it just lost?

I guess most forum regulars don't care about an archive anyway, however at SL there are still many defunct links to godiscussions.com threads.

p2501 · #2

I'd second that.

xed_over · #3

there are several people who have a copy of the sql database dump, but we just need to force ourselves to be industrious enough to figure out how best to present the data.

probably a simple php based site. I don't know php well enough myself, but maybe there's an off-the-shelf solution that'll allow browsing the archive with reviving full forum features (I haven't looked)

tapir · #4

xed_over wrote:

there are several people who have a copy of the sql database dump

I know, but after two years it is probably safe to say, it won't happen, right?

The benefit of a godiscussions.com archive decreases the later it goes online as the number of cross-references will decline steadily. It isn't about the best presentation, but the archive being available and linkable at all. If the people, who have a dump, can't overcome technical difficulties, they could have asked for help within the last two years. I never noticed anything like that, so it seems to be very low priority to all concerned.

jts · #5

I don't think that's exactly fair to xed_over &co. At least one thread from GD has been posted directly into a L19 thread. The issue comes up a few times a year, and the reply is always the same: "We haven't figured out an easy way to do it yet, we need skills that we don't have." I'm not sure how else you want him to ask for help... stand on a street corner, begging passers-by for assistance?

xed_over · #6

Yes, everytime I work on this I get a little further, but then something else comes up and gets in the way -- juggling too many simultaneous tasks all the time (work, family, hobbies -- not necessarily in that order).

Its always in my mind to come back to this (and also my Dosaku thread I started)

I'm still not convinced the archive is complete, and I'm still having trouble tying some of the tables together.

For example, two of the most interesting tables are vb_post and vb_forum, but I can't find a common link between them.

I spent some more time on this today...

Here's the top level view...
mysql> select forumid, title, parentid, parentlist, threadcount, childlist from vb_forum order by forumid;

Code:

+---------+--------------------------------+----------+------------+-------------+------------------------------------+
| forumid | title                          | parentid | parentlist | threadcount | childlist                          |
+---------+--------------------------------+----------+------------+-------------+------------------------------------+
|       3 | GoDiscussions.com              |       -1 | 3,-1       |           0 | 3,14,16,22,8,9,10,11,12,24,-1      |
|       4 | Go Club Forums                 |       -1 | 4,-1       |           0 | 4,33,35,36,13,34,32,30,31,15,28,-1 |
|       5 | Go Gear                        |       -1 | 5,-1       |           0 | 5,21,18,23,19,-1                   |
|       6 | Improve Your Game              |       -1 | 6,-1       |           0 | 6,17,20,29,-1                      |
|       8 | General Go Chat                |        3 | 8,3,-1     |        2099 | 8,-1                               |
|       9 | Beginners                      |        3 | 9,3,-1     |         615 | 9,-1                               |
|      10 | Amateurs                       |        3 | 10,3,-1    |         563 | 10,-1                              |
|      11 | Professionals                  |        3 | 11,3,-1    |         274 | 11,-1                              |
|      12 | Forum Suggestions &amp; Bugs   |        3 | 12,3,-1    |         285 | 12,-1                              |
|      13 | Requests                       |        4 | 13,4,-1    |          18 | 13,-1                              |
|      14 | Introductions &amp; Guidelines |        3 | 14,3,-1    |         524 | 14,-1                              |
|      15 | Knoxville Go Club              |        4 | 15,4,-1    |           0 | 15,-1                              |
|      16 | Off Topic                      |        3 | 16,3,-1    |         708 | 16,-1                              |
|      17 | Game Analysis                  |        6 | 17,6,-1    |        1432 | 17,-1                              |
|      18 | Computer Go                    |        5 | 18,5,-1    |         658 | 18,-1                              |
|      19 | Trading Post                   |        5 | 19,5,-1    |         233 | 19,-1                              |
|      20 | Study Group                    |        6 | 20,6,-1    |         782 | 20,-1                              |
|      21 | Go Books                       |        5 | 21,5,-1    |         551 | 21,-1                              |
|      22 | Announcements                  |        3 | 22,3,-1    |         674 | 22,-1                              |
|      23 | Gobans &amp; Other Equipment   |        5 | 23,5,-1    |         583 | 23,-1                              |
|      24 | Moderator's Room               |        3 | 24,3,-1    |         117 | 24,-1                              |
|      28 | Columbus iGo Club              |        4 | 28,4,-1    |           1 | 28,-1                              |
|      29 | Teachers/Club Leaders          |        6 | 29,6,-1    |         197 | 29,-1                              |
|      30 | New York Go Center             |        4 | 30,4,-1    |           0 | 30,-1                              |
|      31 | Chicago Go Clubs               |        4 | 31,4,-1    |           0 | 31,-1                              |
|      32 | Go Center of Second Life       |        4 | 32,4,-1    |          33 | 32,-1                              |
|      33 | Go Club Discussions            |        4 | 33,4,-1    |         125 | 33,-1                              |
|      34 | KGS                            |        4 | 34,4,-1    |         171 | 34,-1                              |
|      35 | American Go Association Forum  |        4 | 35,4,-1    |          53 | 35,36,-1                           |
|      36 | Tournament Rides/Hosts         |       35 | 36,35,4,-1 |           7 | 36,-1                              |
+---------+--------------------------------+----------+------------+-------------+------------------------------------+
30 rows in set (0.00 sec)

mysql> describe vb_forum;

Code:

+-------------------+-----------------------+------+-----+----------+----------------+
| Field             | Type                  | Null | Key | Default  | Extra          |
+-------------------+-----------------------+------+-----+----------+----------------+
| forumid           | smallint(5) unsigned  | NO   | PRI | NULL     | auto_increment |
| styleid           | smallint(5) unsigned  | NO   |     | 0        |                |
| title             | varchar(100)          | NO   |     |          |                |
| title_clean       | varchar(100)          | NO   |     |          |                |
| description       | text                  | YES  |     | NULL     |                |
| description_clean | text                  | YES  |     | NULL     |                |
| options           | int(10) unsigned      | NO   |     | 0        |                |
| displayorder      | smallint(6)           | NO   |     | 0        |                |
| replycount        | int(10) unsigned      | NO   |     | 0        |                |
| lastpost          | int(11)               | NO   |     | 0        |                |
| lastposter        | varchar(100)          | NO   |     |          |                |
| lastthread        | varchar(250)          | NO   |     |          |                |
| lastthreadid      | int(10) unsigned      | NO   |     | 0        |                |
| lasticonid        | smallint(6)           | NO   |     | 0        |                |
| threadcount       | mediumint(8) unsigned | NO   |     | 0        |                |
| daysprune         | smallint(6)           | NO   |     | 0        |                |
| newpostemail      | text                  | YES  |     | NULL     |                |
| newthreademail    | text                  | YES  |     | NULL     |                |
| parentid          | smallint(6)           | NO   |     | 0        |                |
| parentlist        | varchar(250)          | NO   |     |          |                |
| password          | varchar(50)           | NO   |     |          |                |
| link              | varchar(200)          | NO   |     |          |                |
| childlist         | text                  | YES  |     | NULL     |                |
| importforumid     | bigint(20)            | NO   |     | 0        |                |
| importcategoryid  | bigint(20)            | NO   |     | 0        |                |
| lastpostid        | int(10) unsigned      | NO   |     | 0        |                |
| showprivate       | tinyint(3) unsigned   | NO   |     | 0        |                |
| defaultsortfield  | varchar(50)           | NO   |     | lastpost |                |
| defaultsortorder  | enum('asc','desc')    | NO   |     | desc     |                |
| lastprefixid      | varchar(25)           | NO   |     |          |                |
| imageprefix       | varchar(100)          | NO   |     |          |                |
+-------------------+-----------------------+------+-----+----------+----------------+
31 rows in set (0.08 sec)

mysql> select sum(threadcount) from vb_forum;
+------------------+
| sum(threadcount) |
+------------------+
| 10703 |
+------------------+
1 row in set (0.02 sec)

All the posts are in the vb_post table...
mysql> describe vb_post;

Code:

+----------------+----------------------+------+-----+---------+----------------+
| Field          | Type                 | Null | Key | Default | Extra          |
+----------------+----------------------+------+-----+---------+----------------+
| postid         | int(10) unsigned     | NO   | PRI | NULL    | auto_increment |
| threadid       | int(10) unsigned     | NO   | MUL | 0       |                |
| parentid       | int(10) unsigned     | NO   |     | 0       |                |
| username       | varchar(100)         | NO   |     |         |                |
| userid         | int(10) unsigned     | NO   | MUL | 0       |                |
| title          | varchar(250)         | NO   |     |         |                |
| dateline       | int(10) unsigned     | NO   |     | 0       |                |
| pagetext       | mediumtext           | YES  |     | NULL    |                |
| allowsmilie    | smallint(6)          | NO   |     | 0       |                |
| showsignature  | smallint(6)          | NO   |     | 0       |                |
| ipaddress      | varchar(15)          | NO   |     |         |                |
| iconid         | smallint(5) unsigned | NO   |     | 0       |                |
| visible        | smallint(6)          | NO   |     | 0       |                |
| attach         | smallint(5) unsigned | NO   |     | 0       |                |
| importthreadid | bigint(20)           | NO   |     | 0       |                |
| importpostid   | bigint(20)           | NO   | MUL | 0       |                |
| infraction     | smallint(5) unsigned | NO   |     | 0       |                |
| reportthreadid | int(10) unsigned     | NO   |     | 0       |                |
+----------------+----------------------+------+-----+---------+----------------+
18 rows in set (0.43 sec)

mysql> select count(*) from vb_post;
+----------+
| count(*) |
+----------+
| 143572 |
+----------+
1 row in set (0.00 sec)

mysql> select count(distinct threadid) from vb_post;
+--------------------------+
| count(distinct threadid) |
+--------------------------+
| 10539 |
+--------------------------+
1 row in set (1.19 sec)

But the thread counts don't match, and I can't yet figure out how to list the threads by forum.

The users table has only 12 records (looks like admins and mods only).
I don't know where the forum users table is yet.

mysql> show tables;

Code:

+----------------------------+
| Tables_in_godiscussions    |
+----------------------------+
| bios                       |
| blogs                      |
| comments                   |
| friends                    |
| notes                      |
| product_images             |
| products                   |
| products_tags              |
| reviews                    |
| tags                       |
| users                      |
| vb_access                  |
| vb_adminhelp               |
| vb_administrator           |
| vb_adminlog                |
| vb_adminmessage            |
| vb_adminutil               |
| vb_album                   |
| vb_albumpicture            |
| vb_albumupdate             |
| vb_announcement            |
| vb_announcementread        |
| vb_attachment              |
| vb_attachmentpermission    |
| vb_attachmenttype          |
| vb_attachmentviews         |
| vb_avatar                  |
| vb_bbcode                  |
| vb_bookmarksite            |
| vb_calendar                |
| vb_calendarcustomfield     |
| vb_calendarmoderator       |
| vb_calendarpermission      |
| vb_cpsession               |
| vb_cron                    |
| vb_cronlog                 |
| vb_customavatar            |
| vb_customprofilepic        |
| vb_datastore               |
| vb_deletionlog             |
| vb_discussion              |
| vb_discussionread          |
| vb_editlog                 |
| vb_event                   |
| vb_externalcache           |
| vb_faq                     |
| vb_forum                   |
| vb_forumpermission         |
| vb_forumprefixset          |
| vb_forumread               |
| vb_groupmessage            |
| vb_groupmessage_hash       |
| vb_groupread               |
| vb_holiday                 |
| vb_humanverify             |
| vb_hvanswer                |
| vb_hvquestion              |
| vb_icon                    |
| vb_imagecategory           |
| vb_imagecategorypermission |
| vb_impexerror              |
| vb_infraction              |
| vb_infractionban           |
| vb_infractiongroup         |
| vb_infractionlevel         |
| vb_language                |
| vb_mailqueue               |
| vb_moderation              |
| vb_moderator               |
| vb_moderatorlog            |
| vb_notice                  |
| vb_noticecriteria          |
| vb_noticedismissed         |
| vb_passwordhistory         |
| vb_paymentapi              |
| vb_paymentinfo             |
| vb_paymenttransaction      |
| vb_phrase                  |
| vb_phrasetype              |
| vb_picture                 |
| vb_picturecomment          |
| vb_picturecomment_hash     |
| vb_plugin                  |
| vb_pm                      |
| vb_pmreceipt               |
| vb_pmtext                  |
| vb_pmthrottle              |
| vb_podcast                 |
| vb_podcastitem             |
| vb_poll                    |
| vb_pollvote                |
| vb_post                    |
+----------------------------+
92 rows in set (0.00 sec)

I'm probably missing something obvious. Suggestions welcome.

A couple of well placed, simple sql queries, and it actually should be pretty simple to post an archive on the web. (or recreate some threads here)

Yeah, I did recreate one thread here (I can't remember where it is now), and if there's a specific thread search someone wants, I can probably do the same thing for them too.

hyperpape · #7

A text (HTML? BBCode?) dump of all forums would be interesting even if nothing else can be done.

Actually, for finding references, grep often beats a web interface.

LocoRon · #8

xed_over wrote:

I'm still not convinced the archive is complete, and I'm still having trouble tying some of the tables together.[...]
The users table has only 12 records (looks like admins and mods only).
I don't know where the forum users table is yet.[...]
I'm probably missing something obvious. Suggestions welcome.

Maybe Don removed all the other users (either from the users table that now only has 12 records, or their own table, if they were separate for some reason), to protect their personal data? Although, I suppose he'd have mentioned if he did something like that....

tapir · #9

jts wrote:

I don't think that's exactly fair to xed_over &co. At least one thread from GD has been posted directly into a L19 thread. The issue comes up a few times a year, and the reply is always the same: "We haven't figured out an easy way to do it yet, we need skills that we don't have." I'm not sure how else you want him to ask for help... stand on a street corner, begging passers-by for assistance?

I am not following everything going on here closely, so I may have missed on a couple of threads. I am in no position to help either. For me the practical question is whether I should remove all those defunct links from SL or whether it is worthwile to wait a little longer.

And yes, I would expect more asking around than mentioning it in a low-volume L19x19 thread when additional skill is needed.

xed_over · **#10**

try browsing these thread titles for starters

http://www.eskimo.com/~wheelers/godiscu ... hreads.txt

Dusk Eagle · **#11**

tapir wrote:

jts wrote:

I don't think that's exactly fair to xed_over &co. At least one thread from GD has been posted directly into a L19 thread. The issue comes up a few times a year, and the reply is always the same: "We haven't figured out an easy way to do it yet, we need skills that we don't have." I'm not sure how else you want him to ask for help... stand on a street corner, begging passers-by for assistance?

I am not following everything going on here closely, so I may have missed on a couple of threads. I am in no position to help either. For me the practical question is whether I should remove all those defunct links from SL or whether it is worthwile to wait a little longer.

And yes, I would expect more asking around than mentioning it in a low-volume L19x19 thread when additional skill is needed.

I haven't tested myself to see if it would work, but could you link to the Internet archive for GD threads?

Bonobo · **#12**

Dusk Eagle wrote:

I haven't tested myself to see if it would work, but could you link to the Internet archive for GD threads?

I just tried, and while I seem to get the front page correctly (no idea what used to look, though), but when I clicked a thread link (I think for a page from 2010) I got

Quote:

Page cannot be crawled or displayed due to robots.txt.

http://web.archive.org/web/201004302217 ... post150090

xed_over · **#13**

Dusk Eagle wrote:

I haven't tested myself to see if it would work, but could you link to the Internet archive for GD threads?

do you mean the database?

at the moment, its running on my local laptop.

and there will be security issues with making it publicly available without first scrubbing some personal data and such (which would take me even longer to figure out).

Dusk Eagle · **#14**

Bonobo wrote:

Dusk Eagle wrote:

I haven't tested myself to see if it would work, but could you link to the Internet archive for GD threads?

I just tried, and while I seem to get the front page correctly (no idea what used to look, though), but when I clicked a thread link (I think for a page from 2010) I got

Quote:

Page cannot be crawled or displayed due to robots.txt.

http://web.archive.org/web/201004302217 ... post150090

Right, I remember now that I tried it in the past, but got this error message as well.

godiscussions.com archive

Who is online