Life In 19x19
http://www.lifein19x19.com/

Round Robin: SODOS or Direct comparison?
http://www.lifein19x19.com/viewtopic.php?f=45&t=999		 Page 1 of 3
Author:  willemien [ Wed Jun 30, 2010 3:45 pm ]
Post subject:  Round Robin: SODOS or Direct comparison?
While studying Round robin tournaments (see other tread in this forum)

I discovered something curious

Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?

Example a 6 players round robin tournament
Code:
Round 1    A wins from B : C wins from D : E wins from F
and so on see table:
 
           Opponents
          a b c d e f  Win Sodos  dc
Player A  - W L W W W   4    -   
Player B  L - W W L W   3    6    W
Player C  W L - W W L   3    8    L
Player D  L L L - W W   2    3    W
Player E  L W L L - W   2    4    L 
Player F  L L W L L -   1    -


A is the clear winner with 4 wins

But who is second?
B won the game between B and C
But C has a higher Sodos(Sum Of Defeated Opponents Scores) so had his wins against bigger odds

so player B is the Direct comparison 2nd place
and player C is the Sodos 2nd place

Or more concrete
Does a win against the winner/ better player (A) count for less than a win betweem tied players (B & C) ?

A similar problem occurs between D and E and also here the same question is at stake.

My provisional personal opinion is that C should get the second place cup but i am willing to chance my mind with good arguments.
Author:  Bill Spight [ Wed Jun 30, 2010 4:32 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
willemien wrote:
While studying Round robin tournaments (see other tread in this forum)

I discovered something curious

Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?

Example a 6 players round robin tournament
Code:
Round 1    A wins from B : C wins from D : E wins from F
and so on see table:
 
           Opponents
          a b c d e f  Win Sodos  dc
Player A  - W L W W W   4    -   
Player B  L - W W L W   3    6    W
Player C  W L - W W L   3    8    L
Player D  L L L - W W   2    3    W
Player E  L W L L - W   2    4    L 
Player F  L L W L L -   1    -


A is the clear winner with 4 wins

But who is second?
B won the game between B and C
But C has a higher Sodos(Sum Of Defeated Opponents Scores) so had his wins against bigger odds

so player B is the Direct comparison 2nd place
and player C is the Sodos 2nd place

Or more concrete
Does a win against the winner/ better player (A) count for less than a win betweem tied players (B & C) ?

A similar problem occurs between D and E and also here the same question is at stake.

My provisional personal opinion is that C should get the second place cup but i am willing to chance my mind with good arguments.


To me the question is, given the results, who is more likely to win a game between B and C? (Assuming equal odds a priori.) You can approach this in Bayesian terms, OC. :) But off of the top of my head, here are some thoughts.

Both B and C beat D, so D is irrelevant. Both form cycles with A and E. ( C>A>B>C>... and C>E>B>C>....) So maybe they are irrelevant, too. That leaves B>F>C. The fact that F "lies between" B and C bolsters the notion that B would win a rematch. :)

Edit: The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it? :mrgreen:
Author:  pwaldron [ Wed Jun 30, 2010 5:01 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
willemien wrote:
Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?


This question is known as the ranking problem and has been flogged many times over in the statistical literature. A variety of tiebreaking methods have been proposed, including SODOS, sometimes iterated to infinite order. Others have suggested ordering the players so that the tournament minimizes the number of upsets; in some cases this would place a player with 3 wins behind a player with 4 wins. As Bill suggests, Bayesian methods are also possible.

About the only method that hasn't been proposed is the direct confrontation because it relies on the result of a only single game.
Author:  willemien [ Wed Jun 30, 2010 5:14 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
Quote:
The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it? :mrgreen:


No not really C is 2 sodod points ahead of B below where neither player loses against F

The cycles elimination always gives the same result as direct comparison.
( a list of C > ?? > B can always be extended to C > ?? > B > C and become a cycle so C can never win)




An example where neither loses to F

Code:

           Opponents
          a b c d e f  Win Sodos  dc
Player A  - W L W W W   4    -   
Player B  L - W W L W   3    6    W
Player C  W L - L W W   3    7    L
Player D  L L W - W L   2    5    W
Player E  L W L L - W   2    4    L
Player F  L L L W L -   1    -
Author:  Bill Spight [ Wed Jun 30, 2010 5:38 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
willemien wrote:
Quote:
The fact the the bottom player (F) beat C but not B is what gives C a better SODOS score, isn't it? :mrgreen:


No not really C is 2 sodod points ahead of B below where neither player loses against F


I was pointing out the fact that I was arguing that B>F>C bolstered the notion that B would win a playoff, while that very fact along with the fact that F was the bottom player, was used to argue that C should win the tie break. I thought it funny that the very same fact was used to support opposite conclusions. :mrgreen:
Author:  pwaldron [ Wed Jun 30, 2010 6:01 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
For what it's worth, I ran this tournament through the old AGA Bayesian rating system. The results put A > B > C > D > E > F. The margin between B & C and D & E is awfully small though, well below the sigma.
Author:  Harleqin [ Thu Jul 01, 2010 3:01 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
SODOS does not have a legitimation as tiebreaker in a round robin setting, because having won against stronger opponents means having lost against weaker ones. In summa, two players with equal wins have gained them against the same field of opponents, so there is no distinction to be made there.

Direct comparison ignores cycles. The best that can be done in this direction seems to be Bill's cycle elimination method. This does not always break ties, though.

In my opinion, if you need a clear winner or even a clear total ordering, do not play round robin.
Author:  topazg [ Thu Jul 01, 2010 3:04 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
willemien wrote:
Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?


There is no "better" tiebreaker, merely one that achieves the objectives of your tiebreaker system.

As a general rule with these two in my practical experience, SODOS rewards more strongly the results against higher ranked players, and will often match head to head for the top of the draw, and tiebreak in the other direction at the bottom of the draw. If you argue that the worse performing player should be rewarded for beating a high performing player more than he is punished for the loss against his tied player, then use SODOS. If you think that in all tiebreaks, "who won between them" is the most logical tiebreak, use that instead.

At least with a single round robin, head to head tiebreak always breaks ties aside from jigos.
Author:  topazg [ Thu Jul 01, 2010 3:07 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
Harleqin wrote:
SODOS does not have a legitimation as tiebreaker in a round robin setting, because having won against stronger opponents means having lost against weaker ones. In summa, two players with equal wins have gained them against the same field of opponents, so there is no distinction to be made there.


Define legitimation. It has been used with happiness in chess for over 100 years, and still people are not complaining about it much in top level chess tournaments. That, in my mind, is practical legitimation.

There are plenty of good reasons why someone may choose to dislike the logic SODOS uses to break ties, but that's not the same as the system objectively not being a legitimate tiebreaker. As you say, in a rr setting, winning against stronger opponents means losing against weaker ones (to still end up tied). So, if you want to reward the wins against stronger players more than you punish against weaker player losses, SODOS is an excellent solution. It depends what the tournament director wishes to reward with regards to performance criteria.
Author:  Harleqin [ Thu Jul 01, 2010 3:25 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
topazg wrote:
Define legitimation. It has been used with happiness in chess for over 100 years, and still people are not complaining about it much in top level chess tournaments. That, in my mind, is practical legitimation.


Well, I am talking about theoretical legitimation. People also happily play lottery.

Quote:
So, if you want to reward the wins against stronger players more than you punish against weaker player losses, SODOS is an excellent solution.


Winning and losing are perfectly symmetric. Why should winning against the first place be valued more than losing against the last one?
Author:  RobertJasiek [ Thu Jul 01, 2010 3:43 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
Needless to say, not breaking ties is the best. Second best is playing an equal number of games among the tied players until the tie will be broken. Third best is playing an equal number of games among the tied players until the tie will be broken, but with short(er) thinking times. Don't anybody tell me that 5 minutes games could not be done! And if the tied players won't solve the tie after 3 extra 5 min. thinking time tournaments, then in God's sake let them remain tied! They have proven to be of equal strength so let them be equal!

However, if tiebreaking by numbers is set as a requirement, then first of all critiria of aims expressing the purpose of tiebreaking should be specified. (There is the problem, of course, then one can reverse engineer suitable aims to justify any specific tiebreaker...)

My major concern with SODOS esp. in round-robin is the question "Why not SOLOS instead?" and it would inverse the order. I do not have the same concern as to the complement of a direct comparison tiebreaker because I am strongly impressed by the reasoning that, since the tied players are to be compared, their games against each other are of greater interest than their games against third players.

There are valid reasons supporting opinions against every numeric tiebreaker. Is MutualGameScore (applied to only exactly 2 tied players, else the tie is kept) a tiebreaker that, although one can state reasons against it, at least might have one advantage not seen in any other tiebreaker? The advantage of being the one and only numeric tiebreaker that is socially acceptable for everybody because it is the minimal consensus of all opinions on numeric tiebreakers? Or is there anybody with a strong objection?

Concerning the thread's topic more strictly, in round-robin I prefer DC very much over SODOS, mainly for the stated reason. (I have other reasons, which I would need to dig out, but their importance pales in comparison to the stated reason.)
Author:  topazg [ Thu Jul 01, 2010 3:58 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
Harleqin wrote:
Winning and losing are perfectly symmetric. Why should winning against the first place be valued more than losing against the last one?


Because some people may choose that the values to be assigned to wins and losses are to be asymmetric. There is no requirement of symmetry on the point of rewarding performance criteria. That you feel they should be symmetric is reasonable, but it is not inherently the nature of winning and losing.
Author:  Matti [ Thu Jul 01, 2010 4:36 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
pwaldron wrote:
For what it's worth, I ran this tournament through the old AGA Bayesian rating system. The results put A > B > C > D > E > F. The margin between B & C and D & E is awfully small though, well below the sigma.

I did not get any margin between B & C or between D & E donw to seven digits.
Author:  zinger [ Thu Jul 01, 2010 5:23 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
In my opinion, head to head (American term for direct comparison) should always be the first tiebreaker, if tiebreakers are used at all.

This in particular seems just silly to me:

Bill Spight wrote:
To me the question is, given the results, who is more likely to win a game between B and C?

Err ... they already played each other.

I would add, that I think tiebreakers should only be used for first place. Beyond that there is no need.
Author:  topazg [ Thu Jul 01, 2010 5:37 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
I assumed Bill's question was effectively more of a "if they played each other 10,000,000 times, who would win" question, rather than an individual match.
Author:  RobertJasiek [ Thu Jul 01, 2010 7:19 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
In a context of an assumed possibility of up to an infinite number of games...?

topazg, please explain "symmetric [...] is not inherently the nature of winning and losing"! A win is defined to equal 1 tournament score point, a loss 0. Now I think one can derive symmetry. Would you want to assign other values instead, like +5 versus -1?
Author:  topazg [ Thu Jul 01, 2010 7:42 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
RobertJasiek wrote:
In a context of an assumed possibility of up to an infinite number of games...?

topazg, please explain "symmetric [...] is not inherently the nature of winning and losing"! A win is defined to equal 1 tournament score point, a loss 0. Now I think one can derive symmetry. Would you want to assign other values instead, like +5 versus -1?


Firstly, don't confuse a postulated hypothesis that I put forward with anything I want or a personal opinion of mine. To answer "would I want" would not give you the answer you are looking for.

Secondly, +5 and -1 is just as symmetrical as +1 and 0 - provided in the case of jigos each player receives 2 points. If jigos do not exist, the symmetry will obviously still remain.

It should be very clear some tiebreak systems do not view wins and losses as equal value. SODOS disregards losses completely, and rewards wins against higher performing players to a greater extent than wins against lower performing players. The symmetry here is completely broken - however, that doesn't make the system bad or unfair. If this is the metric of performance that the tournament rules organiser wishes to embrace, then it is the correct system for that tournament. Likewise, as you said, SOLOS will have a different effect, but again is not inherently bad.

A good tiebreaker is one chosen specifically for the advantages and disadvantages it has within the context of the aims of the scoring system, and provided it is explicitly explained to those participating in a way that can be understood (and some of all the scores of the people you beat isn't hard!) the choice can be considered correct.

I agree with you that additional games to break ties are a solution with minimal flaws, but I think they should be played at the same time controls as the main games in the tournament ideally, which takes considerable time (I think this is the only great disadvantage to the system).
Author:  fwiffo [ Thu Jul 01, 2010 8:26 am ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
You can arrive the same SODOS score by taking the sum of the scores of all of a player's opponents, then subtracting the scores of opponents a player lost to. When looked at this way, losses to a strong opponent appear to carry a large penalty, and losses to weak opponents carry a smaller penalty. This seems illogical.

I propose a trivial symmetric alternative - sum the scores of defeated opponents, then subtract an inverted score for the victorious opponents.

Something like:
TIEBREAK := sum(DOS) - sum(MAX-VOS)
( where DOS is each defeated opponent's score and VOS is each victorious opponent's score and MAX is the maximum possible score )

That can be rewritten as:
TIEBREAK := sum(DOS) - (NL*MAX - sum(VOS))
( where NL is the total number of losses )

Apply a little basic algebra:
TIEBREAK := sum(DOS) - NL*MAX + sum(VOS)

Since sum(DOS)+sum(VOS) is just the sum of ALL opponents' scores, it will be the same for all players, and we can just drop it. Likewise, MAX is a constant multiplier which is the same for everyone, so we can eliminate that factor as well, leaving just TIEBREAK := NL.

Thus, we can break ties in the number of wins by simply considering the number of losses. My system is awesome.
Author:  willemien [ Thu Jul 01, 2010 3:21 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
pwaldron wrote:
willemien wrote:
Sometimes Direct Comparison and SODOS give different results, and therefore the question arises What is the better tiebreaker?


This question is known as the ranking problem and has been flogged many times over in the statistical literature. A variety of tiebreaking methods have been proposed, including SODOS, sometimes iterated to infinite order. Others have suggested ordering the players so that the tournament minimizes the number of upsets; in some cases this would place a player with 3 wins behind a player with 4 wins. As Bill suggests, Bayesian methods are also possible.

About the only method that hasn't been proposed is the direct confrontation because it relies on the result of a only single game.


I have heard of the problems with tiebreaking in in swiss and mcMahon tournaments (especialy the combination McMahon/ sodos is fraud with problems)

but i always thought in Round Robin these problems don't exsist and that Sodos and DC always would give the same outcome,
And this was the first time i was confronted with that it for a 2nd place. (for first places DC and sodos always have the same winner , except that sodos sometimes doesn't break the tie)


Please post some links where the ame problem is described then i can have a look how they solve it.
Author:  Javaness [ Thu Jul 01, 2010 4:01 pm ]
Post subject:  Re: Round Robin: SODOS or Direct comparison?
The OOF tiebreaker should be used
Page 1 of 3 All times are UTC - 8 hours [ DST ]
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/