World rankings

[jts]

No, it's objectivity that's the problem, because there is no neutral way to decide how much to weight each dimension. Until you've deciding on a weighting of the dimensions, there's no unitary underlying phenomenon to measure, accurately or inaccurately; and there's nothing that makes one weighting of the dimensions more "objective" than another.

Then please explain how it is that people discover new dimensions to a phenomenon if the underlying phenomenon doesn't pre-exist the determination of its dimensions. It seems to me you're suggesting that someone conducting studies on "falling bodies" who discovers wind resistance is in actuality concocting a wholly new phenomenon rather than discovering a new dimension to the same phenomenon. What makes them think that it is relevant to falling bodies? Isn't it the fact that it clearly affects them? Can't this effect on falling bodies then be quantified and weighted according to that quantification?

Fortunately, you're asking about a literally one-dimensional problem. When we're investigating the rate at which bodies fall, we are only interested in their distance from the ground. That's one dimension. Lots of factors might go into figuring out that distance at a given t: the force of gravity, air resistance, air pressure, jet propulsion, etc. But it's one dimension, and once we've figured out the coordinate where a given set of falling bodies will be at t (and as you say, it may be difficult to do this accurately), it's trivial to rank the bodies according to their position.

But what if we were interested in distances from the ground, and the rates of change of those distances, and the rate of change of that rate of change? Now we have three dimensions of information about the falling bodies, and it's no longer possible to rank them -- unless they happen to fall in a line. This isn't troubling to us, because there's no one concept that x, dx/dt, and d^2/dt^2 all seem to be examples of. If we could get an ordinal ranking of a bunch of falling bodies along those three dimensions (in the case where they lined up perfectly), it's not clear what on earth that ranking would mean. But all of the games that pros play are clearly examples of "playing go", so it seems intuitive that all the different dimensions of the concept should line up nicely.

[hyperpape]

@jts In theory, its possible that there is an underlying factor that has high explanatory value in predicting success across the various playing conditions.  At some level this has to be true for Go.  While some professionals perform comparatively better at blitz or slow games, there's no such thing as a player who can win blitz tournaments, but wouldn't meet professional standards at slow play.

[jts]

@jts In theory, its possible that there is an underlying factor that has high explanatory value in predicting success across the various playing conditions.  At some level this has to be true for Go.  While some professionals perform comparatively better at blitz or slow games, there's no such thing as a player who can win blitz tournaments, but wouldn't meet professional standards at slow play.

Absolutely. The relationships between the dimensions are not wholly random. But unless the correlations are extremely tight, they're useless for establishing objective rankings - you still need to decide how much to weight each dimension. Even if all pros are about evenly matched in both blitz games and slow games, ranking systems that weighted blitz games twice as heavily, that weighted slow games twice as heavily, and that weighted them both equally could each give us completely different top ten lists. Since there are multiple different ways to weight the dimensions, there are multiple different possible rank orderings. This is true whether we're talking about best Go player, or best university, or best country, or whatever the flavor of the week is among people who love rankings.

[Monadology]

Fortunately, you're asking about a literally one-dimensional problem. When we're investigating the rate at which bodies fall, we are only interested in their distance from the ground. That's one dimension. Lots of factors might go into figuring out that distance at a given t: the force of gravity, air resistance, air pressure, jet propulsion, etc. But it's one dimension, and once we've figured out the coordinate where a given set of falling bodies will be at t (and as you say, it may be difficult to do this accurately), it's trivial to rank the bodies according to their position.

Then we are using the terms differently. In any case, I think Go is closer to falling bodies, which is why it seems more intuitively obvious as a phenomenon than the example you give later involving ranking objects according to a variable, the rate of change of that variable and the rate of rate of change together. The ranking is determined by a single measure: game results. Inasmuch as there are not any significant differences in scoring, this ranking can operate. Other conditions may vary, such as game length, location etc... just like air pressure may vary. Ideally these should be accounted for or, even more ideally, made equal.

[jts]

Fortunately, you're asking about a literally one-dimensional problem. When we're investigating the rate at which bodies fall, we are only interested in their distance from the ground. That's one dimension. Lots of factors might go into figuring out that distance at a given t: the force of gravity, air resistance, air pressure, jet propulsion, etc. But it's one dimension, and once we've figured out the coordinate where a given set of falling bodies will be at t (and as you say, it may be difficult to do this accurately), it's trivial to rank the bodies according to their position.

Then we are using the terms differently. In any case, I think Go is closer to falling bodies, which is why it seems more intuitively obvious as a phenomenon than the example you give later involving ranking objects according to a variable, the rate of change of that variable and the rate of rate of change together. The ranking is determined by a single measure: game results. Inasmuch as there are not any significant differences in scoring, this ranking can operate. Other conditions may vary, such as game length, location etc... just like air pressure may very. Ideally these should be accounted for or, even more ideally, made equal.

Right, I think you're close, but you're being misled into thinking that "game results" is a homogeneous category by the fact that we apply "game" to a lot of different things. Question: which falls faster, a feather, or a tiny rocket (much smaller profile than the feather) which is being propelled upwards with a force that is equal to a very small fraction of the (downward) force of gravity?

Answer:

The question doesn't make any sense. Neither "falls faster" than the other. In a vacuum, the feather will fall faster; in a thick atmosphere, the rocket will fall faster.

[RobertJasiek]

How little or much can a single number express about relative player strengths?

1) The number can only describe what is measured. If tournament games are measured but non-tournament games are not measured, then it will not be noticed if different players are weaker or stronger in or outside tournaments. Same if some tournaments are but others are not measured.

2) Even if the numbers suggest a transitive ordering of the players because the numbers themselves have a transitive order, the players cannot be ordered transitively and linearly. This is so beacuse regular cyclical domination like A beats B beats C beats A occurs.

3) A multiple of a double round robin played within a reasonably short time (so that some players do not improve significantly yet) is the best tournament system for getting numbers: They are the number of wins. Double round robin ensures that each player gets each colour equally often. Even so, the numbers do not express a player's general strength but only the one under the particular tournament's conditions like komi, rules and thinking times.

4) A series of regularly occurring tournaments with the same conditions does not assess equally meaningful numbers for all players because some players will not be able to play all rounds or all occurrences of the tournaments. Rather some players' information will be more significant than others'.

5) A set of different tournaments in that all players of the same population play all games do not compare their strengths but only a partial aspect of their strengths because the tournaments will have a particular distribution of tournament conditions, which favour some of the players who play the strongest under them and which will be a disadvantage for others.

6) It is very hard to compare well players with (very) different numbers of played games.

7) Different tournament conditions cannot be expressed well by a single number. Rather every player can have different strengths under different conditions like being weak at lightning and strong at slow games or like being weak at 6.5 komi but strong at 7.5 komi or like being weak under area scoring but strong under territory scoring.

8) If numbers are determined by anything more complicated than simple number of wins, then (rather) arbitrary parameters (and system methods) are required and strengths depend also on how the ranking system is defined rather than only on how a player performs. To make things at least a bit fairer again, a space of various sets of parameters leading to a space of various ranking systems should be tested and expressed as a space of numbers per player rather than a single number.

9) Connecting different player populations (like from different countries) to each other is very difficult because a) relatively few inter-population games are played, b) different players play different numbers / percentages of games as inter-population games, c) tournament conditions in different populations differ, d) the measurement problems in one population occur also on the inter-population scale.

10) Theory of non-trivial ranking systems is being researched more than already well understood.

Conclusion: A world ranking (rating) system relying on a single determined number per player is nothing but an illusion.

[hyperpape]

Robert, 2 is a non-sequitor. A transitive ordering is just a type of logical relation. It requires an interpretation to know what is says about the players. You simply assume that it must say "player a will beat player b more than 50% of the time." But this is not the only thing you can mean when creating a rating system--nor is it what you should mean.

[daal]

It is not a question of whether the ranks are correct or accurate, but rather whether they make the fans spend more money.

[jts]

Robert, 2 is a non-sequitor. A transitive ordering is just a type of logical relation. It requires an interpretation to know what is says about the players. You simply assume that it must say "player a will beat player b more than 50% of the time." But this is not the only thing you can mean when creating a rating system--nor is it what you should mean.

But you would agree, wouldn't you, that if a given ranking system says X is better player than Y, and Y is better player than Z, but Z beats X more often than not, that would be a distinctly odd result, right?

[topazg]

Robert, 2 is a non-sequitor. A transitive ordering is just a type of logical relation. It requires an interpretation to know what is says about the players. You simply assume that it must say "player a will beat player b more than 50% of the time." But this is not the only thing you can mean when creating a rating system--nor is it what you should mean.

But you would agree, wouldn't you, that if a given ranking system says X is better player than Y, and Y is better player than Z, but Z beats X more often than not, that would be a distinctly odd result, right?

Personally, no, I wouldn't. It's pretty common actually. A ranking system for me should approximately order people by their overall results. Say we have 101 players who play every other player 100 times, totalling 10,000 games per player. I would expect the rating to prioritise people with an overall better record, but I could happily believe that 3rd lost to 5th in their vs games 35-65 or something. Some people just find some opponents harder than others.

[TMark]

There was a curious case, some years ago, of a British 4dan who was always just getting close to promotion to 5dan and he would lose to the same 2dan in tournaments. He actually lost 4 games in a row (the fifth was a jigo) before he got the measure of him and got promoted. Sometimes a person's style can be a real bug-bear.

Best wishes.

[hyperpape]

But you would agree, wouldn't you, that if a given ranking system says X is better player than Y, and Y is better player than Z, but Z beats X more often than not, that would be a distinctly odd result, right?

It sounds weird in the case of three players, but it's not at all an issue when you have a hundred--you will have cases of circular dominance. Of course if there were a circle involving one hundred players, you will not have effective ratings. Similarly, if most players alternate between 9 dan and 1kyu playing strength, you will have a great deal of trouble. But we're lucky--things don't turn out this way. (The same can be said for scientific practice in general. If nature is sufficiently uncooperative, you will never discover anything).

[jts]

But you would agree, wouldn't you, that if a given ranking system says X is better player than Y, and Y is better player than Z, but Z beats X more often than not, that would be a distinctly odd result, right?

It sounds weird in the case of three players, but it's not at all an issue when you have a hundred--you will have cases of circular dominance. Of course if there were a circle involving one hundred players, you will not have effective ratings. Similarly, if most players alternate between 9 dan and 1kyu playing strength, you will have a great deal of trouble. But we're lucky--things don't turn out this way. (The same can be said for scientific practice in general. If nature is sufficiently uncooperative, you will never discover anything).

I completely agree with the last point; intransitivity paradoxes are rarely a problem in the real world, at least not for practical purposes. I don't think we'd ever have difficulty assigning handicap stones, for example. But once you get beyond "practical purposes," what's the point of ranking players if transitivity doesn't hold, in general? We agree that if 1 in 3 rankings are intransitive, rankings are pointless; if 1 in 100 are instransitive, the rankings work quite well, with an asterisk. But people in general tend to underestimate the likelihood of transitivity difficulties.

(I should note that this is a separate, and less important, issue than the dimensionality problem I was discussing with Monadology earlier.)

[gowan]

We want ratings to tell us who is stronger than whom. Yet we have situations like this: Player A has a rating 100 points higher than player B yet over their many games together player B has forced A to a two stone handicap. Whether you say A is stronger than B or vice versa depends on your definition of stronger. There is no question that A has a higher rating, of course, but does that alone mean he is stronger?