I like to think of rankings as a necessary evil :) Well, actually I don't think they're evil, I actually enjoy reading them. I like knowing who's at the top of the tree, who's next and so on. I also like the debates that keep popping up about methodology.
Rankings MUST have a judgment element. Results alone give an incomplete picture.
There are those who say rankings can be completely logical, and based on results, but I beg to differ, especially in a world where we have multiple promotions and guys at the top who don't get to fight each other. Fedor/Overeem vs the UFC top group is a case in point. It's difficult to use a pure results-based or lineal method to rank the #1 and top group when you can't use Fedor vs Brock/Cain/JDS results.
The fight's official final result is valuable. But we should also consider the detail of the result.
Also, there will be people who claim the only results you should look at are the final results: W, L, D, or other special cases. Hence a fighter who defeated opponents X, Y, and Z through first round KO would be ranked the same as one who defeated those same via split decision robberies where the opponent seemed to have dominated the fight. I don't agree with that. I believe most people generating rankings do take into account the nature of the win. I believe most people discounted Werdum's win over Fedor as somewhat flukey and that most of the time Fedor would beat Werdum - so after the fight, Werdum did not wind up higher than Fedor.
Potential should not be taken into account.
Let's say for example that Overeem wasn't an MMA fighter but dominated K1 and won the world BJJ championship and division 1 wrestling as well. Let's further say that he'd sparred with top MMA fighters and appeared to dominate them in the gym. We might think potentially he'd be the #1 HW fighter in MMA. But you CAN'T factor that into the rankings unless he demonstrates the skills in official fights.
How should performances be valued?
I'm not a huge fan of pure lineal rankings. E.g. if Werdum (lower ranked) beats Fedor (higher ranked) it's arbitrary to just promote Werdum to #1 and push everyone else down. It implies Werdum, by beating Fedor, has improved to the point where he would beat everyone else between his rank and #1. You could just as logically (or illogically!) DEMOTE Fedor to the spot below Werdum and bump everyone else up, promoting Brock to #1 - which implies that it is Fedor who's got worse rather than Werdum getting so much better. In practice, the rankings have suggested people are doing something in between - and I agree with that. The result signals Fedor has got somewhat rusty; and that Werdum MAY have gotten better. But because the "market" doesn't believe Werdum is actually better than Fedor, he didn't get promoted above Fedor.
Here's an interesting scenario:
Say the #1 fighter today has beat the #4, 5 and 6 fighters with very close decisions. And another fighter comes along, and beats #6, 7 and 8 by total domination (either smashing them to a 30-25 decision or KO/sub). Normally, we'd rank this new guy at something like #5-6 because he hasn't beat anyone above #5. But based on his performance, it might be possible to see him ranked higher. Perhaps even #1. This is why some think Overeem should be ranked higher than what his MMA achievements would suggest. The fact that he has won K1 (which, strictly speaking, should not factor in) strengthens this point of view. People might believe, after he smashed #6 thru 8 so dominantly, that he would easily beat the #1. At this stage, is it reasonable to rank him #1 or not? Is it judging too much on his "potential" to beat the #1? Or is it simply weighing the performances against #6-8 against the #1 fighter's performances against #4-6, and saying "on balance, Overeem has in fact achieved more than the current #1 fighter, so I will revise the rankings to reflect that"?
I open the floor to your comments.