Optimizing a rating system for minimal complains, maximum player engagement, or some similar metric is of course totally valid. It reminds me of Sirlin's story of being hired to design a rating system for Starcraft 2, and optimizing for totally different things that Blizzard wanted [0].
If the author (you?) had just described it in those terms, it'd be hard to object. But the article goes further, and makes claims about the system being more accurate due to a different rating curve. That's the claim that would need to be justified by actually comparing whether the predictions the new system makes really are better.
Part of a matchmaking algorithm that increases user engagement is telling a story about how it's more fair though.
Like our tolerance for losing is acquired. Most normal people losing in League for the first time stop playing, usually forever. Just randomly visit your friend's match histories in League, frequent players have many days of long losing streaks.
If you're just conditioned to play despite losing, great, in a Darwinian way (surviving, being around to be measured) you will be representative of the average player in League. And there are so many League players with such long retention you cannot possibly argue that skill-based matchmaking is the core component of user engagement.
His dataset is interesting because it will necessarily overrepresent people who kept playing despite the old system. That sort of refutes its importance - I mean sure people complain but they keep playing, so was it really that important? So what if complaints go down?
Those are important goals, and also, it's still an interesting twist in multiplayer game design. You just gotta interpret it as a commentary on a whole system even if it doesn't narrowly talk about a scientific objective like performance prediction.
Player retention was significantly lower before the new rating system was introduced. It wasn't just complaints, it's just that is a more direct metric because multiple changes may have affected the retention.
The funny thing about SC2 is that the player's MMR (matchmaking rating) is decoupled from rank, due to design decisions such as demotions not occurring midseason. So a gold league player with a low enough MMR may get matched against bronze league players, despite ostensibly being ranked higher. Actually for the longest time after release, player MMR was not visible. It took 6 years and 2 expansions before it was finally displayed in-game.
I like the league system in sc2, it allows you to see progress and to get to know other people's style (because you meet the same players several times in a quick succession). People build narratives around that (I remember that guy I lost vs him last time and now I had my revange).
If somebody won against you with a particularly dirty strategy you can get back at him and you won't be fooled the next time.
It's better than dota system where you're most likely never going to play vs the same people again (or at least it feels like that).
Also the whole 5v5 game that lasts for 1 hour format is inherently very toxic - most of the time you know you lost after 10 minutes and yet you can't disconnect and have to keep playing for the next 30-50 minutes while everybody blames everybody else on your team.
In sc2 if you know you lost after 5 minutes it's discrespectful to continue playing :)
So there's 10% chance I'm wrong. That's 10% chance of losing some mmr vs 90% chance of losing 30 minutes of my life. I don't have much time to play so in sc2 I would just go to the next game.
Sadly in a team game you can't make that choice because disconnecting too much will prevent you from playing all the fun modes (and will put you into a low priority queue where you play with people who constantly insult everybody or kill their own allies).
That's the root cause of all the toxicity in dota.
I've played Dota for way more hours than I want to admit, and unless you're playing on the highest of levels (Immortal) where people really have a very strong feeling for the win-conditions that they have and when it becomes very hard to reach them, I call BS on your statement. There are very strong reasons why pro players almost never call it GG before at least one rax is down, and often wait to do so for at least one last hail mary fight (depends a bit on the patch though).
The chance that the enemy throws a fight or makes other mistakes that opens up new win conditions (or sometimes that simply one guy in the other team tilts and throws) is just too high in Dota in order to be able to call it quits after 10 minutes.
One of the main reasons for the toxicity in Dota in my opinion is that people don't understand that there are 10 variables in the field, but they can only control one of them. Don't even bother trying to influence the other 9 (apart from positive feedback/encouragement/communication to the team), it's a waste of time and only contributes to the toxicity.
I'm at 2.5k, almost always playing as pos 5 winter wyvern. I'm under no impression that I'm great at dota :)
At this level with the same position and firstpicking ww I get a small subset of heroes I play with and against, so it's much easier to call. I have pudge and sniper every other game for example, and enemy pos 5 is very often cm or ogre :) It's a great day I get to play vs meepo or oracle but it almost never happens. The best I can hope for is axe or legion.
Also half the fancy strats don't work because people don't talk. Split-pushing is just split-feeding. Nobody does 3-lanes. Jungling is a last resort of carry that lost the lane and when he's fat the barracks are already destroyed.
So if we lost laning hard enough and have earlier-game cores than the enemy - we lost, it just takes a lot of time to play out.
But even if I was wrong 30% of the time instead of 1o% - I'd still happily take that trade in mmr if I could. Obviously pro players have different motivation so for them it's "never surrender", but almost nobody is a pro.
Not to mention that the bonus pool feature has long been removed.
More importantly, though, your MMR has now been split up across the three factions that you can play as. If your Terran gameplay is much better than your Zerg gameplay, you'll be facing higher-ranked opponents when you play Terran, than when you play Zerg.
This has made it easier for players to switch their factions up, without being punished for it.
What you said makes me wonder about a totally different way of using a metric of how likely the person is to win a given match.
That is, perhaps the system could be engineered to maintain a more even win/loss ratio so that people don't go on super-long win (or loss) streaks in general by adjusting who they get matched with.
It probably wouldn't work that well towards the edges, but around the middle it might work well enough.
This has what dota 2 does. Every win / loss is worth the same points, the games are extremely close in terms of player rank, so and it's a team game so you get more variance than just your individual skill. Players eventually settle close to 50 percent win rate
The "different rating curve" appears to be the actual historical probability data, not a formula. I think. If that's right then it this estimate of the probability of winning is not a new discovery.
On the other hand I suspect the historical data really is the "best fit" to the historical data.
If the author (you?) had just described it in those terms, it'd be hard to object. But the article goes further, and makes claims about the system being more accurate due to a different rating curve. That's the claim that would need to be justified by actually comparing whether the predictions the new system makes really are better.
[0] http://sirlingames.squarespace.com/blog/2010/7/24/analyzing-...