Tuesday, February 10, 2015

New Scoring System for ABS Nationals

With all the talk going on about the new scoring system at this year's ABS bouldering nationals, I thought I should chime in on the discussion with my two cents. 

First off, I think bouldering is too nuanced to ever be perfectly scored. The new system obviously has its criticisms, but most other methods fall down in certain cases too.* All in all, I think the new system has some real merits. Here's my stab at giving a better explanation of and a rationale for this new method of scoring. 

Imagine a competition series with 4 different events spread out over time. After each event you can rank the athletes by their performance (assuming some scoring system for the event). If you want to crown a winner for the whole series, you need a way to combine the per-event rankings. A simple way to do this is to combine the ranking each person got in the event (i.e., 1st, 2nd). If there's a tie, find some way to assign points (e.g., average the places where the tie occurred -- this is the scheme ABS used). Now the simplest way to do this combination is to average the rankings from each event to get an overall ranking. The person with the lowest average ranking wins.

Now imagine that each event had only one boulder problem in it. Guess what, you now have the scoring system used in this year's ABS nationals with 2 caveats.

1) The total number of tops trumps your ranking (i.e., average ranking is used just to break ties in the total tops score).

2) Instead of taking the arithmetic mean (average) of the rankings, the geometric mean is used. That's all there is to the nth root of the product of rankings. It's just a different way to compute an average. The geometric mean has the advantage that is suppresses outliers (i.e., doing poorly on one problem has less effect on your overall ranking than with the arithmetic mean). See figure and description below.

The individual boulder problem rankings were determined using a system that we're all pretty familiar with. Namely, highest hold achieved is most important and ties are broken by number of attempts to get that high point. If two people are still tied, then they just split the ranking points. See the full description of the scoring method here.

In my opinion, this new scoring system is completely reasonable and may prove to produce more consistent results than previous methods. It just didn't fare too well in its first showing. I agree that it's a bit complex for the viewer to follow, but none of the other methods are that easy to follow either unless you've been watching comps for years. If the scores are constantly displayed for the viewers, this is less of a problem. I do concede that it's a bit strange that with this new method, the relative ranking of two individuals who have finished climbing can be changed by some climber later in the round. For me, that's not really a big deal though... you just have to wait till the end of the round to know the results. There's a lot more to go into here, but one thing I like about this new method is that it strives to assess the difficulty of each problem in a way that is relative to all the climbers.

As a side note, here's a quick analysis that compares the arithmetic and geometric means for a set of 4 rankings. All 4 rankings have the same arithmetic mean (3), but they differ in their geometric mean. I've order them to be ascending in geometric mean and I'd argue that the ordering is roughly consistent with what subjectively seems to be better performance (i.e., ranking 1,1,1,9 is better than 3,3,3,3 in my opinion).


*For example, points per hold methods aren't good when a problem has a crux down low and then lots of easier moves because someone who gets through that move racks up lots of points. Compare that to a problem with easier moves at the start then one hard move at the end. Ignoring the affect of getting a top, the climber who does the hard move at the top of problem 2 doesn't get rewarded nearly as much as the person who gets through the hard move on problem 1.

5 comments:

  1. This is by far the best explanation of the final score calculation for ABS that I've come across. I'm still skeptical about whether this is a better scoring substitute due to real time score calculating difficulty though. Great post and clarification!

    ReplyDelete
  2. Matt, I'd be curious to hear your thoughts on the fairness of splitting the points for a tied ranking, i.e. two people tied for first place get 1.5 point each rather than 1 point each. It seems to me that this was the primary reason for Daniel Wood's loss on Saturday. From a mathematical standpoint, I can understand why one would want to split the difference to create an average score, but that seems to only make sense in a real world scenario such as dividing prize money (if you only have 300 dollars for first and second place and two people tie for first, better to give them both 150). In the competition scenario there are no scarcity of points, so I don't understand why you would average and split the points.

    You say above that after each competition you would rank the athletes by their performance. It seems to me that first place finish, is a first place ranking, regardless of whether or not other people also got a first place ranking. Otherwise you're essentially saying "a first place ranking on this problem is only as valuable as a 2nd place ranking on the previous problem, provided 2 other competitors also ranked first on this problem and only one competitor ranked 2nd on the previous problem". This puts the competitor who is lucky enough to not have anyone tie with him in an advantageous position. Once you start averaging point values, the extra points you get for a tie start rippling through in a big way, ultimately affecting your overall ranking.

    ReplyDelete
  3. Matt,
    As a fellow engineer and climber, thank you for this post. It captures the nuances between the arithmetic and geometric mean in a way that I missed in my score-explanation post. I'll be sure to drop a link to this page onto my post. Great to hear a math driven discussion by someone that also has experience competing at that high level.

    ReplyDelete
  4. hello matt a pleasure to come here daily im a recent climbing enthusists and I recently purchased an yaktrax ice axe from buyclimb.com its great and all but wondering if you had any suggestions?

    ReplyDelete
  5. Welcome to island peak climbing Nepal Himalaya. It is the best option to give mount. Everest base camp as well as other Everest region trekking in Nepal. If you have 17 days, you finished both island peak and Everest base camp.

    ReplyDelete