Judge not...
Much more than a rant about judges, blindboy purges himself of past misdeeds and offers the ASP judging panel some good advice. Real advice! Read on...
It was sometime around 1980 that I swore on whatever it was, now forgotten, but seemed sacred at the time, never to have anything further to do with surfing competitions featuring any other than personal friends. And, in particular, I would under no circumstances take any interest in the subject of judging such events. It was a subject that had never brought me happiness. The only time I actually judged a serious event I was accosted outside the official tent by a wild eyed, apparently ex-friend who accused me of various perversions and atrocities, including placing his brother last in his heat. But it wasn't judging per se that caused me grief so much as the opinions I expressed about it.
Once, under the persuasive influence of those with even less sense than me, I actually entered an event I should have known not to. I wasn't too disconsolate about coming last in my heat, I knew it had been a less than distinguished performance, until an important media entity of the day on walking past remarked "That'll teach you to be a smart arse." But it didn't and I continued, as the chance arose, to make sarcastic remarks about the competency and objectivity of the judging panels.
Despite evidence to the contrary I still claim it was not an obsession but neither was it some sort of quest to make the world of surfing competitions even more perfect than their press releases claimed them to be. In the end it was probably motivated by not much more than a desire to raise an index digit at that smug and self satisfied minority whose main interest in surfing was increasingly to extract as much personal profit from it as possible while hiding behind a charade of charitable benevolence to those, all too often literally, poor bastards doing the surfing.
But time moves on and having forgotten the exact nature of the oath I swore at midnight on those deserted crossroads and having long ago exhausted whatever advantages the servants of satan promised me in exchange for said vow, I now feel free to offer an opinion again. I would like to suggest that my motives are of a purity rivalling the gold holdings of our corporate captains, but you probably wouldn't believe me, so I won't bother.
In truth things are better. I am fairly sure they don't let Claw judge at Bells anymore, and while I can't be sure about the investment status of the current panels nor their employment histories or those of their spouses and children, I am a believer in the benefit of the doubt and " by their works ye shall know them" so I am happy to concede objectivity in that sense. If some of the judges are biased by corporate or national loyalties, they seem to be balancing each other out.
Then there are the numerous technical improvements such as moving from 5 man heats with 120 possible result permutations to man on man heats with only 3. The hiring and training of a professional body has also clearly improved things. So by historical standards there is not much to complain about but I think I am on to something. I like it because it reflects the same kind of hubris that has always simultaneously fascinated and repelled me about pro surfing.
It's the scale! 1-100 or 0.1 - 10.0, if you must. There's a sort of message in that alone. I mean if you are going to judge on a 100 point scale why not just use those numbers? Well, for a start I think jumping up and down and declaring a wave "a perfect 100" might arouse some skepticism. More significantly it might, just through the way it is expressed, cause more people to wonder just what is the difference between a 99 point ride and a 100 point ride. 9.9 and 10 are so much more user friendly, so much less obtrusive.
Taking this further, an analysis of the scoring waves in the Quarters, Semis and Final of the Drug Aware Pro suggests that the 100 point scale is too wide. If the scale was appropriate the judges' scores should fall within a narrow spread but this was not the case. There were only two waves with a spread of less than 0.5 and many were significantly higher even exceeding 1.0 in some cases. Removing the high and low scores improved things but there were still many waves with a spread of 0.5 or more and some still up towards 1.0.
But then there is another factor which is even more telling. If the scale is valid, then the spread of judging values ending in the digits 0 - 9 should be relatively even. That is, there should be roughly the same number of scores ending in .2 as in .5 or .7. On the heats I looked at this was clearly not the case. There were far more scores on .0 and .5 than on any of the others. The obvious conclusion from this is that the judges are frequently reverting to a de facto 20 point scale.
The significance of this analysis is that it very strongly suggests that the results of close heats are influenced by random factors beyond the official criteria. In particular it would suggest that two different panels would be quite likely to come to opposite conclusions about the results of such heats. This should not come as a great surprise to anyone but it is not a happy situation. It could be argued that the problem is intrinsic to the sport and there is a fair amount of truth in that but the problem could be significantly reduced. Using a 20 point scale with a third or fourth wave tie breaker would give more reliable results and introduce a bit more pressure on the competitors and excitement for the spectators.// blindboy
Comments
Solid argument Blindboy. Stirring the Pot again I see. An essential community service...
I totally agree with you and think I mentioned this ages ago when we were all moaning about the judging. Increments of 0.5 would definitely suffice. How the hell can anybody judge a ride within 1/10th of a point?
Loved that bit BB "apparently ex-friend who accused me of various perversions and atrocities, including placing his brother last in his heat". Haha such is life, an acquaint BB not a friend at all............
Like 'ZA' said increments of 0.5.......?
My wrangle is how can a surfer get a perfect 10, or with your judging criteria 100.....?
We don't know what is perfect, surfing and other sports are always evolving.
I honestly think judging someones style of surfing against the emotion of a wave, would be very difficult..?
In years to come judging surfing comps will be compiled through a computer, with zillions of body movements, zillions of different wave faces and tubes etc and zillions of different forces that all can be calculated by this contraption that I'm typing on and and on and on........... Where does the value or measurement stop.....?
And let's remember, unlike other equally debated subjectively scored sports (diving, skating, snow boarding etc), surfing is not performed on the same course. Every wave is different.
I don't mind the 100 point scale. That ride was just a teensy bit better.
On a semi-related point, I once heard a wine judge say that he reckoned wine was judged on a 15 point scale. Because, in competition, the lowest score he had seen was an 83 and the highest he'd seen was a 98.
Blindboy is on the money here,ive often thought how can the judging differ so much seeing they all watch the same replays yet a lot of the time the scores are so different and i wonder how much style preference can affect points given.
Humans are emotional beings, they react and judge accordingly.
No way round that.
Even the esteemed Head Judge Richie Porta has admitted that the judges "feel" a ten.
And the caravan moves on.
Thats why I think computers will be the judge in the future Steve, no emotions prevail....!
Computers love surfing.
The heat between Josh Kerr and Jacob Willcox today was a perfect example of a meaningless result. On the 2 wave total Josh Kerr won by 0.04pt. He was, on the day apparently 4/1000th better! If you look at the variation on the actual wave scores the variation on Josh Kerr's highest scoring wave, even after high and low were removed, it was a full point ie 100/1000. They are kidding themselves!
Another two heats that come to mind today. Was there Parko points and then Slater points in their heats because didn't look real convincing to me?
Kinda feel Melling got the rough end of the pineapple in his heat against Parko
The judging was reasonably consistent in both those heats seal. All the data is here.
http://www.aspworldtour.com/events/2014/mct/655/rip-curl-pro-bells-beach...
70% of the Round 4 scores for the waves that counted ended in either .0 or .5 .
Time for my annual Bells comment ....
Who gives a toss, hurry up and get the circus over and done with and fuck off.
I feel better now
BB, the same kind of scoring phenomena has happened in wine judging. Wine was always judged out of 20. A certain number of points were allocated for, appearance , aroma, palate etc with most wines scoring 14 and above. a few years ago a influential Wine scribe from the states decided the scale was too restrictive and started scoring wines out of 100. Many have followed his lead, even some major wine shows. The problem is nobody uses the whole scale, most wines get points of high eighties or above, the judges are basically scoring the in the old 20 point system. Scores have spread out slightly but not anywhere near as much as you would have thought after increasing the scale x5. Got nothing to do with surfing I know......apologies it's early.
judge not?.... has anyone checked the comment section on the ASP live event youtube feed?
judges are being judged. to 'da max.
If it was to revert to a 20-point scale, how many close results would be skewed because a score had to be rounded up or down to fit the scale? I think what you'll see from your own statistics (bias towards .0 and .05) is the judges setting the scale for each heat and then adjusting from there (because, remember, the scale changes every heat). It makes more sense to initially set the scale using fewer increments, rather that arbitrary numbers, and then from there the 100-point scale lets them squeeze in other scores where they feel they fit in the gaps. You especially see this at the end of heats when they have to make a judgement on whether a wave earns the required score. It lets them fairly crown a winner by saying "that wave was just a tiny bit better that x wave..." but on a 20 point scale, even if the wave is better than the required score, they might have to say "sorry, it's better, but not 5% better" and round the score down to below the requirement.
As for the spread of scores being larger than 0.5 ot 1 points, I think it should be. Having all the scores be almost identical stinks of conformity, as though the head judge is standing there saying "that's roughly a 6.5... anyone who disagrees will need to consider a new line of work". I'd rather see a seven man panel where more subjectivity is allowed - because surfing is subjective - so we should take a bigger sample group and allow for a larger spread of scores.
Fair comment Johnson but my main point is that in close heats the final score is statistically meaningless. If the judges cannot agree to within 10 points out of 100, then sorry but you cannot claim a valid winner by 0.4 pts out of 100. It's not accurate, it's not fair and it undermines the credibility of a system which they have worked hard to improve.
It may be that the judges can reliably do what you say but I don't see the evidence for it. Some degree of subjectivity cannot be avoided but to move in that direction I think is probably wrong headed and likely to cause way more problems than it solves but thanks for the thoughtful comment.
I wholeheartedly agree BB. I've always thought that the 1-100 scale is ridiculous. There just doesn't seem to be enough rigour in the allocation of scores (even allowing for some degree of subjectivity) to merit the use of that scale. Porta's statement (found in an interview in Surfing a couple of years ago, if my memory serves me) that "the numbers just come to me" must provide cold comfort to the surfers whose livelihoods depend on judging.
Your observation that the judges revert to a de facto 20 scale is an important one.
at least Porta was being honest.
there's no mathematical equation involved.
Luckily for all concerned, most heats are pretty clear cut. A four year old could pick the winner.
freeride if you want to pick winners based on the opinions of a set of judges, that's fine with me but once you start using numbers there are agreed upon rules and procedures, such as how to deal with unreliable measurements, which is what the judges scores are in essence. If you don't follow them you just look silly. Yes the ASP has every right to declare that 2 + 2 = 3 when they add up their scores......but it doesn't make it right or wise.
Not arguing with that BB.
Just the essence of where the number comes from in the first place.
Good article blindboy, I agree on a simpler scale with countbacks or sudden-death instead of someone losing by .1. I'd also like to see the ASP is to publish clearer guidelines on how these decisions are made - the weight given to aerial surfing vs carving attacks, or the importance of wave size, or the tube vs the carve seems to change from contest to contest and sometimes from heat to heat. Why not have 20 sample waves at Bells from the last five years up on youtube with a detailed breakdown of what got them the score from a head judge? It might help both competitors and audiences understand the judging rationale a bit better because sometimes it's still a mystery even to the commentators.
It's a mystery and it will remain ever such....how do we get the numbers?
we feel them.....
~strokes chin and exhales~
Might all be pre determined...on the 3pm news update on 3AW, they said Steph had made it to the final against Carissa...According to the ASP site the semis haven't even been surfed yet!
You need a hipster beard as well.. FR76
Judges need to be in individually soundproofed rooms so they don't know what:
- The other judges scores are, nor
- What any given surfer needs to progress through a heat at any point in time
Reckon these very simple process changes would pretty quickly sort out the good judges from the bad, anything to improve accountability is a good thing.
Would also make Richie Porta's role during the contests redundant, he can coach his judges in between events and liaise with the surfers as required ... but the good judges don't (or certainly shouldn't) need someone telling them what scores to give.
Surfing could also follow any number of other professional sports and have 'neutral' judges for all heats (i.e. nobody on the panel of the same nationality as the contestants). Shouldn't be too hard given there's only ever a max of 3 different nationalities in play at once. May take a while to build depth on the panel from outside the big surf nations but it would get there with effort.
And it's nothing to do with perceived bias, it's just good governance if the ASP wants to continue to compete with 'real' sports for sponsorship dollars.
Love those first two points mickj, particularly the second one. A lot of heats come down to the last wave and the judges know exactly what number they need to give for one or the other to win, which has to effect their judging.
Yeah that's the key outcome for me too mate ... it often comes down to last wave of the heat, the scale gets completely thrown out the window and it's a simple question of:
"Who do we think deserves to win this heat" (and/or subsequent world title etc).
This is a cheap, low tech, completely unbiased way of eliminating that equation.
ok here is my solution lets say judging is only accurate to .25 so we only allow them to score in increments of .25 (or out of 40) so we only have 0, 0.25, 0.5 etc to 10.
so if the surfers 2 wave scores are within o.50 it is effectively a draw allowing for these possible inaccuracies. then the surfer with the highest single wave score progresses assuming it is .25 better than the other.(encouraging surfers to go for big manoeuvres etc.) if not we go to 2nd highest wave, then 3rd till we have a clear winner.
judges should have a strong opinion on what makes good surfing and a spread of scores is good to an extent as it shows not being told what range to score each wave by one person. head judge should be outside of scoring room and make calls on interference etc. only one replay on any wave. no fishy long wait for scores.
Classic example BB right there with the Jordy/Julian heat.
Jordy just got so heavily ripped off
Judges blew their scale on Julian's 9.1, Jordy's last wave was a point better than Julian's (at least) every day of the week.
Heat analyser isn't going to make Julian look good.
Ben I did a bit more thinking about this and the fundamental problem comes from averaging the three scores and retaining the decimal place instead of at least rounding off. I have worked out a system and will probably post it later. Basically it uses a 20pt scale and the mode and median of the five scores instead of the average of the three.
So my suggestion would be to use the following system.
All scores to end in .0 or .5. A 20pt scale.
If three or more judges give the same score that becomes the score regardless of the other scores. eg 9, 8.5, 9, 8, 9. The score is 9.
If all five judges give different scores (unlikely but possible) then the median (middle) score becomes the score. eg 7, 8, 9, 7.5, 8.5. The score is 8
If there are two lots of two scores and an outrider the outrider determines which of the two scores is used if it is higher the highest pair of scores become the score. eg 7, 7.5, 7.5, 7, 8. The score would be 7.5.
If there is a single pair of scores and three different scores the median is used. eg 7, 8, 7, 6.5, 7.5. The score is 7.
All ties to be split by a third and then fourth wave and a re-surf if necessary.
I should patent it LOL.
Not bad. I'm giving your scoring system an 8.2 .
Thanks wally but I know you're just teasing.
This is what happens when you over think things. The whole system can be reduced to using the median score! How easy is that!
The current scoring system is great at not producing ties. As you say, the points difference can be spurious.
Fairness or not, ties are not wanted, unless they are very rare. They struggle now to get enough surfable waves to run all the heats in an event. Also, the current system gives them more certainty of being able to say, we WILL be able to run all of Round 3 today.
Also spurious points differentials might be fairer over all, as the more tied heats, the more heats that get pushed into sketchier waves, where perhaps luck would beat skill more often.
And, just from a viewer perspective, these events go on for quite long enough as it is.
The current system uses a 100 point scale to eliminate ties, sometimes spuriously. Your system uses going to a 3rd and 4th wave. I like that, in principle.
The arguments against:
I assume the best 2 wave system replaced the best 3 wave system for a very good reason. Whatever those reasons are, that would be the argument against. I assume one of the reasons was wave supply.
Also, it would discourage the adventure of the high risk manoeuvre if you knew there was more of a chance the wave would count even if you wiped out.
wally there are many ways of splitting ties without the problems you raise. I think under most conditions the third wave is the best option and the odds against getting more than one tie a day that couldn't be split that way would seem to be pretty high looking at the results spread from the last couple of events. If there were concerns about a third wave then the highest single wave score could be used and beyond that the five judge wave totals. Fair and accurate judging need not reduce the entertainment value. Whatever system is in place as long as it is based primarily on two waves I think the surfers would approach it exactly as they do now. Not many heats are that close.
dont think you should rely on 3rd and 4th wave not going to encourage radical moves seems a little bit of a backward step. much more should e placed on single best wave of the heat. i.e. two surfers score 14 points. one with two 7s and another with a 10 and a 4. the guy with the 10 should clearly go through. encouraging people to go for broke on waves.
don't think there is anything wrong with the way highest and lowest score discarded and the average worked out seems pretty solid to me the problem is not the maths but the human scores that are put in.
i really think that there should be a drawn heat option and then the highest single wave goes through seems fairer to me. and jordy would have won, he did win, no he didn't .....yes he did...etc
eteve I have no problem with the high wave being used to split ties but there is a problem with the maths. Lots of instruments have built in errors. For example 3 radar guns pointed at the same moving vehicle could give readings of. 63, 64 and 66. The average of those is 63.3 but you cannot possibly claim that it is the exact speed of the car, your instruments are not that accurate . The same idea applies to the judges scores. If they vary by 10/100 you cannot possibly award a score to 0.1/100, which is what is happening now. Sorry but it is completely illogical.
"Sorry but it is completely illogical." Mr Spock.
Bells. That was a very good event. The best thing you can get is offshore winds. Small swells are still good to surf.
They only seem to run the mens and womens events concurrently in Australia. I don't know why. It must say something about these ordinary Aussie beaches to deliver waves of some sort. That is not being nationalistic, the surf is the just a happy accident of geography.
I think I counted right. The mens is 51 heats. Womens is 27 heats.
78 heats. It's a lot to find surfable conditions for.
Running the mens and womens events concurrently is the ideal. Does anybody know why this only seems to happen in Australia?
Correction, Brazil is also concurrent.
I think we agree that judges are not accurate to .01. would be an interesting experiment to get 10 judges show them footage of a 9.86 and a 9.87 same wave same heat and see how many could pick the difference.(you could then increase the points difference until 10 out of 10 judges could pick it to find out how accurate they are. but heats are often decided on these types of decisions.
the car thing is quite interesting because your car does have an actual speed if you want to waste your time you could find a very accurate way to measure the speed of it. like for world records etc. however a surfer does not have an actual score for a wave. it is all very subjective. what I see as good surfing you may not. they have a criteria I know but again it is very subjective and open to interpretation. In other words surfing is one hell of a contrived sport and the maths of working out the score is not the weak link here.
You are right about the subjectivity eteve but the maths adds to the problem and that is something that can be easily fixed while the subjectivity will always be an issue. Most heats have clear winners, some of the closer ones the judges agree on but there are always those heats in which there is no clear decision and pretending that there is one on the basis of dodgy maths is unfair in the final analysis.
2X1=2
5X4X3X2X1=120
I enjoyed your article.
The most substantial issue with surfing is how complex it is to judge it. there are lots of criteria to judge a ride against and then the judges must decide how to weight the scores according to the conditions. your hear the commentators go on all the time about how todays conditions call for more aerials or more power manoeuvres etc etc. In one of the bells heats I heard a couple of the commentators talking about how "style" plays into how well a manoeuvre is scored. However I believe that "style" is not one of the official ASP scoring criteria and in itself is incredibly complex to judge against (one persons like is anothers dislike).
At the end of the day the most important thing is that the judges scoring is "tight". dropping the low and high scores helps with this. if the scoring is still too loose then the criteria/methods for judging and the guidance given to the judges needs to be improved but not the maths. whether the scoring is 0.1 or 0.5 doesn't really matter provided that the step-size is small enough to give the judges enough play. A judge needs to be able to make the decision about which ride was better. a 0.5 step provides a decent way of resolving one ride from another but 0.1 is better. in fact strictly speaking making it 0.01 technically improves the accuracy BUT eventually it hits the law of diminishing returns and the last thing we need to see is more delays in scores hitting the screens due to judges trying to resolve impossibly small values. making it 0.5 wouldn't hurt much but its not the real problem at hand.
averaging is very suitable way of determining the "likely" true score. when coupled with discarding the lowest and highest scores the this improves further. Statistically, the best way to improve the mathematical accuracy further is to add more judges, but eventually it again tests the law of diminishing returns. I'm confident that using the "median" is not an improvement. statistically speaking using the "mean" or average will over time hit the true score more often. of course without radar guns and spray counters the true score in surfing will always be in part subjective.
Accuracy is one thing happy but reliability is another and the spread of scores indicate that the judges struggle to reliably use a 20pt scale. If the scores are unreliable then the spread indicates the possible error....in most heats over 0.5pt. You cannot then average them and claim 0.01 accuracy!
Blindboy, another point that has changed this year. Maybe i´m wromg, but where´s the judges nationality info? A few years before, you could see the scores per judge AND from where the judge is. Isn´t easier now for a judge to benefit a countrymen?
only if you are not fishing 76, any chance of an analysis of todays actual SN vCW at bells?
here's another angle on the ASP judging.....I think Porta is too weak a head judge,it is his responsibility to set the criteria before each event......
after being at Bells for the last week...I have never heard or seen so much confusion from the athletes and their coaches......so many wild calls eg the Jordy/Julian heat....best wave ridden all year(Quote Kelly and the rest of the people in the competitors area) needs a 9 .72 doesn't get it 3 judges score him a 10 the others 9 . 7 & 9.8...only 1/2 a point better decide than Julians 9 .2 .....Jordy's wave was clearly 2 points better.....confused...we all are.......... not much consistency anywhere...
One of the problems is too high a score at the start of a heat.....Micks 9.7 in the 1st minute of the 1/4's.......no margin anywhere...easy to fix ......make a rule that the highest 1st score can only be a 7....leaves margin for the rest of the heat and the judges don't get boxed in with someone getting and insane wave and tearin the bags...
At the end of the day the head judge is responsible and need s to be a very strong character...and be able to eloquently explain any inconsistency's...we need a new head Judge!!!
Great article BB. Recently I was (reluctantly) put in a judges box, I knew I was doing well when the PA blared "...and there's some very strange judging decisions going on today..."
That said, let's come out of left field and suggest a total democratisation of judging. Why not, with the near-total penetration of iDevices, have a central wireless at a comp where everyone watching can input wave scores, instantly? So can TV audiences. In this way we will reach the 'consensus' value of the wave as derived by those watching. The technology now exists to do this, a clever app is all that is needed.
Of course there will be pitfalls - the public may not be as savvy as trained judges in rating criticality, etc. Some people may really love/not love a certain competitor and score accordingly. Maybe they won't be able to keep up (and like voting in the US, it will not be compulsory). But, added up in totality, a 'market' view of a ride will be established and will eliminate any accusation of bias, for we all made the decision. I suspect you will get a 'bell curve with tails' like in stock and commodity pricing as to the value of each ride, and each rider. But it will be the sum of the ride as viewed by those watching.
velocity I don't see your idea flying as way to judge the heat but get someone to put together an app and sell it to the ASP as an add on. It would give the commentators something to talk about! Or you could shop for a sponsor yourself like the Go Pro thing. All assuming that you can withstand the barrage of bullshit you would have to put up with to make it work.
Actually there's been a few people try that crowd-judging idea (web based) with limited success. Would be interesting to see if it worked with an onsite crowd though!
What the judging needs is a clearer definition before each heat starts of what they are looking for,seems as though no one knows even the commentators dont know,which is a worry.Brutus is right cause that heat with JW and Jordy was unbelievable but that wave of Jordies was the best wave ridden all year and not given a 10 was a joke,seems they wanted JW to win.I think they shoud have 5 past world champions as judges as well as the judges they have,seeing they know how hard it is to surf to the criteria ,but they need a more defined criteria before each heat.Ten judges would be more fair,scores would be way more fairer i reckon.
If you use my system of taking the median scores Jordy gets his 10 and wins the heat. Using an average and keeping the second decimal place is simply displaying an embarrassing degree of innumeracy. If they can't bring themselves to do anything else they should at least round off the averages.
the idea that a 10 should be for a perfect wave is kinda stupid. perfect is not the right word at all. we often see waves surfed perfectly that don't deserve a 10. If someone gets a tiny wave and tears it apart with perfect surfing should they get a 10? does that mean any mistake and you should not get a 10? so if a guy gets a 10 second barrel pulls a massive air then does the most amazing turn but falls on a cut back he cant get a 10? Jordy did not get the 10 because there where a couple of tiny errors which made it all the more amazing of a wave but not perfect and that's what the judges where asking themselves was that perfect not was that amazing or exciting or the best wave we saw all comp.
Good article, and judging (sorry about the pun) by some of the scores being handed down at Bells last week, especially with the women's competition, some judges have a long road ahead to get it right still.
7676??
Damn it 76! We are too late! This mornings actual was actually changed to this afternoons actual on SN! How's that! Anyway good subject for this,,, Judge not 76, Hope you got a fish and be sure to respect your fellow fishermen, Many might have as much, if not more salt in their veins than oneself.
can anyone help me understand more precisely how the judges arrive at their scores?
for example do the judges provide scores for individual criteria and then those criteria are weighted to produce a final score for a wave, or do they just score a single number based on their overall "gut" feel about how well the surfer did? im sure not the latter but i don't really know. in my mind there should be some solid method of how they arrive at it that must use some sort of rationale that can be backwardly traced.
happy I just spent 10 minutes on the ASP web site and I couldn't find the official version, but this is what is reported elsewhere. It's a bit weird that they are not on the site or, if they are, are so very carefully hidden.
1. Commitment and degree of difficulty.
2. Innovative and progressive manoeuvres.
3. Combination of major manoeuvres.
4. Variety of manoeuvres.
5. Speed, power and flow.
With the added 2 notes to the 5 criteria
NOTE:
It’s important to note that the emphasis of certain elements is
contingent upon the location and the conditions on the day, as well as
changes of conditions during the day.
NOTE:
The following scale may be
used to describe a Ride that is scored:
0–1.9 = Poor;
2.0–3.9 = Fair;
4.0–5.9 = Average;
6.0–7.9 = Good;
8.0–10.0 =Excellent
It's in Article 134 (page 57) of the ASP Rule Book. The PDF document.
http://www.aspworldtour.com/assets/2014_ASP_Rule_Book_2-4-14.pdf
Gday H ass. The judges are normally at the site on daybreak. Whilst watching the surfers in the water practising they throw scores amongst themselves based on their knowledge of the criteria to begin to establish some sort of scale they can agree on. Once the first heat is underway and a scoring wave has been ridden, that is, not a short ride easily discernible as poor, they will write their score down. The judges cannot see anyone else's score so the Head Judge will ensure they are all around the mark. This may require judges changing their score up or down. The ideal scenario is that all further waves ridden are compared to this score and are either better or worse or possibly similar. If conditions in the water don't change much judges can still refer back to waves ridden early in the day for comparison when scoring. It is not an exact science. It is a tough job, believe it or not. Concentrating all day from dawn til dusk. Recalling waves ridden in early heats and having an opinion that is not easily swayed. Quite often the first to arrive, the last to leave and watching great waves but not getting the chance to surf yourself. The judges take their role very seriously and I can't believe they score according to sponsors wishes or with the knowledge of what a surfer needs on a wave to progress. You spend your heat trying to get the score correct in your scale on your own scorecard. You the spend the rest of the night arguing with your fellow judges about whether your score was correct or their score was.
While attending Surfing Australia's Greenroom Conference I had the "pleasure" of talking with the ASP's head judge Richie Porter! During our discussion I told Richie about an idea I had for a computerised judging aid for surfing that I was considering developing as part of my thesis for a masters in electrical engineering. I explained that I could assess all of the criteria through signal processing such as power, speed and flow. I was bit taken back when this idea was met with obvious disgust and fear by MR Porter but upon reflection I cam to understand his reaction. It would be like telling a parking meter attendant that you have come up with a parking meter that maintains its self and issues the fines!
Sadly though, the ASP will always suffer from corruption more often than not fuelled by nepotism! When you have a company like the ASP and so many people aspiring to be involved with such a company, those on the inside tend to play their roles in a defensive manner in an effort to maintain what they have. This is exactly what Richie Porter was doing.
Sadly what MR Ported did not realise was that this computerised judging aid will give the sport and it's judging credibility! Due to this general fear from the surfing community I changed my specialisation and am doing renewable energy system. Sadly for surfing fan's and the ASP they will have to pay me to develop such a system in the future. Yet another opportunity missed for the surfing community due to lack of founded knowledge and adequate understanding and vision.
Maybe the ASP should merge with the Catholic Church! They both seem to proving their own irrelevance and both believe they are infallible!
You should not give up on that idea. I had a similar idea in my head though I am in a position to follow it through as I'm just a software developer and long time surfer. I thought it pretty poor that Richie discounted your idea. I was thinking that they could spend a year or two calibrating it to the actual visual scoring and as you said it could just be used as a guide. It would definitely act as scientific control or baseline and sway against some of the emotional variants that we see so often in judging.
ksurf. thanks for this...I didn't know that they did benchmarking during practice...seems very sensible though.
I still cant help but feel that there should be more rigour as to the methods that judges arrive at their own individual final scores. blindboy has pointed to at least 5 criteria that the judges use. im also aware of others such as wave size/quality, and clean finish, and apparently also "style" - although im not so sure that really features in the scores so much. given all these criteria exist then possibly these can be individually used to score against and then somehow merged (i.e. added/ multiplied whatever - get some math guru to work it out) to produce a final score for each judge.
the judges clearly already do all of what ive proposed but in their heads and so what ends up on the page is their overall "gut" feel after having thought about and mulled over everything. the problem with this is that even for the most intelligent human complex patterns can get difficult work through and prone to error.
methods like this might help the head judge to better breakdown into the raw elements how individual judges are either on the mark or not, and better understand how different judges perceive the same ride, without taking away from the experience, expertise of the judges which they clearly have.
interesting thought - how do Olympic judges (say gymnastics) arrive at their scores? its obviously completely different in that gymnastics is a set routine performed in a stable environment and surfing is almost exactly the opposite to this but maybe there are some opportunities to learn from?
look up Wikipedia for "Code of Points (artistic gymnastics)". apparently back in 1996 gymnastics discovered issues with objectivity of the judging and they took some measures to address. I didn't look at it in detail but clearly the problem is not confined to surfing.
Here's some food for thought: Blindboy says his system (judging surfers out of 20 not 100) only works when the heat difference between surfers is 0.9 out of 10. Well, in this interview with Richie Porta he says "Last year out of the 510 heats, 96 of them were decided by less than 3 quarters of a point."
So that's at least 96 heats that could've gone the other way with a tighter judging scale.
The system I would suggest is simply one way around the problem of trying to be too precise. In essence it would use the median of the judges scores or an average of the three central scores rounded off to a single decimal place. Judges would be restricted to scoring .0 or .5 on a 10pt scale or go to a 20pt scale. Ties would be split by a third wave. If I was the ASP I would be taking advice about this as there may be other simpler or fairer ways to do it.
In his interview Richie Porta says how often he receives complaints and questions about judging but he does not address the issues I have raised except, as Stu has pointed out, to demonstrate how frequently the present system is put under stress.
BB your way makes more sense and seems like it would be more fairer.....you nailed it.."trying to be too precise."is exactly the problem.
Now does anybody know which power turns Richie Porta was talking about that Double John was supposed to have done after his air in the "10" point ride at Bells??
Was it the wiggles to the inside or the incomplete layback????
Porta won't last.
A basic education and some communication skills are required for the gig......and he just doesn't cut the mustard as a frontman for the most important job in the house.
His stock in trade response is a belligerent, wilful ignorance.
when he does get the boot 76, can he come to you for advice on how to get the maximum outta social welfare and also some fishing advice?
One thing I like about Richard Porta is that he takes the weight. There are lot of bosses who step aside and point at a staff member when things get tough.
But when there is a contentious score, Porta always takes responsibility. He takes the heat. Apparently, he is very available to the surfers and their coaches to discuss the scores. I'm sure that is not always pleasant.
In my view, when you are a boss, you've got to prepared to be a shit umbrella. Wear the poop so that the staff can get on with the job. When you are the judge who gave the lower score, I'm sure you appreciate Porta is out there facing the music, not you. Makes it easier to call it as you see it. What some see as arrogance, might also be a little of bit being a boss who's got your back.
It's a tough job being a judge. All those waves. Taking as much care over a 1.7 as a 9.7.
I agree with you wally. And this might be stating the obvious, but, at the highest level of the sport, with clearly incredibly tight heats, some of those heats can and will obviously go either way, it's subjective, we're talking hundredths of points separation and even the likes of say Slater and Parkinson would say oppossing surfers won in their opinion and they are probably influenced by who best suits their campaigns as to who they "see" as the winner. Someone loses. there's a breed of detractors who love to jump on and kick the judges whenever controversy arises and they band together as tho everyone thinks the judges got it wrong, when in fact there may be many silent people agreeing with the outcome. I did not analyse Jordy V Julian but I did analyse the Fanning V Nicol Pipeline one which created immense kicking and I could just as easily see Fanning as the winner, or Nichol, it was that close. But, hell, that's no good, we gotta kick and bitch on judging because a lot of us are experts and/or we are gay and in love with Dane Reynolds.
Forgive me if someones already stated this...
We simply need a system that ranks the best waves - the current system does a poor job at this.
1 point for the best wave
2 points for the second best wave, 3 for third best, 4 for fourth etc.
Lowest score wins.
That way you are simply pitting wave against wave. First wave of the heat is automatically a 1, if a better wave gets ridden the 1st gets pushed to 2nd etc etc.
The average is taken from the judges score. 1, 1, 2, 2, 2 = 1.6 = 2nd best wave.