Lately, it seems like every event has at least one score that just doesn't quite add up, or one fight where judges turn in drastically different scorecards. The question is, why? 'Bad' judges might be part of the equation, but it happens so consistently that I became convinced there had to be a deeper, more systemic issue.
After discussing the problem with my long-suffering co-host and Bloody Elbow colleague, Steph Daniels, we decided to go directly to the source and talk to Rob Hinds, an MMA referee, judge and Association of Boxing Commissions licensed MMA judge trainer, and NJSACB [New Jersey State Athletic Control Board] counsel, Nick Lembo.
I feel like to properly debate the issue as fans, we need to have a strong sense of exactly what goes into choosing and training judges, and how that contributes to the issues with judging. This will serve as an informative primer into the world of judge training, as an examination of the problems in the current system and as an argument for the changes needed to improve the situation.
How judges are chosen
The exact process for this varies from state to state. New Jersey is widely recognized as having perhaps the best athletic commission in the country, and Nick Lembo details the NJSACB procedure here:
First, there has to be an opening for a position. You have to have a substantial combat sport background, such as actual MMA fight experience, actual combat sport fight experience (kickboxing, Thai), grappling contest experience, BJJ belt rank, wrestling experience and martial arts training. If you are selected by the Commissioner to fill an opening, you have to attend a scoring seminar. At the end of that seminar, you have to score rounds of past fights that have been selected. You then start as a shadow inspector in our amateur program. Then, you work your way up to a chance at being a shadow judge. As a shadow judge, you score as the fourth judge at amateur shows, your scores are collected in real time but they don’t count. The scores turned in are reviewed against the 3 actual judge’s scores. Deviation round scores are reviewed with the shadow judge and discussed with me and the actual judges after the show.
Then, if your scores are good and after shadowing for a while, you may get a chance as an actual amateur judge. Some judges never make it to the pro ranks, some judges quit, and some take over 5 years. It varies greatly per individual. The average amount of time is three years of actual and repeated agency supervised amateur judging.
From there, you score bouts for a time and may get an opportunity to shadow pro judges at pro events and then work your way into the pro ranks. While I don't think all fighters are going to make good referees or judges, I feel that it can be of great benefit to have a fighter's perspective. As a base, solid officiating starts with a detailed understanding and knowledge of striking, wrestling, positioning on the ground, submissions, and a feel for who's controlling the flow and location.
To my mind, that's a fantastic way to choose judges and covers everything I would want a program recruiting judges to cover. Unfortunately, not all states are quite as thorough and adept, as Rob Hinds explains:
Some states and tribal commissions require training, but some do not. Some require that people get updated every 1-2 years, but some do not. Something we're very concerned about is, if you've been a boxing official for 10 years, you're automatically going to become an MMA official at the highest level. Those are the things that are challenging. Along with the rules, states and tribal commissions are allowed to do whatever they want to. They have recommendations from the ABC about what they should do, but whether they do it or not is up in the air.
Evaluating judging performances and ongoing training/testing
As with the procedure for choosing and originally training judges, the amount of ongoing training or scrutiny they receive varies pretty substantially from state to state. Generally speaking, there is no requirement for judges to be tested regularly or be forced to undergo regular training.
In New Jersey at least, training is made available to judges. There are both mandatory and optional seminars, and judge performances are reviewed after events. Nick Lembo details the procedure in New Jersey:
A post event analysis is typically made of officiating decisions. You have to continually evaluate your officials. Judge performance is rated after each event and also is reviewed on an annual basis when licenses are to be renewed. As an aside, MMADecisions.com is a useful tool to track judging scores.
Training is a key component. Attending ABC reviewed training courses and athletic commission seminars can be beneficial. Training is a big slice of the pie, but definitely not the entire pie. You could take several training courses and still not perform well under the pressure of a live event. We don’t want the best test-taker, this agency wants the individual who can perform best under fire.
I like to continually train by providing videos of a round in an actual contest. I like to show specific fights and talk about what’s going on in that fight. I like to show fights that feature a contestant utilizing an active threatening guard and that you should be able to score points from the bottom; rounds that are clear 10-8's; rounds that are lively and highly competitive, and rounds where hardly anything happens.
There is a lack of cohesion among states in terms of the scrutiny they place their officials under and the standards they are expected to meet, both in training and in practice. Rob Hinds finds this problematic:
That's one of the major challenges with officiating; training is only sometimes required, and sometimes it'll be required, but once you're trained it could be five or ten years before you get trained again, or you may never get trained again. One of the challenges we're seeing with experienced officials is that they haven't been updated on the sport or the rules in years.
More than the system, more than the rules, more than anything else; it’s the officials at different levels not being on the same page. In baseball, umpires strike zones may be a little bit difference, but ultimately, there's a guideline there. With combat sports, there's no specific guidelines anywhere.
Exactly how fights should be scored is perhaps the most controversial aspect of judging. There’s little agreement about it among fans, and that extends all the way up to officials. The guidelines provided by the ABC about what constitutes a 10-10, 10-9 or 10-8 rounds are pretty threadbare, and it’s left up to the people training the judges to fill in the gaps. While the ABC reviews and signs off on the materials and tests the trainers use, there will still be some inconsistencies between individuals. Worse, because some judges receive their assignment after establishing themselves as boxing judges, their view on how to score a round will be completely different from someone who received specific training. Rob Hinds had this to say:
The written unified rules that you see on the ABC website, not only are they outdated and not updated, but it’s a blanket statement. You don’t see any other rulebook in any other sport that doesn’t have great detail about everything. What the ABC does, is they recommend those general rules, and then it’s up to us, the trainers, to go out there and train the officials on the details, like the definition of a 10-8 round, and going into more detail about what a 10-10 round really looks like, and all of those things.
It’s a great disservice for nobody to explain to the fighters exactly what's expected of them with a 10-9 round, or a 10-8 round. You have somebody like a Rich Franklin, who will tell you, ‘My whole career, nobody has ever explained to me what the difference between a 10-9 and a 10-8 round is. Everybody has an opinion on it, but hardly anybody has the true definition of it.’
Nick Lembo made the point that fighters and their trainers are allowed, at least in New Jersey, to ask to see the official score card for fights and to have the scores explained. He also noted that the ABC is currently looking into providing a better definition of a 10-8 round:
Fighters and trainers should already be very familiar with the scoring system and criteria. If they are not, they should become familiar with such. After the fight, fight camps can ask for the official score card and an explanation of the scoring. Prior to the fight, fight camps can question the selection of the judges or file a protest. The ABC is currently working on the possibility of providing judges with an improved definition of a 10-8 round.
The definition provided by Nick Lembo of a 10-8 was very similar to, but not exactly the same as, the definition provided to us by Rob Hinds. While Rob was very specific about a 10-8 round generally requiring one fighter to dominate the other for the majority of the time, e.g. more than three minutes of the round, Lembo’s detailed definition did not specifically focus on the time, but on the dominance and impact of one fighter over the other during the round as a whole.
The fact there are differences such as this even among some of the most educated and high level officials in the sport suggests to me that an improved definition of a 10-8 round is really something of a necessity at this point.
We've established that some of the problems in judging are systemic; the lack of a truly unified recruitment and training program, and the differences between states in the quality of training and support they are capable of providing. Sometimes though, the problem is just that a judge doesn't have the knowledge required to accurately evaluate the outcome of a mixed martial arts bout. What happens then?
As with almost everything, the exact procedures vary from state to state. Nick Lembo explains New Jersey's procedure:
There are judges that we have never used, or stopped using either temporarily or permanently.
[If a judge is under performing] we have compelled required training and forced additional shadowing. There will always be controversial calls in combat sports, as well as in non-combat sports, no matter what scoring methods are utilized. We also need to understand the difference between variance of opinion in closely contested fights versus incorrect judging. I do believe that you have to replace judges who are no longer capable of providing quality scores on a consistent basis. The selection of judges needs to be closely scrutinized. We owe it to the fighters to put the best qualified judges out there.
Half Point System & Other Fixes
There have been a few things suggested to help improve the issues surrounding judging in MMA. One of the largest problems, in my opinion, is the huge variability in a 10-9 round. A 10-9 round encompasses everything from one fighter landing a couple more punches than the other, to a fighter landing dozens more punches and threatening with offensive grappling for much of the round. To combat that, Doc Hamilton proposed a half point system, which I feel would help improve the situation in fights like this.
Nick Lembo gives his opinion on the half point system:
First, I must open by stating that I do not think that any scoring system can achieve perfection. Any system needs to be applied by individuals who are subjectively interpreting the action and applying what they see to that system. There will always be controversial and even regrettable calls in sport no matter what scoring systems are in place.
For the half point system, I believe that the trials done were sufficient in scope. I don't believe that a switch to that particular system would be a prudent move at this juncture. I'm not in favor of putting in a system that's even more difficult to figure out. Let's focus on getting the 10-10, 10-9, 10-8 and 10-7 right and see how that works before we start with half points and having to debate was that a 10-9.5?, or was that a 10-10?. Was that a 10-8 or a 10-7.5 ? The half point system just makes it more complicated. The key right now is to get a consistent application of a 10-8 round where warranted.
There are other commission representatives that are very experienced in MMA. Jeff Mullen of Tennessee was a UFC Judge for some very historic contests. Andy Foster of California has significant cage time experience and practical training knowledge. Bernie Profato of Ohio oversees more MMA bouts on an annual basis than anyone in the nation. Keith Kizer has long standing experience overseeing the highest profile and most closely scrutinized contests ever held in the sport. We all seem to be on the same page on this subject matter.
Rob Hinds felt that while the half point system would be useful in limited circumstances, the other problems surrounding judging would overwhelm any improvement they may bring:
Would something like a half point system be helpful? Yes. If that closes the gap between a close round versus a clear round versus a partially dominant round versus a completely dominant round, whether it’s 10-9.5, or 10-8.5, whatever that looks like, that would be helpful.
What it ultimately comes down to, is the proper training of the criteria, and the proper training in assessing what is a completely dominant and damaging round. I think that has more play than the numbers system, but that half point numbers system is helpful.
Nick Lembo brought up another potential improvement he would like to see:
Sometimes I wonder if judges should be in noise proof rooms watching without sound on high definition big screen televisions, because high level televised promotions usually give you the perfect angles while sitting cage side may provide obstructed or limited viewpoints.
First, I need to echo the words of Nick Lembo, who said "I would be remiss if I did not say how difficult a task that MMA judging can be." Even among MMA media members and hardcore fans, who spend more of their waking moments watching and dissecting bouts than perhaps anyone should, even with perfect camera angles for every punch, take-down and submission, there is often substantial disagreement about how exactly to score a fight. As a group we can perhaps be quick to judge the judges, and a better understanding of the difficulties of their job would go a long way.
With that being said, after speaking with Mr. Lembo and Mr. Hinds, my feeling that there are systemic issues with judging remains. In my opinion, the most important issues, in no particular order, are as follows:
Judges from state to state use different training materials and are taught how to score fights in slightly different ways. While the differences may generally be small, the inconsistencies between how two judges score the exact same action can have a major impact on the outcome of fights.
Judges from state to state are held to different standards. While a state like New Jersey has a comprehensive program for evaluating and training officials, most states do not have a program of the same quality. Judges from different states will have different levels of fight education.
Different combat sports don't always have different sets of officials. In many states, being a qualified boxing judge also makes you a qualified MMA or kickboxing judge. The assumption is if you can judge one combat sport proficiently, you can judge them all proficiently.
The good news is that there are people like Mr. Lembo and Mr. Hinds who recognize the system is imperfect and who are trying to change it from within. We have commissioners like Andy Foster in California who are forming teams of officials specifically with MMA in mind. As with anything in government, these changes tend to come about slowly, but I’m optimistic for the future of officiating in the mid to long term, purely because I’m optimistic about the people who are in place trying to improve it.
You can follow Nick Lembo via his Twitter account, @NickLembo