A gymnast sprints down the vault runway. The cheers of hundreds of fans fuel her soaring body as she flips and twists through the air, but the judge cannot hear them. It doesn’t know she’s last in the lineup, nor does it know that the gymnast before her stuck a beautiful vault cold. All it sees is the gymnast’s performance, stripped of all expectation and meaning.
Because the judge is a robot.
The future of automated judging may be closer than you think. In 2017, the International Gymnastics Federation (FIG) announced a partnership with Fujitsu, a Japanese IT giant that has developed 3D sensory software capable of recognizing gymnastics skills and form deductions. It works by emitting 2 million lasers per second, allowing the technology to capture a 360-degree view of skeletal motion.
This “judging support system” was tested at the 2017 world championships in Montreal, and the FIG plans to use the technology to assist with scoring at the 2020 Olympic Games in Tokyo. For now, the system will not replace human judging but will instead be used as a fallback in case inquiries or scoring controversies arise. Still, its debut marks a momentous change in a system of scoring that has remained largely stagnant throughout the history of the sport.
An Imperfect System
The FIG believes this technology will help with a number of problems in elite judging. The first is fatigue, as judges often work long hours for many days in a row, which can have an impact on attention span and accuracy. Another is inherent bias in human scoring. Studies have shown that judges tend to score gymnasts of their same nationality higher than those representing other nations.
These issues are not unique to elite competition. Due to a shortage of NCAA judges, many have to leave quickly from one meet to the next, often traveling long distances between them. Integrity is also questioned at the collegiate level, where factors like a judge’s alma mater or an athlete’s prior career successes can have a subconscious effect on scoring. This season, inflated marks became so problematic that the national judging association sent out a memo warning all judges to “make sure your scores are justifiable when reviewed by knowledgeable professionals.”
Jen Llewellyn, head coach of 2019 USAG national champion Lindenwood, believes overscoring is a problem in the NCAA but that it often balances itself out. Her DII team tends to get lower scores when competing against DI teams but then receives the benefit of the doubt when it goes up against other DII or DIII programs.
“My biggest frustration is just the consistency or lack thereof from coast to coast and region to conference to time of day even, or fan base,” she said. “There are just so many factors that can persuade or affect the human factor that comes with judging.”
But what if those factors could be eliminated? If Fujitsu’s technology is successful in elite gymnastics, it is only a matter of time until it trickles down to the collegiate and club levels. Could artificial intelligence finally put an end to “crack scoring” in NCAA gymnastics?
Why Not Try?
It would certainly help keep things consistent across the country, according to an NCAA gymnastics judge who has asked to remain anonymous. “That’s also one thing people complain about, and this is a valid complaint: that depending on where your team is competing and who the judges are and where they’re from—it changes the scoring. If you have robots that don’t know which region they’re in, it would definitely help in that regard.”
The judge agrees that bias can be a problem in collegiate scoring. “I feel it sometimes when I’m judging,” she said. “The judge I’ll be working with will have this huge score, and I’m like ‘well, we’re in range, but no offense—how did you get that score?’”
Kathy Johnson Clarke, a member of the 1984 U.S. Olympic gymnastics team and longtime sports commentator, said it is hard to comment on a system that’s still largely theoretical. But if it proves to be a successful tool in elite, she would be open to the idea. “Technology can be amazing,” Clarke said. “If they can figure out a way for technology to help, supplement or even facilitate better judging, great. Why not try.”
Llewellyn expressed a similar sentiment. “It’s so cool to think about it and talk about it and explore because there’s beauty in technology and everything it can do for us as a society,” she said. And while it’s still unclear how automated judging will be used in practice, “it’s definitely an interesting conversation to have.”
While this artificial intelligence has the potential to revolutionize the way collegiate gymnastics is scored, it also introduces many key points of concern. The first revolves around logistics: What happens if the technology fails during a competition?
The NCAA judge pointed out that at a meet she worked this season, the live scoring system failed, and the head judge had to input all of the scores manually from the competition. “So if we have robots, what’s going to happen if the server with the robot is not working?” she said.
Llewellyn echoed these concerns. “What if your Wi-Fi or your hardwire technology doesn’t work? What’s your back up? With judging now, there’s always a backup mechanism,” she said.
Another major issue Llewellyn recognized was the cost. She estimated that half of NCAA gymnastics programs have to worry about budget restrictions. This 2019 season, the UIC cut both its men and women’s programs due to funding-related reasons.
“How much of an investment would the athletic department or program have to make long term and short term, and would it be something that would make an administrator not want to keep the sport?” Llewellyn said.
AI has the potential to level the playing field between divisions, but what if it actually ends up doing the opposite. “Some administrations would be all for it and would be like, ‘Oh my gosh, this is brilliant, whatever you need to do to be successful and to create a better environment for our student athletes and our fans,’” Llewellyn said. “And there’s the other half that just can’t [afford it] because it’s just the nature of the beast.”
The Human Factor
Then there is the heart of the discussion: What happens to gymnastics if robots replace humans? In what ways will it fundamentally change, for better or for worse?
“There are just things you have to take into consideration that I don’t know a computer can do,” Clarke said. “It certainly can’t judge artistry or expression or choreography.”
She also explained that the human eye isn’t as perceptive as you’d think. One tool the team uses at ESPN is the “split-o-meter,” which tracks the degrees a split leap or jump reaches in the air. Sometimes a gymnast’s leap looks short of 180 degrees, but there may be one or two freeze frames where she actually did reach a full split—the human eye just couldn’t see it.
So to hit that perfect mark, “they really have to go beyond,” Clarke said. “[There needs to be] 10, 15, 20 frames where they’re at split or more to register it.” This is an example of the type of nuance between a human and a computer that will have to be considered before implementing AI scoring.
Llewellyn also spoke to some human-centric challenges with this type of technology. “There’s that aspect of seeing a judge flash a 10 or seeing a judge flash a score that they don’t like, and you see that fan interaction with the judges,” she said. These things factor into the entertainment experience, which is incredibly important at a time of growth in collegiate gymnastics. “There’s always a human element that needs to be there that I don’t think technology could ever quite recreate.”
How will disputes be resolved if the human evaluator and AI system disagree, the NCAA judge wondered. “Is the robot automatically correct because it’s a robot?” She foresees the automated system making errors too, like mistaking one element for another. Relying solely on technology would be a mistake, “but I wouldn’t mind it as a supplement,” she said.
Fujitsu’s marketing claims the technology is “evolving gymnastics to the next stage.” If this is true, the next stage has to be defined. It will be important for the NCAA to carefully evaluate all of the ways technology will impact collegiate gymnastics before radically changing the system. But if the next stage means an improvement in the overall athlete experience—why not try?
Article by Katie Norris
Like what you see? Consider donating to support our efforts throughout the year!
