Examining Umpire Performance in the KCL

Author: Cameron Cischke

 Umpires have a really difficult job. We’ve tasked them with telling us whether a sphere less than three inches in diameter traveling at speeds faster than cars are allowed to drive and thrown with the intention of deceiving is in a 2-foot rectangle or not. And then they have to do it again and again, hundreds of times in a single game. Every one of these split-second decisions can have massive consequences and greatly affect the outcome of games. What’s more, these decisions have come under baseball’s microscope more and more in recent years, with the advent of ball-tracking technology allowing us to see how good they really are at their jobs. Every day, 400,000 followers check in with UmpScorecards on X (formerly Twitter) to see reports on every Major League Baseball game showing missed calls and estimating the effect they have on the game’s final score. But at lower levels of baseball, umpires are largely free from this level of scrutiny. High school and even many collegiate umpires will never be shown anything like this on their performance, and yet we expect them to do their job perfectly? That doesn’t seem fair. However, with the introduction of the Yakkertech camera system this season at the Corn Crib I was able to provide some of this data to the umpires. My hope is that this kind of information will help umpires understand their performance from an objective viewpoint and improve their accuracy moving forwards. I also took a deeper look at the strike zone and walked away with some interesting observations along with some lingering questions about umpire performance at this level of baseball.

The first thing we need to establish is what exactly Yakkertech is and what it gives us. BaseballCloud describes its product as “patented optical technology that captures ball flight data.” What this practically means is that two cameras mounted on the home dugout record every pitch and calculate velocity, spin rate, pitch location, exit velocity, and much more. Throughout the summer, the analytics team worked with the data given to us by this system a lot, gaining valuable experience and providing information to coaches, players and umpires. I chose to focus on pitch location as it pertains to ball/strike calls. If we have the true pitch location and can figure out where that lies relative to the strike zone, we can determine how good an umpire is at calling balls and strikes. 

So how do we translate that into something our camera system can understand? To get this information we had to talk to our Yakkertech representative, Alfredo Caballero. The Yakkertech cameras measure pitch location in two coordinates, recorded as PlateLocHeight and PlateLocSide. PlateLocHeight records the ball’s vertical location while PlateLocSide is the horizontal location. Both of these are measured in feet. Alfredo told us that the strike zone is the area from -1 ft to 1 ft horizontally and 1.5 ft to 3.5 ft vertically, so that’s the area I’ve used for all of the analysis in this project. Admittedly this is a little bit strange, as this is a square strike zone and most interpretations of the rulebook strike zone end up being rectangular. But I’m going to go with what I was told by the company that’s providing the system. This does bring up an obvious issue though. It’s important to note that the vertical portion of this digital zone does not change with the height of the batter. In other words, if Aaron Judge and Jose Altuve dug into the box at the Corn Crib our system would register the exact same strike zone for both of them despite the fact that Judge is over a foot taller than Altuve. This means that all of the “missed calls” my research indicates at the top or bottom of the zone should be taken with a grain of salt. Some of them may be genuinely wrong, but some might just be pitches thrown to a particularly tall or short hitter.

Now that I had the strike zone I was ready to start looking at pitch locations and classifying calls as correct or incorrect. But as I started to plot some of these “missed calls,” I ran into my next issue. I was simply checking to see if the pitches coordinates were inside that strike zone and comparing the “correct” call to the call that was made on the field. However, PlateLocHeight and PlateLocSide only measure where the center of the ball is when it crosses the plate. That’s not how the strike zone works! If any piece of the ball touches the zone, it’s a strike. I had to go back into my code and update how I was classifying balls and strikes. I went through a couple of iterations, and finally ended up using the distance formula and the radius of a baseball (around 0.12 feet) to determine if any piece of the ball was inside the strike zone. This greatly improved the accuracy of my classifications and allowed me to make sure that all my plots were precisely to scale.

Once the season was over, I was excited to take a look at a full season of pitch data and apply my code to look at umpire performance across a quite respectable sample size of over 12,000 pitches. Below is a boxplot of the single-game umpire accuracies from the 2025 KCL season.

Overall, umpires consistently called between 80% and 85% of pitches correctly in any given game. That’s not great. Around 2 out of every 10 pitches in the KCL were called incorrectly. For something as crucial to the game as the strike zone, this is less than ideal. Numerous studies have shown how hitter performance changes drastically when they’re in a favorable count versus when they’re in a pitcher-friendly count, so if a hitter or pitcher is earning a favorable count and then having it stolen from them by an inconsistent umpire 2 out of 10 times this is going to affect player performance in a significant way. This plot also displays a large inconsistency surrounding pitches inside the digital strike zone area. This is especially concerning, as it indicates that the strike zone is vague and inconsistent from umpire to umpire. The zone is supposed to be a clear and consistent area, and if it isn’t then the game is being affected.

So what does this strike zone actually look like in practice? With 12,000 pitches tracked I had plenty of data to take a look at this question.

Above is a plot of the strike zone (from the umpire’s view) divided into 2.4 inch squares, colored by the called strike percentage in that square across the entire season. I had to experiment with the sizes of these squares, and landed on 2.4 as the size that allowed each square to have a significant enough number of pitches thrown into it while still displaying enough nuance to be a useful graph. I’m just using the center of the ball as reported by PlateLocSide and PlateLocHeight, no fancy geometry needed for these plots. The thicker line is the rulebook strike zone that I’ve described earlier. The thinner line is drawn around the area where pitches were called strikes 50% of the time or more, or the effective/expected strike zone. This plot concerns me. If Yakkertech is recording all of this accurately, there are pitches nearly 10 inches off of the plate towards a right handed batter that are being called strikes more than 50% of the time. There has to be some kind of explanation for it. My first thought was that those pitches are being called only against left-handed batters and that maybe the outside corner is just really big. Only 33% of pitches thrown this past KCL season were to left-handed hitters but if many umpires were calling a huge outside corner to them specifically that could result in the strike percentages shown above. Unfortunately, this explanation didn’t pan out. Below is the same plot, this time split between right and left handed hitters.

If anything, the zone to left-handed hitters shows less expansion on that side of the plate. So, my outside corner explanation doesn’t hold up. Does this mean that umpires are actually calling pitches nearly a foot off the plate inside to righties? Those pitches would nearly be hitting them. I also took a look at the sample size and this trend remains even filtering for boxes that have more than 20 pitches thrown in them. This leads me to wonder if there’s an error in the data collection process, like a slight miscalibration of YakkerTech or incorrect readings. We noticed consistent issues in the cameras’ performance as it got later in the evening, and this side of the plate is also the side that is further away from the home dugout where the units are mounted. Maybe this is causing a skew in its readings on that side that cause it to exaggerate how far off the plate certain pitches are?

These questions don’t make me ready to throw out all of my research or question my conclusions. There’s a significant amount of this data that makes sense, and the fact that it seems slightly skewed on that side of the plate doesn’t change that. No matter how you interpret that bulge out towards a right-handed batter it’s clear that the edges of the strike zone are nowhere near as consistent as they should be. But how could we expect it to be consistent if umpires never receive any more feedback than coaches yelling at them when a call goes against their team? I don’t want to come across as attacking umpires. I spent nine years when I was younger refereeing youth ice hockey, so I understand how challenging officiating sports can be and I know that officials really do just want the calls to be right. So I tried to help provide them with some of the feedback that players get in abundance. For each game, I generated a one-page pdf file shamelessly stolen from the amazing work done by the @UmpScorecards Twitter account containing some basic metrics and visuals detailing the umpire performance. I was in contact with the KCL umpire coordinator, and sent these over to him for every game of the season. An example report is below, from what my accuracy model thinks is the best called game of the season (all of the reports along with my code to generate them and the plots I’ve used in this article are on my GitHub).

The report begins by listing some details on the specific game, such as the teams that played and the date. The next section is focused on overall accuracy, or just simply the number of pitches my model thinks were called correctly out of the total number of pitches the umpire was responsible for calling (called balls or strikes, excluding all pitches the batter swung at). It also displays the percentile ranking of the game’s accuracy for the KCL season. The final section focuses on the missed calls, showing a plot of every pitch my model thinks was called incorrectly alongside ring graphs showing the umpire’s accuracies inside and outside of the digital strike zone. Another thing to note on the missed calls strike zone plot is the red outline of the KCL Expected Zone that I mentioned above. Once again, this is a contour around the area where umpires were calling pitches strikes 50% of the time or more. I added this to put the umpire’s missed calls in context of what was consistently being called.

If umpires can be provided with data like this after their performances, I would confidently suggest that performance would improve drastically. I only started this research partway through the season so that kind of trend was not something I was able to follow. As I continue to build on this research, I would love to be able to track umpire performance over time as they are provided with these reports. 

It seems like Major League Baseball is moving towards an automated system of determining balls and strikes, and I’m generally in favor of that. The testing of the strike zone challenge system in this year’s spring training was generally well-received, and there are rumors that a similar system is being considered for usage very soon in the regular season. I think this is good for baseball. It’s important that the right calls are made. But even if MLB goes fully robo-ump, we need to consider the lower levels of baseball. Automatic Ball-Strike systems are well out of the reach of leagues like the KCL, and if players are developing in an environment where the strike zone is a vague, poorly-enforced concept it’s going to hurt the game going forwards. Hitters are going to be penalized for having a good eye for the zone. Pitchers are going to skate by on calls that aren’t going to be there at the upper levels of the game. Beyond this, right now what’s being considered at the Major League level is still only challenge-based. Getting the call right the first time is still extremely important, so we need to give our umpires feedback to help them develop the same way we do with players. The more stadiums that are willing to install a system like YakkerTech and help our umpires with their impossible job, the better the game of baseball will be.

I want to thank Jarret Rodgers and the Cornbelters for the opportunity to do this research. This past summer was an invaluable experience for me, and I’m incredibly grateful for the chance I got to work with real world baseball data and go to work at a baseball stadium every night. I also want to shout out the amazing group of guys that made up the Corn Crib Analytics Crew in 2025. It’s hard to find a group that loves baseball as much as these guys do, and it was incredible to get to experience a great summer of baseball with all of them.