Stuff+ in Collegiate Baseball

Author: Max Quirk, Computer Science and Mathematics Major at Illinois State University, Normal CornBelters Analytics Intern.

0. Introduction

Grading pitches and scouting players has been a central focus for baseball scouts and organizations for decades. As technology has advanced, so too have the tools used by scouts, players, and front offices. One of the most important factors in evaluating a pitcher’s future value is a simple but critical question: How good is his stuff? This paper introduces a data-driven approach to answering that question. By leveraging advanced pitch-tracking data and machine learning, I present a model that grades pitch “stuff” based on its movement profile and velocity characteristics. The goal is to provide a consistent, objective framework for evaluating pitchers’ raw ability. Complementing traditional scouting methods and enhancing player development strategies. Stuff+ is a way to describe the effectiveness of a pitcher’s arsenal.

1. Data Acquisition

All data used in this Stuff+ model was sourced from Yakkertech, a dual-camera tracking system widely used across collegiate and professional baseball. Yakkertech captures high-resolution pitch-level data including velocity, spin rate, spin axis, release point, and both vertical and horizontal movement. The system provides reliable and consistent tracking metrics that are essential for pitch characteristics. The dataset used in this study includes all pitches thrown during the 2025 CornBelters and KCL seasons.

1.1 Data Selection

The features we are going to be using for the Stuff+ model are listed below followed with a short description.

• Release Speed (RelSpeed) represents the velocity of the pitch as it leaves the pitcher’s hand.
• ZoneSpeed measures the velocity of the pitch as it crosses home plate.
• Spin Rate is recorded in revolutions per minute (RPM) and reflects how fast the baseball spins after release.
• Induced Vertical Break (IVB) measures the vertical movement of a pitch after removing the effects of gravity. It measures how much the ball appears to “rise” or “drop” due to spin and seam orientation.
• Horizontal Break (HB) captures the lateral movement of the pitch compared to a spinless trajectory. Negative values are towards first base side, while positive values represent movement toward third base.
• Release Height (RelHeight) is the vertical distance between the ground and the pitcher’s release point.
• Release Side (RelSide) is the horizontal distance from the center of the pitching rubber to the release point. This metric varies significantly between pitchers depending on their position on the mound and their arm slot.
• Extension measures how far down the mound the ball is released from, relative to the pitching rubber.
• Vertical Release Angle (VertRelAngle) measures the angle at which a pitch is released vertically, relative to a flat horizontal plane. It reflects the tilt or over-the-top nature of the delivery.
• Vertical Approach Angle (VAA) is the vertical angle at which the pitch crosses home plate. Fastballs with flatter VAAs tend to generate more swing-and-miss, as they appear to “ride” through the strike zone.
• FF_diff is the difference in velocity between an offspeed pitch and the pitcher’s average fastball.
• IVB_diff measures the change in induced vertical break compared to the fastball.
• HB_diff is the difference in horizontal movement relative to the fastball.
• ang_diff represents the difference in vertical release angle between the offspeed pitch and the fastball.

These differential metrics help quantify how each pitch type deviates from the pitcher’s fastball, capturing elements of deception, tunneling, and pitch shape variation — all of which contribute to the effectiveness of a pitcher’s arsenal.

1.3 Target Selection

The target variable for this Stuff+ model is Run Value (RV). A measure of how a given pitch contributes to the expected number of runs scored. While sites like FanGraphs provide run value metrics for Major League Baseball, these values are calibrated to MLB-specific run environments. To tailor the model to the collegiate level, I created a custom Delta Run Expectancy (ΔRE) framework. In short, delta run expectancy measures the change in expected runs scored from before to after a pitch outcome, capturing both immediate runs scored and changes in future scoring potential.

 It is calculated based on the number of outs and the configuration of base runners before and after the play. For example, a strikeout with runners on may reduce run expectancy by increasing the out count without advancing runners, while a single could increase run expectancy by moving runners into more favorable scoring positions, or scoring them outright. This approach relies on a run expectancy matrix, which averages the number of runs scored for each of the 24 base-out states (combinations of 0, 1, or 2 outs and all possible runner configurations). By comparing the expectancy before and after a play, delta run expectancy isolates the specific impact of each pitch or outcome within its game context. Incorporating this metric into the Stuff+ framework allows us to more accurately evaluate pitch quality based not only on physical traits (like velocity and movement) but also on how those pitches impact run prevention.

2. Modeling

To train the Stuff+ model, I used a pipelined XGBoost Regressor. XGBoost is a popular gradient boosting algorithm that works well with structured data and is known for its speed and accuracy. It helps the model learn patterns in the data by combining many smaller decision trees into a strong overall predictor. Before training, I applied a Robust Scaler to the input features. This step reduces the impact of outliers, which are common in baseball tracking data, especially with variables like spin rate or release angle. By scaling the features first, it helps the model make more stable and reliable predictions. Using a pipeline ensures that all steps are done in the right order, both during training and when applying the model to new data. The model is trained to predict delta run expectancy, allowing it to learn how pitch characteristics relate to run value. After training, I tested the model on the rest of the data. To further analyze feature importance to understand which inputs have the most impact on the model’s predictions. This helps identify which pitch traits matter most when it comes to generating stuff.

2.1 Feature Importance

Understanding which features drive the model’s predictions is essential for interpreting what defines a pitcher’s “stuff.” After training the XGBoost model, I examined feature importance scores to identify which variables had the greatest impact on predicted run value. The most influential features include Release Speed (RelSpeed), pitch type (specifically sliders), Extension, Spin Rate, horizontal break differential (hb_diff), total horizontal break (HorzBreak), and Induced Vertical Break (IVB). These variables consistently showed the highest gain in the model, meaning they contributed most to reducing prediction error.

3. Evaluation

Once the XGBoost model predicts the expected run value impact of a pitch (based on its physical characteristics), the output is transformed into a standardized Stuff+ score using a custom scaling method. The predict_stuff_plus function takes a pitch event as input and checks that all required features are present. If so, it feeds the feature values into the trained model to predict the raw output. A continuous value representing the pitch’s estimated impact on delta run expectancy. To make these values more interpretable, the raw prediction is standardized using the mean and standard deviation of the model’s training outputs. The standardization formula is:

Stuff+ = 100 + (prediction − mean std × scale factor)

Here, a mean of -0.021 and a standard deviation of 0.1689 are used, based on the training set distribution. The scale factor is set to 10 so that each standard deviation above or below the mean translates to a 10-point change in the Stuff+ score. A score of 100 represents league-average “stuff,” while scores above or below 100 indicate better or worse pitch quality, respectively.

3.1 Metric Discussion

Evaluating the performance of the Stuff+ model requires a comprehensive set of metrics that capture different aspects of predictive accuracy and generalizability. Since the model outputs the change in run expectancy, I am using root mean squared error to see how close the predictions are to the real values. The goal is not only to minimize error but also to understand how well the model generalizes across different pitch types, pitchers, and game situations.

I used Root Mean Squared Error as the primary evaluation metric for the model. RMSE penalizes larger errors more heavily than smaller ones and provides a direct interpretation in the units of the target variable (ΔRE). A lower RMSE indicates that the model’s predictions are more accurate. The final RMSE on our held-out test set was 0.1736, suggesting good predictive performance given the variability inherent in baseball outcomes.

4. Reports and Visualizations

The Stuff+ model provides a metric to evaluate pitcher performance across both leagues in our dataset. To demonstrate the model’s effectiveness, I present four visualizations highlighting the top 15 pitchers by average Stuff+ in the KCL and the CornBelters, as well as the top 15 pitchers across all pitch types for each team. It’s important to note that we lacked data from previous seasons, which means the model was trained and tested on data from the same season. This limitation may skew the results somewhat. Having access to prior seasons’ data would allow the model to be more finely tuned and provide more robust evaluations. There is a minimum of pitchers who have thrown at least 20 pitches and at least 10 pitches thrown of that type or the pitch is unreliable for full season data.

4.1 Top KCL Pitchers by Stuff+

4.2 Top CornBelters Pitchers by Stuff+

4.3 Top KCL Pitches by Stuff+

4.4 Top CornBelters Pitches by Stuff+

4.5 Cross-League Comparison and Insights

As you can tell by the rankings, some pitches are skewed by not having enough pitches thrown or being so unique there are no other pitches like them, so they perform well on their own. But nevertheless, these visualizations demonstrate the model’s ability to identify and rank elite pitching performance while providing insights for player development and scouting applications. The standardized Stuff+ scale allows for meaningful comparisons both within and across leagues, supporting its utility as an objective evaluation tool.

4.6 Why certain pitches perform well

As you can see in the rankings, some pitch types are more tightly grouped than others. This happens because certain pitches in the league tend to get hit harder. For example, sliders gave up a few big home runs, which led to a higher delta run expectancy and caused them to be graded more harshly. On the other hand, changeups performed well in both the KCL and CornBelters leagues, so they were graded more favorably on average.

4.7 Limitations in Evaluating Command and Location

One important limitation of the model is that it does not take pitch location or command into account. As a result, pitchers with strong control or consistent command may not be ranked as highly, even if their performance on the mound is effective. For instance, pitches like Graham Kasey’s changeup are expected to perform well not necessarily because of location, but due to how the pitch moves. His changeup features a lower spin rate and a significant difference in velocity, induced vertical break (IVB), and horizontal break (HB), and a consistent vertical release angle compared to his fastball, which makes it more deceptive and harder to hit despite the model not accounting for command.

5. Conclusion

Developing a Stuff+ metric for the KCL and CornBelters was a big task and I’m proud of both the model’s performance and the coding skills I developed throughout the process. This represents my first real step toward building an applied model for evaluating pitch quality using advanced data. While the model is currently limited by its reliance on a single season of data, it lays a solid foundation for future work. Especially in the aspect where sliders performed extremely well so much in fact it includes the pitch type being a slider as a heavy factor in being a good pitch or not. With the addition of multi-season data, the Stuff+ framework can evolve into a more predictive tool for assessing pitcher performance across all levels of competition.

6. Sources

Nestico, Thomas. “Modelling tjStuff+ v3.0.” Medium, medium.com/@thomasjamesnestico/modelling-tjstuff-v3-0-10b48294c7fb. Accessed 22 July 2025.

Langin, Chris. “Pitch Design: What Is Stuff+? Quantifying Pitches with Pitch Models.” Pitch Design: What Is Stuff+? Quantifying Pitches with Pitch Models, Driveline Baseball, 7 Apr. 2023, www.drivelinebaseball.com/2021/12/what-is-stuff-quantifying-pitches-with-pitch-models/.