About Split The Upright
An NFL game outcome prediction system built on ensemble machine learning, validated season by season against the games it never saw, and published alongside the market so you can see exactly where it agrees and where it does not.
Background
I'm Keirnen, a University of Colorado Boulder graduate with a degree in Information Science specializing in data science. Split The Upright represents the culmination of applied machine learning research aimed at quantifying the predictability of NFL game outcomes, and, just as importantly, at measuring honestly how much of that predictability is real.
The project integrates multiple data sources, a leak-proof feature engineering pipeline, and an ensemble of four classifiers to generate win probabilities for every game. Every change to the model is judged by a walk-forward backtest before it ships, and every published pick is graded against the final score on the Track Record page.
Project Overview
Each week, the system ingests team performance metrics, the official injury report, kickoff-time weather forecasts, quarterback ratings, and the market spread to produce three numbers for every scheduled game: what the model thinks, what the market implies, and the published pick that combines them.
This platform is the public implementation of that pipeline. The methodology below is the one actually running; where an earlier version of the site described something different, the text has been corrected rather than the claim defended.
Navigation Guide
Weekly Picks
Game-by-game picks with the model's probability, the market-implied probability, and the published probability side by side, plus the model's own point-spread projection for reference.
Model Edges
The handful of games each season where the published pick lands on the opposite side of the market favorite. These are counted and graded separately, because they are the only place the model can claim an edge over simply following the line.
Confidence Tiers
Picks are grouped into High (70% and up), Medium (55 to 70%) and Low (under 55%) tiers from the published probability. Tier accuracy is tracked live so the labels can be checked against results.
Stat Lab
Season-to-date team and player metrics from play-by-play data, including the same Elo rating the model uses, available for the current season and the three before it.
Track Record
Every published pick, graded against the final score, with weekly and cumulative accuracy, tier calibration, the Model Edges record and Brier score. No selection: if a winner was published, it counts.
Player Props and Parlays
A separate projection engine for rushing, receiving, passing and touchdown props against sportsbook lines, validated on held-out seasons, with three pre-built parlays per week.
Matchup Simulator
The Matchup Simulator provides on-demand predictions for any NFL matchup, using the same models and the same preprocessing as the weekly picks while allowing control over the contextual variables.
API Architecture
The simulator runs on a FastAPI backend deployed on Render. It loads the exact model artifacts the weekly pipeline publishes, refreshes its team ratings from this site every thirty minutes, and returns calibrated probabilities with a per-model breakdown.
Technical Implementation
Prediction Engine
Requests pass through the same four-model ensemble (Logistic Regression, Random Forest, XGBoost, LightGBM) used for weekly picks. The four probabilities are combined with fixed weights into one ensemble probability and capped at 75%. There is no post-hoc calibrator: a walk-forward test showed the raw ensemble is already well calibrated and that isotonic regression made it worse.
Model-Only Output
Because a hypothetical matchup has no betting line, the Simulator shows the model's own probability rather than the market-anchored number published for real games. Expect it to be bolder than the weekly board.
Feature Vector Construction
For each matchup, the API builds the 53-feature vector the models were trained on: rolling team form, Elo, quarterback ratings, opponent-adjusted efficiency, matchup differentials and the user-specified context, then runs it through the training pipeline's imputer and scaler.
Transparency
Every response includes the individual model probabilities, the confidence tier, and the spread regressor's point projection, so the decision can be inspected rather than taken on faith.
Configurable Parameters
Environmental Conditions
Weather: Temperature, wind speed and precipitation are encoded as passing-condition scores. Dome games override them with neutral conditions.
Neutral Site: Removes the home-field term from the Elo calculation and the location-based rest and travel features.
Injury Scenarios
Quarterback: Marking a starter out drops that team's effective quarterback rating to a replacement level, the same adjustment the weekly pipeline applies from the official report. Skill-position and defensive injuries feed the injury features as EPA lost.
Team Strength Modifiers
Manual strength adjustments of up to 15% in either direction for roster changes, coaching transitions, or anticipated regression that the season-to-date data has not caught up with.
Historical Comparisons
Simulate any matchup using team profiles from the current season or any of the previous three (for example, the 2023 49ers against this year's Seahawks) via the season selectors.
Statistics Display
Following each prediction, the interface presents a side-by-side comparison of team statistics across three categories:
Advanced Metrics
Elo rating, Pythagorean win expectation and point differential: season-level indicators of team strength.
Offensive Statistics
EPA per play, pass and rush EPA splits, success rate, yards per play, third-down conversion, completion percentage, turnover rate and red-zone touchdown percentage.
Defensive Statistics
EPA allowed per play, pass EPA allowed, opponent success rate, yards allowed per play, takeaway rate and red-zone defense.
The stronger team in each category is highlighted, which makes the model's reasoning easier to follow even when its number disagrees with your instinct.
Cold Start Notice
The prediction API runs on Render's free tier. The first request after a period of inactivity may take 30 to 60 seconds while the server starts; subsequent predictions return in under two seconds.
Technical Methodology
A detailed examination of the pipeline, from raw data through the published probability.
Data Sources
The system aggregates data from several sources to construct its analytical foundation:
Play-by-Play Data
Complete play-level data from 2015 to the present via the nflverse project (nflreadpy), aggregated to roughly 6,200 team-game observations including playoffs.
Injury Reports
The official league injury report via nflverse, supplemented during the week by a direct pull from NFL.com so Friday designations reach the Saturday refresh. Players are matched to their EPA contribution to weight what a team is missing.
Weather
OpenWeather forecasts requested for the kickoff hour at the stadium's coordinates, so the features reflect expected game-time conditions rather than the forecast at publication.
Market Lines
Closing spreads from the nflverse schedule for history, and the current line at publication for the week's board. The model never sees a line; the published pick does.
Advanced Analytics
Expected Points Added (EPA) and success rate from play-by-play, Next Gen Stats tracking data, and FTN charting.
Sportsbook Props
Player prop lines from the Odds API for the props engine, archived each week so that projections can later be scored against the lines that were actually posted.
Feature Engineering Pipeline
The pipeline constructs more than 200 candidate features per team-game. Rolling windows were consolidated to the most informative length per statistic and low-importance features were removed, leaving 53 features. That list is now frozen: it changes only when a new list beats the current one in the backtest, because re-selecting features on every weekly run was quietly producing a different model each week.
Feature Taxonomy
Offensive Efficiency
EPA per play, success rate, yards per play, completion percentage, third-down conversion rate, turnover rate
Defensive Performance
Defensive EPA allowed, opponent success rate, takeaway rate, yards allowed per play
Ratings and Matchups
Elo rating and Elo win probability, opponent-adjusted offensive and defensive ratings, offense-versus-defense matchup differentials, turnover battle
Personnel and Context
Starting quarterback ratings for both teams, EPA lost to injuries, rest differential, weather, home field, division game
Rolling Form
Team statistics enter as exponentially weighted rolling averages over the last 3, 5 and 8 games (decay 0.35, so the most recent game carries about a third of the weight). Longer memories were tested and made no difference; the season-long ratings below already carry it.
Quarterback Rating
Each starter is rated on EPA per dropback over his last 32 games (decay 0.05), scaled 0 to 100 with 50 as league average. A quarterback's quality is far more stable than a team's weekly form, and lengthening this window from the team-stat default was a measurable improvement. When a starter is ruled out the team's effective rating falls to a replacement level; a true backup-specific rating is a known gap.
Opponent-Adjusted Ratings
Each team's offense and defense are rated by a ridge regression that explains every game's EPA, success rate and yards per play as offense strength minus opponent defense strength plus home field. Ratings are refit each week from that season's completed games only. For the first three weeks of a season, when there is too little data, last season's final ratings are carried in.
Elo
A standard Elo rating (K = 20, 55 points of home advantage, margin-of-victory multiplier) run week by week since 2015. Ratings carry across seasons but regress one third of the way to 1500 each offseason, a change adopted after it improved every backtest metric. It is the single most important feature family, and the same rating is shown in the Stat Lab.
Live Features Are Training Features
The feature notebook also writes the upcoming week's unplayed games as rows, built by the same code that builds training rows. The weekly pipeline reads those rows rather than reconstructing features separately, which guarantees that what the live model sees is exactly what the backtest tested.
Feature Importance Analysis
Feature importance from the XGBoost component of the current production model. Elo dominates, the quarterback features come next, and the opponent-adjusted ratings and matchup differentials fill out the top of the list. Weather, rest, head-to-head and division features rank near the bottom in every model; removing them was tested and made no difference.
Ensemble Architecture
The prediction system employs a heterogeneous ensemble combining four algorithms with complementary inductive biases:
| Model | Weight | Contribution |
|---|---|---|
| Logistic Regression | 35% | Heavily regularized linear baseline. On its own it is as accurate as the full ensemble, which says something about how linear this problem is. |
| Random Forest | 25% | Bagged decision trees for non-linear patterns, with built-in robustness to outliers. |
| XGBoost | 20% | Gradient boosted trees with native handling of missing values for the sparser tracking features. |
| LightGBM | 20% | Histogram-based gradient boosting for ensemble diversity. The weakest head individually; dropping it was tested and did not help. |
Weights are fixed. They were chosen on a validation season and are not re-tuned week to week. Hyperparameters for all four models were found with Optuna and are likewise frozen until a re-tune passes the backtest.
Probability, Calibration and the Market Anchor
A probability is only useful if it means what it says. A model that outputs 70% should win about 70% of those games. This pipeline gets there in three steps, two of which are deliberately simple.
Raw Ensemble, No Calibrator
The weighted ensemble probability is used as is. Isotonic regression, Platt scaling and temperature scaling were each fit on earlier out-of-sample seasons and scored on the next; none beat the raw number, and isotonic was worst, collapsing every probability between 50 and 65% onto a single value. An earlier version of this site used it.
Confidence Ceiling (75%)
NFL outcomes carry substantial irreducible variance. The probability is capped at 75% (and floored at 25%). The cap is neutral on Brier score; it exists so that no pick is presented as a near certainty.
Market Anchor
The published pick is a 50/50 blend, in log-odds, of the model's probability and the probability implied by the market spread (a logistic fit of past outcomes against the closing line). The model never sees a line; the blend is applied after the fact and both inputs are shown on every card.
Why Anchor to the Market
Across six backtested seasons the model alone was 64.5% straight up, and when it disagreed with the market favorite it was right only 44% of the time. Letting the data choose the blend weight by log loss chose the market almost entirely: on probabilities, the model adds nothing the closing line does not already know. For picks, a 50/50 blend scored about 67% while cutting the disagreements from roughly 45 a season at 44% to roughly 15 a season at 57%. Those surviving disagreements are the Model Edges on the picks page. That record is small and will be noisy for years; it is also the only honest claim of an edge this site can make.
Training and Temporal Weighting
Team composition and league environment drift from season to season, so older data is informative but less so. Models are trained on every completed season through the most recent one, with each season's games weighted by recency:
| Season | Sample Weight |
|---|---|
| Most recent complete season | 2.0x |
| One season earlier | 1.5x |
| Two seasons earlier | 1.2x |
| Three seasons earlier | 1.0x |
| Four or more seasons earlier | 0.8x down to 0.1x |
The in-progress season is not used for training. Refitting mid-season on the games played so far was tested in the backtest and produced no gain, so the model that starts the season is the model that finishes it. In-season information reaches the picks through the features (Elo, rolling form, adjusted ratings, injuries), not through retraining.
Validation and Data Integrity
Early iterations reported accuracy above 75%, traced to subtle leakage of future information into features. The pipeline was audited and rebuilt around a few rules that are now enforced mechanically:
Temporal Isolation
Every rolling, rating and adjustment feature uses only games completed before the one being predicted. Week 5 features see Weeks 1 to 4.
Walk-Forward Backtest
The reference test trains on all seasons before a given season and predicts that season, repeated for 2020 through 2025: 1,688 games, each predicted by a model that never saw it. Every proposed change runs through it and is adopted only if it improves accuracy and Brier score.
Live Equals Tested
The weekly pipeline loads the exact artifacts the backtest validated and reads features produced by the training code path. A 2026 audit found the live path had drifted from the tested one in several ways; closing that gap is the reason this section exists.
Determinism
Feature list and hyperparameters are frozen, tie games are handled consistently, and the starting quarterback for each game is identified deterministically, so two runs on the same data produce the same model.
Validated accuracy on genuinely unseen data is the only number that matters here. Inflated metrics from leaked information are not reported, and neither is the best single season on its own.
Model Performance
Walked forward one season at a time over 2020-2025 (1,688 games), the model alone scores 65.3% straight up with a Brier score of 0.219, and the published market-anchored pick scores about 67%. Season-by-season the model ranged from 60% (2023) to 68% (2024). Live, week-by-week results for the current season are on the Track Record page. Contextual benchmarks:
Vegas Favorite
Wins straight up about 66 to 67% of the time, Brier 0.210 (closing line, 2016-2025)
Always Pick Home
About 53 to 56% straight up over the same span
This Model
65.3% straight up and Brier 0.219 with no betting line as an input; about 67% once the published pick is anchored to the market
Weekly variance remains substantial. Individual weeks range from roughly 50% to 85% depending on how many favorites fall. The objective is a demonstrable, repeatable edge over full-season samples, not a hot week.
Key Technical Insights
Data integrity supersedes model sophistication
Architectural complexity yields minimal benefit when the underlying pipeline contains temporal contamination. The biggest accuracy gains in this project's history came from removing leaks, not adding models.
The live pipeline must be the tested pipeline
A backtest validates one specific configuration. If the weekly code reconstructs features its own way, retrains with different settings or re-selects inputs, the published number belongs to a model nobody tested. Loading the validated artifacts is not a shortcut; it is the point.
The market is the baseline, not the opponent
The closing line is the most accurate single forecast available. A model's value is not in disagreeing with it often; it is in the small, systematic corrections that survive being blended with it, and in being honest about how few of those there are.
Stable things deserve long memories
Quarterback quality and team strength change slowly; week-to-week form is mostly noise the market has already priced. Lengthening the quarterback rating's window helped; lengthening the team-form window did not.
Irreducible uncertainty defines the ceiling
Favorites win about two thirds of NFL games and no public model sustainably does much better. The goal is a calibrated probability and a small, real edge, not deterministic prediction.
Disclaimer
Important Notice
This platform is a personal research project developed for educational and analytical purposes. The predictions and analyses presented do not constitute financial or gambling advice. Anyone who incorporates them into wagering decisions does so at their own discretion and risk.