Sack Prediction with Logistic Regression

I trained three models on 60,000 dropbacks to predict sacks from what's visible before the snap, then tested them on the 2023 season. They found real signal, but much less than the usual metrics make it look.

The setup

A sack is a quarterback tackled behind the line before he can throw. Each one costs an offense roughly 1.5–2 expected points, which makes sacks among the most damaging plays a defense can make. If a defensive coordinator could pick the snaps most likely to end in a sack, or an offensive coordinator could pick the snaps that need extra protection, that would be worth a lot.

The data is every regular-season dropback from 2020–2023 that has participation data: the number of pass rushers and defenders in the box, charted from broadcast film. That comes to 81,390 plays. The models train on 2020–2022 and are tested only on 2023, so every result below is out-of-sample and in time order, the way a team would actually use it.

Look at rates, not counts

A common mistake in sack analysis is to chart sack counts. Sending four rushers produces by far the most sacks, but that's only because teams rush four on 64% of dropbacks. The useful question is the rate.

Figure 1: Sack rate by number of pass rushers. All 2020–23 dropbacks with participation data.
Figure 2: Sack rate by down and distance. Cells with fewer than 100 dropbacks are left blank.

Third down nearly doubles the sack rate (9.3% vs. 4.9% on first down). The worst cells are 3rd and 4th & 7–10, at about 11%. On obvious passing downs the defense can pin its ears back, and quarterbacks hold the ball longer waiting for a receiver to clear the sticks. Interestingly, 3rd & 11+ is less dangerous (8.9%). A likely reason is that offenses facing very long yardage often settle for a quick check-down instead of waiting for a downfield route.

Three models

The features are all things you could know before the snap: down (one-hot encoded), distance, field position, game clock, score differential, defenders in the box, number of pass rushers, shotgun formation, nflfastR's pre-snap pass probability (xpass), and an "obvious passing down" flag (3rd & 6+). Strictly, the number of rushers isn't known until the snap, but a defense chooses it beforehand, so it's a fair input for a model meant to help call a defense. 

from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import brier_score_loss, roc_auc_score

FEATURES = ["down", "ydstogo", "yardline_100", "game_seconds_remaining",
            "score_differential", "defenders_in_box", "number_of_pass_rushers",
            "shotgun", "xpass", "obvious_pass"]

train, test = df[df.season < 2023], df[df.season == 2023]
X_train = pd.get_dummies(train[FEATURES], columns=["down"], dtype=float)
X_test = pd.get_dummies(test[FEATURES], columns=["down"], dtype=float)

model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000))
model.fit(X_train, train.sack)
p = model.predict_proba(X_test)[:, 1]

baseline = np.full(len(test), train.sack.mean())
print("Brier:", brier_score_loss(test.sack, p), "vs. baseline", brier_score_loss(test.sack, baseline))
print("AUC:  ", roc_auc_score(test.sack, p))
Figure 3: How well each model ranks 2023 dropbacks. AUC = the probability that a randomly chosen sack is rated riskier than a randomly chosen non-sack.

The logistic regression scores a Brier score of 0.0627, which looks excellent at first: it's the mean squared error of the predicted probabilities, and lower is better. But a "model" that predicts 6.2% for every single play scores 0.0634. The real improvement is 1.1%. The fancier models do no better. The random forest ties, and XGBoost is slightly worse.

That doesn't make the model useless. Ranking plays is a different job from predicting them precisely, and on ranking the model does have real signal (AUC 0.62, where 0.5 is a coin flip). Sorted by predicted risk, the top tenth of 2023 dropbacks were sacked at 3.1 times the rate of the bottom tenth.

Figure 4: Calibration on 2023: predicted vs. actual sack rate. 2023 dropbacks split into ten equal-sized groups by predicted probability.

What drives the prediction

With every feature standardized, the logistic regression coefficients become directly comparable. Figure 5 converts them to odds ratios per one standard deviation of each feature.

Figure 5: Effect of each feature on the odds of a sack. Odds multiplier for a one-standard-deviation increase (for down and shotgun, for that category being true). Bars start at 1.0, which means no effect.
  • Pass rushers (×1.28 per SD, about 0.8 rushers): the biggest single lever, and the only one the defense directly controls.
  • xpass (×1.19): when everyone in the stadium knows a pass is coming, rushers can commit to it.
  • Shotgun (×0.91): starting five yards deep gives a quarterback time to see pressure coming and get rid of the ball.
  • Defenders in the box (×0.98): basically no effect once the number of actual rushers is known. A crowded box is a disguise, not a pass rush.

Where the missing signal is: pressure

Why can't a pre-snap model do better? Because sacks are decided after the snap, by pressure. Starting in 2023 the participation data includes charted pressure from FTN. It shows that 29.5% of dropbacks were pressured, and 23% of those pressures ended in a sack. The data codes every sack as a pressure, so pressure is effectively a requirement.

Figure 6: Pressure rate and sack rate by number of rushers, 2023. Share of all 2023 dropbacks in each group.

Blitzing doubles the pressure rate, from 21% with three or fewer rushers to 43% with six or more, but the conversion rate from pressure to sack stays between 20% and 25%. Whether a given pressure becomes a sack depends on things no pre-snap model can see: a missed block, a quarterback holding the ball, a receiver who never gets open. That's where most of a sack's unpredictability lives.

Is it the quarterback?

A common claim in football analytics is that sacks are partly a quarterback stat: some QBs get rid of the ball under pressure, and others hold it and take the sack. With three seasons of charted pressure now available (2023–2025), that's testable. For every QB with at least 100 pressured dropbacks in a season, I computed his pressure-to-sack rate: the share of his pressures that ended in a sack. 

qb = (dropbacks.groupby(["season", "passer_id", "passer"])
               .agg(dropbacks=("sack", "size"),
                    pressures=("was_pressure", "sum"),
                    sacks=("sack", "sum"))
               .reset_index())
qb["p2s"] = qb.sacks / qb.pressures               # pressure-to-sack rate
qb["pressure_rate"] = qb.pressures / qb.dropbacks
qb = qb[qb.pressures >= 100]

# Pair each QB-season with the same QB's next season
nxt = qb.assign(season=qb.season - 1)
pairs = qb.merge(nxt, on=["season", "passer_id"], suffixes=("", "_next"))
pairs.p2s.corr(pairs.p2s_next)                    # 0.16
Figure 7: Share of pressures that became sacks, by quarterback, 2023 (minimum 100 pressured dropbacks).

In a single season the spread is large. Josh Allen (13%) and Patrick Mahomes (14%) turned only about one pressure in seven into a sack. Bryce Young and Zach Wilson (both 30%) took a sack on nearly one in three. That's a bigger gap than anything the pre-snap features produced.

But a skill should repeat, and this one barely does. Across 39 cases of a QB qualifying in back-to-back seasons, pressure-to-sack rate in one year correlates with the next at only r = 0.16. With so few pairs the 95% interval is wide (−0.21 to 0.51), so this doesn't prove the skill is absent. It does mean one season of it can't be trusted. How often a QB gets pressured (r = 0.26) and his overall sack rate (r = 0.30) were a little more stable.

Figure 8: Each QB's pressure-to-sack rate in one season vs. the next. Points on the diagonal would mean the rate repeated exactly.

Does knowing the QB improve the model?

As a direct test, I retrained the logistic regression on 2024 dropbacks and tested it on 2025, once with the original pre-snap features and once with two more: the quarterback's pressure-to-sack rate and pressure rate from the previous season. Using the previous season keeps the test honest, since a team would know those numbers going into a game. QBs without enough history get the league average.

The QB features made essentially no difference: AUC moved from 0.593 to 0.595, and the Brier skill score slipped from 0.62% to 0.57%. Some of the sack tendency belongs to the quarterback, but last season's numbers are too noisy a measure of it to help a model. A better version would pool several seasons per QB and shrink small samples toward the league average.

Caveats

  • The pass-rusher count is charted after the fact. A real-time version would have to estimate it from the pre-snap look, and some rushers are disguised.
  • Team and player identity aren't features. A model that knew it was a backup tackle against a top edge rusher would do better.
  • Sack rates rose every season in the sample (5.7% in 2020 to 6.8% in 2023), so a model trained on earlier years slightly under-predicts later ones.

Data and code

Play-by-play and participation data from nflverse (participation charted by NGS through 2022 and FTN from 2023). The quarterback section uses 2023–2025. Dropbacks exclude spikes and penalties that wiped out the play. Random forest: 300 trees, min 50 samples per leaf. XGBoost: 300 rounds, depth 4, learning rate 0.05. Every number and chart comes from football-analysis/sack_prediction.py.

← Back to all posts