Applying Sean Payton's Algorithm to Every College Football QB Season Since 2015
Plus, the mystery of the hidden interception?
Hello everyone,
This is my last write-up from rural Italy as I’ll be back stateside by the time you read this. I’ve enjoyed this vacation and it’s allowed me to recharge before a busy fall that’ll be full of in-season college football and NFL content as well as another grad school class. But the ball is rolling here at The Spade and we can’t stop it any time soon.
Today we’re talking about the most advanced algorithm I’ve ever seen and we’re going to try and reverse engineer it. I’m referencing Sean Payton’s algorithm cited here in this ESPN article from last week. It gives a glimpse into how one of the most innovative offensive NFL minds of the last 20 years thinks about college football quarterback scouting. I found Seth Wickersham’s article, which is an excerpt from the book American Kings: A Biography of the Quarterback releasing on September 9, to be a really great read, specifically this portion here:
So what we have laid out here are the blueprints to Sean Payton’s top secret proprietary quarterback algorithm. Take the number of ‘bad plays’ and divide it by the total number of pass attempts, and multiply by 100. That gives you a cumulative score that I have yet to create a name for, but we’ll call it the SP score for the sake of this write-up.
So does Payton follow his own process? Let’s find out by creating an SP score for each quarterback from the past decade of college football using the CFBD data API.
Calculated SP Scores
I can provide a tutorial later down the line if you’d like, but I’m not running anything crazy here. I’m using CFBD data to sum up the number of pass attempts to run the formula I’ve described above. Here’s how every quarterback ranked in SP score from 2015 to 2025, with every year’s top 10:
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
Some thoughts in general on the Sean Payton score:
I was shocked to not see Bo Nix in the top 10 for the 2023 season
This should be renamed to the Lincoln Riley score, as his QBs have dominated the ranks.
Over the last 10 seasons, apparently Zach Wilson is the all time single season Sean Payton score leader? Maybe that provides some insight into his NFL draft hype.
I’d love to hear what you think about these rankings in the comments below!
The Mystery of Cam Ward’s Phantom Interception
Here’s an overall existential tangent on data integrity itself, and why you should always take data visualizations you see online with a grain of salt and not as the truth.
Flash back to week 4 of last year’s college football season, to Miami’s 50-15 win against South Florida. It’s the first quarter and Cam Ward drops back and fires a pass to Jacolby George, then this ensues:
George catches it, then coughs it up into the air, and the ball is intercepted. Off screen, this play is whistled dead by a personal foul, and USF receives possession.
Here’s how ESPN logged that drive:
The play text describes it as “Cam Ward pass incomplete” but it’s describing the play shown above. So when pulling this data in from CFBD which uses the ESPN API to gather their play by play data, we also only see ‘Cam Ward pass incomplete'. No reference to the interception that occurred, whether you think it was Ward’s fault this interception occurred or not.
As a data engineer, events like this are borderline impossible to parse unless you delicately comb through each and every play that occurred. This was found due to a discrepancy error with my code and the number of interceptions it tallied for Cam Ward vs. the number seen on ESPN. ESPN did indeed count this as an interception thrown by Cam Ward, but just in the box score and not in their underlying play text API description. College Football Reference also counted this interception, but they don’t have any underlying play descriptions I can reference to see how they brought this over.
So when seeing visualizations about football online, always include some personal, mental degree of error to each graph. Sometimes it’s from things outside of the author’s control, and sometimes it’s from things outside the quarterback’s control.