QB Performance Over Time: How Would Dan Marino or Joe Montana Fare in Today’s NFL?
Note: this piece originally ran on Football Outsiders in March 2022 (former link: https://www.footballoutsiders.com/stat-analysis/2022/how-would-marino-montana-have-fared-2021)
ntroduction
Most of my prior projects on this website have been dedicated to in-game strategies that can actually help teams win. This one is not. Instead, it’s meant to help settle an ongoing debate within the football community over the past decade. It’s no secret that the NFL is in a “Golden Age” of passing, with passing becoming both more common and more efficient in recent years. Of the 17 individuals to have a 70+ completion percentage in a season (min. 200 attempts) in NFL history, 14 have come since 2009. 13 of the 14 all-time 5,000-passing yard seasons have come since 2008, with Dan Marino’s legendary 1984 campaign being the lone exception. The top eight QBs in NFL history in career passer rating (min. 1,500 attempts) were all active as recently as 2020, though Drew Brees has since retired.
It’s a given that NFL QBs are succeeding like never before, but what’s not universally agreed upon is why it’s happening. Is it primarily because it’s never been easier to be an NFL QB, due to offense-friendly rule changes, more innovative play design, and better athletes lining up out wide, as has been argued by outlets like The Ringer, Sports Illustrated, Yahoo, and The Ringer again? Or has some old-fashioned “Darwinism” led to this generation’s crop of passers just being that much better than those that came in the 20th century, as has been pitched by outlets including The Herald Bulletin and Fansided? We’ll never know the true answers to those questions without having human time travel, but with Pro Football Reference’s data and R programming, we can piece together the next-best solution.
Methodology
To take this topic on quantitatively, I’ll use a modified version of a concept I first discovered in Wayne L. Winston’s 2009 book called “Mathletics”.
In that book, the writers were attempting to estimate what Ted Williams’ batting average would be in the 2006 MLB, if he were to be in his 1941 form (when he became the last player to hit .400 in a season). They took the combined batting averages of all players who played in 1941 who were not in their last seasons, and then compared those averages to the same players’ collective averages in 1942 (i.e. no 1942 rookies), in order to see how much the league’s pitching and defense – the factors outside the hitters’ control – improved from 1941 to 1942. Then, they took all players who played in 1942 who were not in their last seasons, and compared that group’s batting averages to the same group’s stats in 1943 (i.e. no 1943 rookies). This process was repeated for every pair of seasons up to 2005 and 2006, to see how much better the MLB’s pitching/defense had become year by year, since the 1940s, without being skewed by the new batters entering the league each year. The core principle of Winston’s methodology was that, “since young players tend to improve with experience and older players tend to decline in their later years, it’s a fair assumption that this group’s batting abilities won’t drastically change from year to year.”
This assumption provides the starting point for my project. I use a similar concept with five common NFL passing “rate stats”: completion percentage, touchdown percentage, interception percentage, yards per pass attempt, and yards per completion. As was the case with the “Mathletics” Ted Williams study, the goal is to see how much all of the “external factors” that aren’t directly related to a QB’s individual skill have evolved over time, and how much those factors have influenced passing stats. These factors include, but aren’t limited to:
More innovative offensive play design (in particular, play-action becoming more common)
The inception of more offense-friendly rules
Better athletes playing around the QB on offense (particularly at receiving positions)
Better athletes playing against the QB on defense
More complex defensive coverage schemes
Advances in sports medicine and nutrition
Reductions in the legal number of full-contact practices per NFL team per season
However, my methodology has two key modifications to the “Mathletics” format. One of them is the institution of a passing attempts minimum. When comparing any two seasons, the method of excluding all retirees in the first season and all rookies in the second season conceptually makes sense, so that the group of QBs is (theoretically) unchanged between each season. But in practice, there are numerous instances like Patrick Mahomes, who only played one game in his rookie year of 2017 but played all 16 games in 2018. In this example, the QB group wasn’t really the exact same in 2017 as it was in 2018, because even though Mahomes was a part of both, he had 35 pass attempts in the prior year and 580 in the latter. As a result, any jump in league-wide stats from 2017 to 2018 was artificially inflated by Mahomes (and Deshaun Watson, for that matter) playing far more in 2018 than in 2017. This phenomenon can work in the opposite direction too; e.g., Tom Brady only had 11 pass attempts in 2008 due to an ACL injury after having a then-record 50 pass TD in his historic 2007 season. To counter this, my data collection for this project involved looking at Pro Football Reference’s statistics for two seasons at a time, and only extracting data for QBs who hit 150+ attempts in both seasons (regular season only). While still not flawless (e.g., a QB could’ve had 153 attempts in one season and then 603 in the next one), it allows us to avoid the egregious examples like 2018 Mahomes and 2008 Brady. I compiled all passing statistics for QBs, using this format, for every pair of seasons since 1960 and 1961. I started at this point because 1960 was the first season of the AFL (American Football League, i.e. what the current AFC was called before the 1970 AFL-NFL merger), significantly increasing our year-to-year sample sizes compared to the pre-AFL era.
The other modification is that, in the Ted Williams project, the writers attributed all of a season’s statistical changes to external factors. I.e., if players not in their final season in 1953 had a collective batting average of. .270, and players who weren’t rookies in 1954 had a collective average of .260, that means that pitching and defense improved by an estimated .010 points in that offseason. But playing QB is different than hitting in the sense that there’s much more decision-making involved. While there are some choices that come at the plate, the mental side of playing QB involves far more complex decision-making on a play-to-play basis, with audibles, pre-snap coverage reads, and post-snap improvisations all playing major roles in where (if at all) a QB throws the ball. Because of this, it isn’t fair to exactly copy the “Mathletics” assumption regarding a group of players collectively not improving or regressing noticeably from one year to the next. While the purely physical QB traits like arm strength and accuracy likely follow that ideology, an extra year in the league can significantly influence a QB’s mental capacity. Because of this, I modified Winston’s formula so that half of the league’s year-to-year statistical changes are attributed to the “external factors” like play-calling, rules, and defense, while the other half are attributed to the QBs’ own change in performance.
Since this all sounds confusing, here’s a specific example of how it works, using completion percentage. Let’s start with the 1960 and 1961 seasons as our baseline. There were 16 QBs who had 150+ pass attempts in each of those seasons. In 1960, those QBs had a collective completion percentage of 50.75. Those exact same QBs then had a collective completion percentage of 51.10 in 1961. This means that those QBs had an estimated collective increase of 0.175% via their own performances, but also that due to the external factors discussed above, it became an estimated 0.175 percentage points easier to complete passes between 1960 and 1961, thus combining to make the +0.35 gap. To phrase the latter part about external factors more succinctly, we would say EXT_CompPct_1960to1961 = +0.175. Repeating this process for 1961 and 1962, we find that EXT_CompPct_1961to1962 = +0.895. Because our eventual goal is to find EXT_CompPct_1960to2021, we simply have to repeat this process over and over again: EXT_CompPct_1960to2021 = EXT_CompPct_1960to1961 + EXT_CompPct_1961to1962 + EXT_CompPct_1962to1963 … + EXT_CompPct_2020to2021. As you’ll see below, it turns out that this number is 13.2, meaning that it was approximately 13 percentage points easier to complete a pass in 2021 than in 1960 by our logic.
Data
While we saw a taste of our data in the Methodology section, below is the real final data, with explanations to follow. This data only shows results by decade rather than by individual seasons to save space, but I can share the year-by-year data to anyone interested.
These numbers show us that it has overall become easier to have success passing the ball over the past 60+ years, but that doesn’t mean that each individual statistic has necessarily seen a reduction in difficulty. By our project’s logic, it’s actually become slightly more difficult to throw touchdowns in today’s league, and passing yards per attempt has been relatively stable, with a slight increase in difficulty. In contrast, completion percentage and INT percentage have been consistently and steeply trending in QBs’ favor throughout the past six decades. If we look at our far right column about yards per completion, we get a good explanation why. Yards per completion has collectively taken an extremely sharp decline in the NFL since 1960, which leads us to one major conclusion: QB statistics in the NFL are better today largely because of a systemic league-wide embracing of shorter, less dangerous passes in recent years. Completion percentages are higher and INT rates are lower because the league has shifted toward shorter passes, whereas TD pct and yards per attempt haven’t seen the same upward vault because they largely depend on increased downfield aggressiveness from QBs. (Notably, there hasn’t been a qualified passer to have 10+ yards/attempt in a season since Norm Van Brocklin in 1954).
With that being said, while progress has been noticeable in several QB metrics since 1960, that progress hasn’t been linear. In order to take a deeper look into how the difficulties of being an NFL QB have changed year-to-year, we can use R graphs. Again, we can start with completion percentage to explain our methodology. The following graph details how much easier it has been to complete passes based on external factors, going one year at a time, to allow us to see which specific seasons took jumps or declines from the immediately preceding one. In other words, the bar you see on the far right (just over 0.0 pct) represents EXT_CompPct_2020to2021, while the vertical red lines mark the start of each decade (excluding 2020):
In contrast, the ensuing graph (and the one I find more informative) details how much easier it has been to complete passes in the NFL, but on a cumulative basis since 1960 rather than looking at one year at a time. So, the bar on the far right here, at just over 13 percent, represents EXT_CompPct_1960to2021, rather than EXT_CompPct_2020to2021:
One interesting phenomenon that becomes evident from these graphs is that the whole is greater than the sum of its parts; in other words, while any given year-to-year jump might be minimal (and in fact, many of them are negative), we still have a drastic upward trend when we look at a longer span of time. I ran some two-sample T-tests to further verify this point, which I cut here for space but can share to anyone interested. Before getting into further observations, I’ll throw in the same style of graph as the immediately preceding one – i.e., going cumulatively since 1960, instead of year-by-year – for the other statistics we’ve measured.
Yards per pass attempt:
TD percentage:
INT percentage:
Yards per completion:
You can scan the graphs to find your own observations, but these stood out to me:
In addition to being the first-ever 16-game season, 1978 is known for being when a year where two significantly pro-offense rule changes were enacted: contact between defensive backs and receivers was restricted more than five yards downfield, and offensive linemen were given more freedom to use their hands on pass plays without being called for holding. As expected, these made a noticeably positive impact on QB stats. Among 20 QBs to have 150+ attempts in both 1977 and 1978, they collectively improved in completion percentage, yards per attempt, and INT percentage in 1978, with a particularly large +2.37% jump in completion pct. To clarify, those are the full statistical increases between the 1977 and 1978 QBs, not the versions that are cut in half to account for just the impact made by our external factors.
In 2004, the NFL further cracked down on restricting contact by DBs more than five yards beyond the line of scrimmage, and that led to an absolute boom in passing stats between 2003 and 2004, including the largest year-to-year jumps in yards per attempt (+0.449) and TD pct (+0.643) for any pair of seasons since 1960, excluding the strike-shortened season of 1982. This season is particularly known for Peyton Manning’s historic output, as he set then-NFL records in passing TD (49) and passer rating (121.1). Broadly, most QB stats, particularly completion pct and INT pct, have seen noticeable improvements over the past 20 seasons.
2011 was known as the “Year of the Passing Game”, still standing as the only season all-time to have three 5,000-yard passers (Brees, Brady, Stafford) – none of whom even won NFL MVP, as that honor went to Aaron Rodgers and his still-standing record 122.5 passer rating. Unsurprisingly, this came after a rule change as well, as 2011 was when the NFL changed rules to further define and protect “defenseless receivers”. While TD percentage and INT percentage weren’t actually positively impacted between 2010 and 2011, the jump of +0.211 yards/attempt between the seasons is the 5th-largest since 1960.
2018 was the year of Patrick Mahomes torching everything in his path in his first year as a full-time starter, but he wasn’t the only reason that QB numbers spiked that season. Rule changes in 2018 included a penalty for any defender who lowers his helmet before contact, and the infamous “body weight” rule regarding defenders tackling QBs. Subsequently, the group of QBs with 150+ attempts in both 2017 and 2018 collectively improved in the latter season in completion percentage, yards per attempt, INT percentage, and TD percentage. The jump of +3.16% in comp pct is the largest between any pair of seasons since 1960, while the jump in yards per attempt (+0.275) ranks 3rd in that span.
An obvious trend in these observations is that rule changes have directly led to improved QB stats. The NFL’s Competition Committee even said the following in 2012: “If someone wants to accuse the National Football League of promoting offense to make the game more exciting, [the committee] believes the league should plead guilty.” While other factors, particularly the evolution of play-calling, have likely also played a role in the offensive explosion, our data suggests that the NFL’s rule changes have had the primary (and desired) effect of getting more points on the board.
Real-World Examples of Player Projections
With all of this data in our back pocket, now we can answer the titular question: just how good would the top QBs of the past be in today’s game? To start off in this task, here are the real statistics from five Hall of Fame QBs in their MVP seasons:
Now, here are the projected adjustments, based on external factors, regarding how much easier it got in each statistical category between those MVP seasons in question, and 2021:
And, finally, here are what those old-time QBs’ projected stats would be in 2021, if we adjusted their MVP numbers to account for the external factors that have changed since those seasons. For the sake of comparison, we also include the real-life numbers for several big-name QBs in 2021:
We must view these projections for the old-timers with a grain of salt, because the real-life seasons we based those projections on, by virtue of being MVP years, are inherently major outliers. In other words, the takeaway shouldn’t be that Marino, Montana, and Young are all actually far better football players than Patrick Mahomes; rather, it should be that the absolute best seasons of each of those guys’ careers compare favorably to what will surely end up being considered a below-average season in Mahomes’ career. This phenomenon is largely my own fault for choosing to use some of the greatest seasons of all-time to analyze here, but I figured the average reader cares more about Marino and Montana than about Steve Spurrier or Doug Pederson.
With that disclaimer being said (and more to come in the bottom section on Sources of Error), these projected numbers are still largely impressive. 1984 Dan Marino’s modern-day projections would set the all-time passing yards record and tie Peyton Manning’s pass TD record, which makes sense given the discourse about his 1984 season. 1989 Montana and 1994 Young both were projected to set single-season records in completion percentage and beat all 2021 starters in passer rating, which isn’t too surprising given that they were the only qualified passers to have a 110+ passer rating in the entire 20th century. However, as most fans would expect, the “average” QB of 2021 would be closer to the level of players like Montana and Young than their contemporaries were. For example, Steve Young’s “Passer Rating Index” (a measurement of how one’s passer rating compares to the rest of the league of that season, where 100 is standardized as league average, similar to OPS+ in baseball) in 1994 was a staggering 147. But his projected Passer Rating Index in 2021 would’ve been approximately 131, even though his actual projected passer rating would’ve been higher in 2021 than in 1994. These projections lead us to a reasonable conclusion: while the collective QBs of today’s game are still better than they have ever been, the vastly inflated statistics we see today are largely due to the external factors like rule changes and play-calling. It isn’t fair to call the “it’s never been easier to be a QB” ideology correct and the “QBs are just better nowadays” ideology incorrect, since both sides are right to an extent, but I believe the former trait is what ultimately has been more impactful on modern-day QB statistics.
Possible Sources of Error/Other Comments on Methodology
While any football analytics project has some built-in risk of error, this one has particularly major caveats given that it attempts to deal with time travel. The biggest one is that the attribution of half of the changes in QBs’ stats to their own skill changing, and half to the external factors around the league, was a somewhat arbitrary allocation. While just about every other trait of this project was determined with a data-driven process, as shown by the barrage of R graphs and tables, the decision to go with a 50/50 split here was an intuitive call based on my beliefs on how QBs benefit by having extra experience in the league. While that intuition fortunately led to reasonable projections (e.g., we weren’t told that Johnny Unitas would have an 87% completion rate in today’s league), there’s still no way to ensure that it was the correct distribution. If anything, there probably was no single uniform correct distribution; i.e., perhaps attributing half of the changes to external factors was too low for a year with drastic rule changes like 1978, but too high in other seasons.
The other primary flaw, and one that is common across football analytics projects, is that every team in a certain year is treated the exact same. We say that it’s roughly 13 percentage points easier to complete a pass in 2021 than in 1960, but in practice, that would obviously depend on which team a QB was playing for. Those external factors like play-calling and skill position talent don’t just vary from year to year; they also vary from team to team within a specific year, which this project doesn’t account for. This distinction is particularly important when discussing guys like Joe Montana and Steve Young, since lining up next to prime Jerry Rice is an ideal asset for any QB. While we estimated it was roughly 6% easier to complete passes between in 2021 than 1994 for average teams, that gap wouldn’t likely be as big if we compared the average 2021 team to the 1994 49ers specifically. As such, the modern projections for those two QBs, and the other old-timers as well, were likely a bit inflated. By virtue of winning MVPs in those seasons, they inherently were probably in situations where their team’s coaching and supporting cast made it easier to succeed than it would have been on another roster. Unfortunately, this was another systematic flaw we had to accept, since sample sizes would have been trivial if looking at one team at a time. These are the primary sources of concern, though there are more minor ones as well (e.g., the flaws of passer rating as a stat have been well-documented for years, but you can’t exactly use DVOA back to the mid-1900s.)
Pertaining to methodology, some readers may be wondering how I estimated the raw number of attempts for old-time QBs in their theoretical 2021 statistics. The way I did it was a "scale" based on each individual season's leader in pass attempts. To elaborate, Joe Montana had 386 pass attempts in 1989, and the 1989 leader in pass attempts was Don Majkowski at 599, meaning Montana had 64.4 pct of the leader's attempts. The 2021 leader in attempts was Tom Brady, at 719. So, using the same scale, the theoretical 2021 Joe Montana would have (.644 * 719) attempts, or 463. Of course, it's not a perfect assumption to make (largely because we can't assume each old-time QB's theoretical 2021 team would have the same play-calling tendencies as his real-life team), but it's a solid proxy in the sense that it will give numbers that are at least feasible for the 2021 season.
Thanks for the read, and I’m happy to hear any feedback or further questions about methodology.
Cole Jacobson is a Next Gen Stats Researcher at the NFL Media office in Los Angeles. He played varsity sprint football as a defensive lineman at the University of Pennsylvania, where he was a 2019 graduate as a mathematics major and statistics minor. With any questions, comments, or ideas, he can be contacted via email at jacole@alumni.upenn.edu and @ColeJacobson32 on Twitter.