The Statistics that Matter and the Ones that Don't

You may or may not have realized this about me, but I watch a lot of football. And about the only thing I do as much as watch football is read about football. I read pretty much any article I come across on football, from lighthearted humor pieces to intensive analytics.

Over the course of all this reading I come across a lot of statistics. Most of them are the basic ones that are quoted on pretty much every football broadcast, passer rating and total yards and turnover margin and simple concepts like that. But I’ve also run across a lot of other numbers, ones that are becoming more commonly used in football analysis.

So I decided today to drill down into some of these lesser known stats. The basic numbers have a part to play, but they often get too much attention, missing key aspects of the game. Many of these statistics are necessary to give a more complete picture of what is happening on the field on any given Sunday. Not all of them are heavy on the math, and many are actually old concepts, just ones that are becoming better understood and more relevant to the modern NFL.

Wide Receivers

Stat to use less: Receptions

Stat to use more: First downs

At the most basic level the total number of receptions is the easiest way to measure a receiver’s productivity. After all, a receiver can’t catch the ball if he doesn’t get open, and an offense doesn’t gain anything if the receiver doesn’t make a catch. And in past eras of football this was a reasonable measure, as most passes were completed down the field where they added value to an offense.

One of the many changes in offenses over the past decade has been an explosion in short, quick passes that are often used as an extension of the running game. The target of the throw is decided before the ball is snapped, and the receiver often makes the catch behind the line of scrimmage with blockers in front of him much like a running back getting the ball in the backfield. And of course, we don’t give running backs more credit for receiving more carries, since this is a function of the playcalling rather than a reflection of anything they have done.

There aren’t readily accessible stats that track receptions based on screens versus receptions at the end of routes run. However, simply counting how many of these receptions result in a first down is an easy filter to check which receptions provide real value and which are just wastes of plays. And looking at the top ten of each category, we can see a couple names jump off the lists.

	Receiver	Receptions	1st Downs
1	Adam Thielen	67	43
2	Zach Ertz	57	31
3	Julio Jones	53	38
	Odell Beckham Jr	53	33
	Michael Thomas	53	32
6	Saquon Barkley	49	14
7	Stefon Diggs	48	20
8	DeAndre Hopkins	47	34
	Davante Adams	47	25
10	Emmanuel Sanders	46	24

	Receiver	Receptions	1st Downs
1	Adam Thielen	67	43
2	Julio Jones	53	38
3	DeAndre Hopkins	47	34
4	Odell Bekcham Jr	53	33
5	AJ Green	40	32
	Robert Woods	41	32
	Michael Thomas	53	32
8	Zach Ertz	57	31
9	Mike Evans	40	30
10	Tyreek Hill	41	27

Obviously Adam Thielen jumps out at the top of both leaderboards, but beneath that which list looks more like a ranking of the league’s best receivers? Elite playmakers like Julio Jones and DeAndre Hopkins have collected a lot of receptions, but they are even better at using those receptions to move the chains. Players like AJ Green and Tyreek Hill don’t appear on the first leaderboard but do on the second, while the first has players like Saquon Barkley and Stefon Diggs who are fed the ball but rarely do anything productive with it.

Running Back

Pay less attention to: Yards Per Carry

Pay more attention to: Success Rate

Yards per carry is the most common efficiency measure used for running backs, and it does make some amount of sense. If the number of carries is determined by the system the running back is in, we should use a number that doesn't bias towards running backs who receive more carries. The problem with this number is the same as with any average. It can be skewed up by individual big plays, and it can miss a lot of ineffective, drive killing plunges into the line of scrimmage.

Success rate is a measure developed by Football Outsiders. It is useful across a broad spectrum of analyses, but it is particularly useful measuring running backs. It is, as its name suggests, a measure of the percentage of plays that are “successful” with success defined as picking up 40% of the required yards on first down, 60% on second down, and 100% on third down.

As above, here are the leaderboards in each category.

	Runner	Y/Carry	Succ Rate
1	Matt Breida	6.5	51%
2	Kerryon Johnson	6.4	52%
3	Phillip Lindsay	5.8	47%
4	Austin Ekeler	5.8
5	Isaiah Crowell	5.7	38%
6	Melvin Gordon	5.1	49%
7	Saquon Barkley	4.9	39%
8	Kenyan Drake	4.9	47%
9	Christian McCaffrey	4.8	49%
10	Tariq Cohen	4.8

	Runner	Y/Carry	Succ Rate
1	Todd Gurley	4.8	60%
2	Kareem Hunt	4.6	58%
3	Alvin Kamara	4.6	57%
4	Sony Michel	4.4	53%
5	Marshawn Lynch	4.2	53%
6	Joe Mixon	4.6	52%
7	Kerryon Johnson	6.4	52%
8	Jordan Howard	3.5	51%
9	Royce Freeman	4.4	51%
10	Matt Breida	6.5	51%

(Austin Ekeler and Tariq Cohen fall short of the minimum number of carries used by Football Outsiders to calculate Success Rate).

Once again, the leaderboard based on Success Rate gives a more intuitive list of the league’s top backs. Gurley has been clearly the best running back in the league this year, but he doesn’t have quite enough explosive runs to make it into the top ten of yards per carry. On the other hand, running backs like Isaiah Crowell and Saquon Barkley have looked good based off of a couple of explosive plays, while actively hurting their offenses on nearly two-thirds of their other carries.

The two running backs for Chicago tell an interesting story here. We don’t have enough data to see what sort of Success Rate Cohen has (he has only 38 carries, and at this point of the season he would need 56 to qualify), but his edge in Yards per Carry over Howard has made many people say he should be getting the majority of their work. Yet Howard is still one of the most productive backs in the league in terms of getting what is needed on a given play. There is probably a better balance to be found (especially considering Howard’s ineffectiveness in the passing game), but this suggests that there is still some use in feeding their bigger, steadier back.

Pass Rushers

Pay less attention to: Sacks

Pay more attention to: QB Hits

Obviously sacks are very important. Any time you can steal a play from an offense and send them marching in the wrong direction, it is a huge turn of events. Sacks kill drives, fluster quarterbacks, and completely change games. That is why pass rushers are the second highest paid position in football, and the easiest way to get money is to rack up the sack numbers.

The thing is, not all sacks are created equal. Sacks are very rare events, and even a player who collects a sack on only 2% of passing plays will likely put up double digit sacks over the course of the season, a really good statistical year. But there are other ways that pass rushers can affect games, and often these are more sustainable and more indicative of their ability to generate pressure in the future.

There are a number of different metrics that purport to measure this, from pressures to hurries to QB hits. The first two are difficult to define and often buried in services that require a paid subscription, which is great for intense football fans but makes it a pain for more casual analysis. That’s why my favorite of these is QB Hits, which is collected by the free to access Pro Football Reference.

Here are the current leaderboards of these two stats.

	Player	Sacks	QB Hits
1	Danielle Hunter	8	11
	Aaron Donald	8	13
3	Von Miller	7.5	13
4	JJ Watt	7	12
	Myles Garrett	7	14
6	Chandler Jones	6.5	8
	Bradley Chubb	6.5	9
8	Jason Pierre-Paul	6	11
	Geno Atkins	6	13
	TJ Watt	6	11

	Player	Sacks	QB Hits
1	Fletcher Cox	4	16
2	Michael Bennett	3	14
	Myles Garrett	7	14
4	Geno Atkins	6	13
	Aaron Donald	8	13
	Dee Ford	5	13
	Von Miller	7.5	13
8	JJ Watt	7	12
9	Danielle Hunter	8	11
	Melvin Ingram	3.5	11

The first thing that jumps out from these tables is that there is a lot of overlap, much more than in the first two categories. The best pass rushers get the most sacks, and they also get the most QB Hits. It’s interesting that the top two on the QB Hits leaderboard are both Eagles who have underperformed in terms of collecting sacks, suggesting there may be some positive regression in store for both them and their defense.

It also looks like QB Hits rewards interior pass rush more than sacks. Not every interior rusher can be Aaron Donald, wrangling down quarterbacks so quickly they can’t get the ball away. It is much easier for passers to spot pressure coming up the middle than off the edge, but the pressure in their face can still disrupt a play even if it doesn’t result in a sack. Fletcher Cox, Michael Bennett, and Geno Atkins all do most of their damage coming up the middle, and QB Hits rewards them for their efforts.

Defensive Players

Pay less attention to: Tackles

Pay more attention to: I don’t know

This is a tough one, because tackles is a terrible stat but there haven’t been a lot of great alternatives put out there. The best I have seen is “stops”, roughly along the lines of tackles within a defined distance of the line of scrimmage. But it is ill defined and hard to find outside of subscription websites, so I don’t make much use of it.

But whether or not we have an alternative, we still shouldn’t be using tackles. It is a number that is nowhere near as unambiguous as you would expect it to be, often skewed by the biases of whoever is collecting the stats (isn’t it interesting how when a running back crashes into a big pile, the star linebacker always gets credit for the tackle?) It also rewards players for poor coverage, giving them credit for plays where they stay close enough to wrap up the receiver but not close enough to stop them from catching the ball.

I don’t know what a good alternative is, but honestly I think you’re probably better knowing nothing than trying to quote tackles as a measure of defensive performance.

Total Offense and Defense

Pay less attention to: Yards and Points

Pay more attention to: DVOA

DVOA is a statistic I have mentioned often in this blog. Typically when people refer to the “Number 1 Offense” or “Top Ranked Defense” they are either using points or yards per game, as those are the simplest measures to make and have been used basically since the game of football was founded. But both metrics are flawed for a variety of reasons, which is why I prefer the more recent development of Football Outsiders DVOA.

The biggest problem with yards per game is how dependent it is on pace. Not every team faces the same number of plays defensively in every game, and you can end up with wildly different results. Arizona and New England are both allowing roughly 385 yards per game, but Arizona faces about five more plays per game on average. It doesn’t sound like a lot, but if you look on a per play basis the Cardinals rank eighth in the league, while the Patriots are all the way down at 23.

Points per game is even more flawed. Field position is such a crucial factor in scoring, and nothing provides excellent field position like turnovers. An offense with an aggressive, turnover producing defense is going to score points that they don’t deserve credit for, and a defense with an offense that protects the football is going to do a better job keeping the opponent off the board. The Patriots have alternated between mediocre and just straight up bad on defense for the past ten years, but they have consistently been near the top of the league in points allowed thanks to an offense that rarely turns the ball over.

DVOA is an attempt at a solution for this. Standing for “Defense-adjusted Value Over Average”, it evaluates teams on a play by play basis looking at success rate and then adjusting for the level of competition. It is calculated for both offenses and defenses, as well as the team as a whole, and it is the best single valuation of a team’s performance over the course of a season. (I’m hesitant to include any numbers from it now, as I’m not sure I put much stock in its various moving pieces after only 7 weeks. But by the end of the season I think it gives a good idea of who the best teams and best units are.)

Quarterbacks

Finally we get to the most important and thus most scrutinized position in football. There have been many attempts over the years to produce a single metric to evaluate quarterback play, and they have met with varying levels of success and failure. There are currently three commonly used numbers out there, all of which I’ve touched on at points. They each have their advantages and disadvantages (though I have a clear favorite, as you will see below), and they each tell a different part of a large, very complex story.

Passer Rating

The oldest and most commonly used of the metrics, passer rating makes up what it lacks in statistical rigor with familiarity that allows easy communication. By now every football fan has heard enough about passer rating that they know off the top of their head what counts as a good or a bad game, and for quick, rough analyses this often gives it the edge.

Passer rating was developed by the NFL in the 1970s as a way to rank quarterbacks. It takes into account five basic variables: completions, attempts, yards, touchdowns, and interceptions. These numbers are weighted using coefficients designed to give a number that mimics traditional school grading systems—above a 90 is good, something in the 80s is acceptable, 70s is passing, and anything below 60 is failure.

This is where things get a little dicey. Because passer rating was calibrated to these targets more than 40 years ago, the weights given to the variables don’t really mean much today. Quarterbacks today complete a higher percentage of passes and throw significantly fewer interceptions. This has caused a massive inflation in passer rating numbers, to where the league average now exceeds 90. And in this way the familiarity actually works against it, as people see a quarterback rating in the high 80s and do not realize that this actually signifies a bad game.

QBR

Often confused with passer rating, QBR is a completely separate statistic that was engineered by ESPN within the past decade. It is a proprietary formula, meaning we don’t know how exactly it is calculated, but in terms of what it takes into account it is the most statistically rigorous number. It is calculated based on an Expected Points Added model that takes into account everything a quarterback does—touchdowns, interceptions, throwaways, sacks, rushing yards. It adjusts for dropped passes and strength of competition. In short, it takes into account everything you would want to take into account.

There are reasons to like QBR. It has shown a stronger correlation to actually winning games than the other measures, as well as a better serial correlation against itself, suggesting it is a truer measure of a quarterback’s skill than the others. And on purely anecdotal terms, I’ve found that the numbers given by this metric better align with the impressions I get from watching quarterbacks play on the field.

Even with these advantages though, I still don’t use QBR very often. The biggest issue is its lack of transparency. Because the formula is proprietary, any use of it has to draw the numbers directly from ESPN, limiting the flexibility for analysis. There is no easy way to grab just plays that happen on third down, or even average across the first half of the season. It’s a useful tool for ESPN publishers and broadcasters, but it doesn’t help outside analysts. It also only is available back to 2006, making real historical analysis basically impossible.

ANY/A

If you’ve read this blog at all before, you know what I’m going to say here. Adjusted Net Yards per Attempt is my favorite single number for evaluating quarterback efficiency. As its name suggests, it is built around the yards per attempt number, which is then adjusted to account for other factors. The first of these is sacks, subtracting from the total yards and adding to the attempts. It then adds 20 total yards for each touchdown and subtracts 45 for each interception, based on analysis done on the average expected points value of each yard compared to a touchdown and a turnover.

This formula excels because of its simplicity. The 20 and 45 values are a bit strange, but they are more intuitive than the weird system of coefficients and max/min functions used by passer rating. Anyone can calculate ANY/A with a basic spreadsheet, and can choose to include and exclude plays to their heart’s content. It is versatile in a way QBR isn’t, making up for some of the information it lacks.

That information is something we should still consider however. This formula penalizes a quarterback for drops, throwaways, and spikes, and it offers no rewards for rushing yards. It doesn’t differentiate between yards through the air and yards after the catch, and much like rushing yards per carry it can be skewed up by big plays (though this is less serious, since the distribution of passing yards isn’t as skewed as rushing).

New tools are brought out every year, and it is likely that someday soon someone will attempt to build a number with all these advantages and none of the disadvantages. Until then we are stuck using the tools we have, but that doesn’t mean we shouldn’t at least be smart about what tools we pick to use.