Offense and Defense

A quick note before I begin. This is going to get a bit math heavy as I go on. I will be using the statistic known as DVOA in order to measure a team’s offense and defense. DVOA is the favored metric among the football statistical community because it is the most comprehensive means of measuring an offense or a defense. The details are a bit complex, but it essentially judges a team on a play by play basis then adjusts for strength of schedule and league average. Each team is measured based on percentage points varied from the average. On offense you want a positive value, and on defense you want a negative value.

We are through seven weeks of the NFL season, and we are starting to get an idea of what we’re dealing with from each team. Some things have come as expected, and some things have caught us by surprise. Below I’ve listed the top eight and bottom eight in both offensive and defensive DVOA.

OFFENSE				DEFENSE
Top 8		Bottom 8		Top 8		Bottom 8
Broncos	31.4	Cardinals	-11.2	Lions	-24.9	Browns	5.4
Packers	19.9	Lions	-11.3	Broncos	-18.2	Steelers	6.7
Cowboys	16.5	Raiders	-16.0	Dolphins	-11.1	Raiders	7.2
Ravens	16.2	Jets	-16.2	Bills	-10.5	Rams	12.2
Chargers	15.7	Bills	-16.5	Cardinals	-10.0	Panthers	14.0
Seahawks	15.5	Buccaneers	-25.8	Ravens	-8.8	Buccaneers	15.0
Saints	11.8	Jaguars	-28.7	Seahawks	-8.1	Falcons	16.0
Colts	8.7	Vikings	-29.8	Eagles	-6.3	Saints	17.7

Do a quick eye test of the listed teams and compare them to whatever preseason expectations you had. On the offensive side, pretty much everything lines up the way you’d expect it. It’s a small surprise to see the Ravens ranked so high, and a bit shocking to see the Lions so low, but every other team is right about where you would think they belong. The Broncos, Packers, and Saints are good on offense, and the Raiders, Bills, and Jaguars are not.

Then look at the other side of the equation. Immediately you see the Lions sitting as the top rated defense, an outcome that would have been laughable at the beginning of the season. Look down the list and you see other units—the Eagles jump out—who would not have been out of place in the bottom eight coming into the year. A similar phenomenon is seen on the other side, where highly touted units like the Browns, Panthers, and Buccaneers languish among the worst in the league.

(It has not slipped my attention that the bottom four teams in defensive DVOA are all in the NFC South, which has probably displaced the AFC South as the worst division in football. Congratulations to the Titans, Jaguars, Texans, and Colts. Mostly the Colts, who are the only thing saving the division from the abyss.)

At first glimpse it looks as if the defensive side of the ball is more volatile from year to year, and possibly the easiest way for a team to improve. This raises a number of intriguing questions if true, but first we’re going to need more data.

I collected the DVOA statistics from the past eleven years in an attempt to figure out if this is a one year pattern or if it there is something more to it. I performed a simple linear regression of each season compared to the previous year in order to see which had the most consistency from one season to the next. Below are the graphs with the best fit lines plotted atop the data.

 

I didn’t use identical scales on each graph, so it is a bit tricky to make sense of it just by looking at the distribution. But the equations of the regression line in the top right tell a great deal on their own. For those without much statistical background, the R-squared values are a measure of how effectively the line fits the data. A number close to one means a strong correlation, while a number close to zero means very little. These correlations are not strong, but the correlation between year to year offensive DVOA is noticeably higher than the correlation between year to year defensive DVOA.

This raises a number of questions. First and foremost is: why? Is there something about the defensive side of the ball that leads to more inconsistency? Is there perhaps more year to year turnover on defense than there is on offense? Or are injuries more likely on that side of the ball?

I searched for a while, but I wasn’t able to find any sort of convenient database to measure roster turnover. The only option I could think of would be to go through and compare year to year starting units for each and every team, something I don’t have the time or the patience for. Injuries are similarly difficult to track, especially comparing offense to defense. The best I found was this graph put together by Football Outsiders using their statistic “Adjusted Games Lost” to measure the top and bottom eight in the league in injuries on offense and defense in 2013.

Top 8				Bottom 8
Offense	AGL	Defense	AGL	Offense	AGL	Defense	AGL
CHI	6.9	TB	9.9	NYG	82.1	NO	74.4
CIN	12.1	KC	11.2	TB	76.6	NYG	62.5
NO	14.2	PHI	12.2	IND	76.4	CHI	55.7
WAS	16.3	BAL	14.7	GB	59.3	DAL	51.4
KC	16.4	TEN	15.9	PIT	55.8	NE	49.8
DAL	16.6	SEA	17.3	ATL	54.0	DEN	47.5
BUF	18.5	CLE	18.4	OAK	50.7	SF	47.3
PHI	21.4	MIA	19.4	NE	50.1	SD	46.3

Obviously, this isn’t close to a reasonable sample size. But it does suggest that injuries are actually more frequent on the offensive side of the ball. The results are far from conclusive, but they certainly don’t support the idea that defenses are more vulnerable to variation due to a higher rate of injuries.

The one obvious difference between offense and defense is the quarterback. There is no player on the defensive side of the ball that wields the same influence over his unit’s performance as the quarterback does on offense. At first glimpse this would seem to suggest that offense would be more volatile. Individual players are subject to ups and downs during games and seasons, and normally you would expect a unit of eleven equally weighted players to be able to handle that more than a unit highly dependent on a single individual.

There is also a rationalization that can be made for the quarterback as a stabilizing force. If a quarterback’s performance is less erratic than an average player’s, he can absorb a great deal of the volatility of his teammates. Since quarterback is one of the positions least likely to suffer an injury or see turnover during the offseason, it stands to reason that an offense will largely stay the same from year to year provided their quarterback is consistent.

Another possibility, one I lean towards, is that this is simply the nature of a game in which offense dictates almost everything. On each play the offense knows exactly what they need to do before the snap, while the defense is forced to react after the play begins. By allowing the offense to dictate the flow of each play, the defense leaves itself vulnerable to more factors outside its control. Over the long run, this could manifest as the sort of inconsistency we see above.

The one final thing we need to consider is the metric being used. DVOA is the height of current football analytics, but it is still a single number trying to capture the entirety of a team’s performance. Other sports have plenty of experience with how difficult it is to measure defensive production, and it wouldn’t be a surprise if the NFL has the same troubles. The uncertainty in the numbers could be caused by the metric itself. I’m less inclined to believe that, if only because the finding that defense is less consistent than offense passes the eye test as well as the statistics test.

The next question raised by this discovery is what it can tell us about trying to forecast a team’s future. The simplest answer is that it is safer to pick a team based on the strength of their offense than the strength of their defense. Teams with an above average offense and a below average defense are more likely to remain the same on offense and regress to the mean on defense than vice versa.

I’ll give a couple examples from this year to back this up. Last year San Diego snuck into the playoffs as the 3^rd ranked offense and 32^nd ranked defense. I picked them to miss the playoffs because I expected their offense to fall back to earth and I saw nothing to expect their defense would improve. But so far this year they are 5^th offensively and 23^rd on defense. The gain in defense was larger than the loss on offense, and they are well on their way to earning a playoff spot.

I was wrong about San Diego, and I was even more wrong about New Orleans. I picked the Saints to reach the Super Bowl because I saw a team that finished last year ranked in the top ten in both offense and defense. I expected their defense to continue playing at the same level, and possibly even improve, but instead they plummeted to last in the league.

There were eight teams last year with an offense ten ranks or more ahead of their defense. Those teams—Denver, New England, San Diego, Philadelphia, Chicago, Green Bay, Atlanta, and Dallas—are a combined 37-19. Reverse that criteria and you find eight teams—Cincinnati, Arizona, St Louis, Buffalo, Tampa Bay, Baltimore, Houston, and the Giants—with a combined record of 26-25-1.

All this evidence seems to point in the same direction. Offense is the more trustworthy quantity in the NFL, and it is what you should build around if you want to find long term success. But let’s say you are an NFL coach of a bad team, and the fans of your team are currently screaming for your head. The owner has made it very clear that your job depends on making the playoffs this season. If it is significantly easier to improve on defense than it is on offense, wouldn’t you be safer to build on that side of the ball?

To answer that question, I plotted each team’s annual change in wins against its change in offensive and defensive DVOA. The plots are shown below.

 Look at those R-squared values. Not only is offense more sustainable in the long run, it has a much stronger correlation with a team’s improvement. Even if your team makes a massive jump on the defensive side of the ball, it likely will not result in as many wins as if they had made a similar jump on offense.

Improving on offense is better for a team in the long term, and it is better for a team in the short term. So there we have it. Hours of data collection and manipulation, along with nearly 2000 words, all to reach the conclusion that has been obvious all along.

In the NFL, offense wins.