Toss Out the Predictions

We’re four weeks into the NFL season, and everything is more or less going according to plan. The Patriots and Chiefs have coasted to undefeated records. The NFC North is a bloodbath of elite defenses and shaky quarterback play. The Broncos, Redskins, Bengals, Cardinals, and Dolphins all have yet to win a game.

Oh, and also the Eagles, Falcons, Chargers, and Browns are all .500 or worse, the Lions and Bills have only lost the Chiefs and Patriots respectively, and the 49ers haven't lost at all. So maybe not quite everything has gone according to plan.

We enter the season each year with certain expectations, and it never takes long for those expectations to fall to pieces. Of course, four weeks in is still probably too early to say for sure that we should cast aside everything we thought heading into the year. Four weeks into last year the Colts, Cowboys, and Chargers were disappointments, while the Bengals and Dolphins were pleasant surprises. Things can change quickly in this league, and they can just as quickly change back.

So at what point in the year do we start trusting what our eyes are seeing? At what point do we cast aside everything we thought we knew coming into the year, and at what point do we believe what has unfolded on the field?

There are a couple different ways to look at preseason expectations. The simplest is to just look at how the team did a year ago. After all, NFL rosters don’t turn over that much from year to year, and generally good teams stay good and bad teams stay bad. This is obviously a huge oversimplification, but there is some value to it. Year over year, a team’s final record correlates 32% to their record from the season before. (All calculations in this go back to the 2000 season.)

Of course we know a lot of stuff does happen in the offseason, and there is an easy metric available to see what the expectations were before the season, factoring in both a team’s success from the previous year and changes they made during the offseason. I was able to pull the preseason Vegas win over-under values for each team going back to 2000, and I compared them with the records these teams realized at the end of the year. The correlation was 40%, better than just by looking at the previous year’s record but not dramatically so.

So if we can get correlations with the final season record of 30-40% before the year begins, at what point can we do better just by looking at the results from within the season? Knowing now that the 
Bills are 3-1, is that more or less meaningful than knowing they were predicted to win 7 games at the beginning of the year?

The table below shows the correlations achieved by comparing each team’s record at the end of each week with their record at the end of the year (again for a sample from 2000-2018).

Week	Record to this Point
1	34%
2	45%
3	57%
4	62%
5	68%
6	74%
7	77%
8	81%
9	85%
10	89%
11	91%
12	93%
13	94%
14	96%
15	97%
16	99%
17	100%

As you can plainly see, it doesn’t take long for the records within the season to eclipse the predictiveness of the preseason expectations. The outcome of the very first week is more meaningful than the record from all 16 games of the previous year, and by the second week the correlation is already higher than using the preseason win-loss totals.

Of course, there’s a pretty clear flaw in this analysis. The games a team has won through the first three weeks count towards the season end total. The 49ers are 3-0, which means there is no chance of them finishing 2-14. It means that to finish with the 8-8 record that Vegas predicted before the season they would have to play significantly below .500 from this point on.

So that raises the next question: how effective are these varying metrics at predicting the performance of a team for the rest of the season? Does the fact that a team like Buffalo has started off hotter than expected mean that they will carry this for the rest of the year, or is it more meaningful that they were expected to be a below average team coming into the season?

The table below has three columns. The first shows how a team’s winning percentage at the end of each week correlates with its winning percentage over the remainder of the season. The second column shows how the winning percentage of the rest of the year correlates with the record from the previous year. And the third column shows how the remainder-of-year winning percentage correlates with the preseason Vegas win totals.

Week	Record to this Point	Last Season Wins	Preseason Win Total
1	19%	31%	43%
2	24%	30%	43%
3	32%	29%	42%
4	33%	28%	41%
5	36%	27%	40%
6	40%	25%	37%
7	41%	24%	36%
8	43%	23%	35%
9	44%	23%	34%
10	48%	23%	34%
11	47%	21%	32%
12	45%	21%	30%
13	37%	20%	28%
14	34%	19%	26%
15	28%	14%	20%
16	24%	19%	21%

Four weeks into the season, we already know more about the rest of the season by looking at each team’s record than we do looking at their record from 2018. And yet, the Vegas win totals are still more meaningful predicting the rest of the season than each team’s record to this point. We need to wait for two more weeks before we reach the point that the two flip, and what we have seen in the season so far becomes more predictive than what we expected coming into the season.

More than any other sport, football is chaotic and random. A sixteen game season, with a quick aging curve and high injury risk, all combines to make it thoroughly unpredictable. Typically nearly half of each playoff field turns over from one year to the next. And yet, despite all this, there remains a great deal of value in looking at our expectations from before the season, value that lingers even weeks after games have started to be played.