Why QBR Is Pointless

This is going to be filed into the unpopular opinion category, specifically unpopular to analytics lovers. I buy into the analytics movement as much as possible. For instance, five years ago, I thought first down was a time to run the ball in most scenarios. Five years ago, I thought you had to establish the run to set up play action. Since then, I have learned (through analytics) that isn’t the case. When it comes to quarterback play, however, I simply can’t buy into QBR as a good metric to judge who the leagues best quarterbacks are.

What is QBR?

First, we need to understand just what QBR is. Essentially, for each individual play, QBR asks the question “how successful was this play for this team, given the plays context” for a quarterback. Context is down and distance, distance to the endzone, and time remaining until half. Before the ball is snapped each play, the context of the play gives the offense a “net score advantage” to expect, also known as Expected Points Added (or EPA). After the play, the change in those factors (either positive or negative) lead to a change in the team’s net point advantage.

That change in the expected points caused by the outcome of the play represents the play’s value, or its Expected Points Added (EPA), given all the context. For all plays in which a quarterback is involved -– passes, rushes, sacks, penalties, fumbles, etc. -– the team-level EPA is calculated and then divided among a quarterback and his teammates. In other words, was the play successful and how much of that success is a result of a quarterback’s skill? (via ESPN.com)

The total EPA is then added up and divided by the total number of clutch weighted plays. The final number is translated on a scale of 0 to 100, with 50 being average and 75 being pro bowl level. There is more to it, but if I explained it all, this would be a 5000 word column, so let me jump into my issues with QBR.

It’s Weighting Isn’t Judging Correctly

I can’t pretend I know everything about this metric, largely because I don’t do a lot of studying on analytics I don’t trust or use. For example, up until today, I knew that QBR weighted a quarterback more or less based on situation, but didn’t know exactly which situations mattered more or less. Personally, I assumed that QBR would REWARD a quarterback who is playing well while losing by a lot. The reason behind this is because I know that a quarterback that is getting blown out is on an inferior team that is being out coached and wouldn’t have the advantage of a run game. In addition, a quarterback playing from behind has another disadvantage of the defense playing the pass full time, with the defense not having to worry about run fits, allowing the pass rushers to pin their ears back and pass rush while seven or eight defenders drop into coverage. I assumed, that for all of these reasons, a quarterback would be REWARDED for playing well while losing big (assuming the starting defense is still in). WRONG. As it turns out, QBR has what’s called a “down weighting” feature for this situation. If a quarterback is losing 35-0 and throws three touchdowns in the second half to make it 35-21, those aren’t graded the same because a defense is playing conservative and dropping more players into coverage (which makes life harder for a quarterback in many ways). This is just my opinion, but this seems counter productive to what QBR is trying to do.

Blowout Wins Are… Supreme?

Knowing what we know about EPA being divided and distributed to the whole offense, and knowing that QBR down-weights a quarterback if they are getting blown out, why are blowout wins all over the all time highest QBR games? If a quarterbacks team is blowing another team out, it’s a great indicator that that team is probably a lot more talented, and the offense as a unit is playing extremely well. Why does a quarterback get down weighted for playing well on the inferior team but praised for playing well on the superior team? In fact, if you look at the top 49 all time highest QBR games, you’ll only find two games where there’s a close margin of victory, and you’ll also see zero losses. Look for yourself.

They apparently “adjust for opponent” now, unlike in the earliest versions of the stat, but definitely not much if you look at the list above. For a stat that’s based on ideas such as “circumstance“, “context” and “difficulty of situation“, and it “doesn’t grade all plays equally because this turnover is worth less than this one” and so on and so forth, this seems very hypocritical for their all time best QBR grades being from cupcake wins.

QBR Doesn’t Think Touchdowns Are Equal

QBR is a metric that uses weights. Therefore, QBR weighs a 65-yard touchdown throw with 57 air yards heavier than a one yard touchdown throw from the goal line. Sorry analytics guys, a touchdown is a touchdown. Whether or not a quarterback throws the ball five or 99 yards, it scored the same amount of points. Sam Darnold’s 92 yard touchdown throw is no better than Tom Brady’s one yard touchdown throw. The analytic faithful want to reward a quarterback for throwing a long touchdown but not for a short touchdown, despite the score board saying six points for both.

QBR Is Not The Best Indicator Of Winning, TD:INT Ratio Is Actually Better

QBR truthers claim QBR is the ultimate indicator of a quarterback winning. Unfortunately, if you look at the quarterbacks in the top 10 QBR ranking, you’ll find that they are a combined 30-26-1.

Total QBR Leaders and Record (through Week 6)

  1. Dak Prescott (3-3)
  2. Russell Wilson (5-1)
  3. Patrick Mahomes (4-2)
  4. Deshaun Watson (4-2)
  5. Carson Wentz (3-3)
  6. Lamar Jackson (4-2)
  7. Matt Ryan (1-5)
  8. Daniel Jones (2-2)
  9. Matt Stafford (2-2-1)
  10. Joe Flacco (2-4)

Recently, there’s been a lot of push back against touchdown to interception ratio being used to judge quarterback play. I understand the push back because a quarterback that plays one game and throws three touchdowns and no interceptions will obviously have a higher ratio than most quarterbacks that play the whole season, which isn’t necessarily a great reflection of who the better quarterback is. But as far as winning goes, TD:INT ratio is actually a better indicator of winning than QBR. The top 10 quarterbacks in TD-INT ratio are 40-15-1 (minimum 4 starts) according to my calculation.

Highest TD:INT Ratio and Record (through week 6)

  1. Russell Wilson (5-1)
  2. Patrick Mahomes (4-2)
  3. Kyle Allen (4-0)
  4. Matt Stafford (2-2-1)
  5. Carson Wentz (3-3)
  6. Deshaun Watson (4-2)
  7. Aaron Rodgers (5-1)
  8. Tom Brady (6-0)
  9. Jacoby Brissett (3-2)
  10. Kirk Cousins (4-2)

In no way am I claiming TD:INT ratio is THE best indicator of winning, but you can’t consider high QBR grades a good indicator of quarterbacks winning football games.

QBR Isn’t An Indicator Of The Leagues Best Quarterbacks

Do you want to know the best way to judge a quarterback? Your eyes. This may seem odd to you, but QBR doesn’t have eyes. In many cases, the most talented quarterbacks in the league are in the top 10, but so are many quarterbacks that clearly don’t belong in the “elite” category of quarterbacks.

Exhibit A

Currently (week 7 of 2019), Daniel Jones, a rookie quarterback having thrown five touchdowns and six interceptions, with a 2-2 record (both wins coming against mediocre opponents), and is on pace to throw 21 interceptions, is ranked 8th in the NFL in QBR. This is good enough to be ranked higher than Matt Stafford, Tom Brady, and Aaron Rodgers, to name a few. Anyone that has to pick from one of those names to win a game isn’t taking Daniel Jones.

Exhibit B

In 2018, Mitchell Trubisky finished the season ranked THIRD in the NFL in the QBR rankings. This was ahead of Roethlisberger, Luck, Brady, Wilson, and Wentz. Anyone who objectively observed Trubisky with their eyes at any point last season would have known he was an average quarterback with a great supporting cast and brilliant offensive play caller that elevated him. QBR was not an objective observer.

Exhibit C

The final QBR ranking of 2017 had Case Keenum ranked SECOND in the NFL among all quarterbacks. Case Keenum has made a career by being a journeyman quarterback. He’s played on seven different teams in his career. In fact, he was let go by the team he played for THAT season. No quarterback that is the second best in the NFL would ever be let go by their team.

ESPN Doesn’t Even Trust It’s Own Formula

After digging for the greatest quarterback games of all time according to QBR, I found that for a long time the highest QBR in a single game ever was by Charlie Batch, a backup quarterback for the Pittsburgh Steelers. In that game, he threw for a whopping ONE touchdown and two interceptions. After this got scrutinized by fans, they edited their formula to fix that bad blunder that their prized metric created. There’s no issue with trying to make a formula better, but changing it because your initial one was way off gives a good indication that their general idea of good quarterback play is not measured properly.

“Designed With Tebow In Mind”

Tim Tebow was never even an average quarterback. Never had the potential to even be middle of the pack. But sometimes he would make a splash play and it would make his QBR skyrocket. In the 2011 wildcard round, Tebow posted the highest QBR of all quarterbacks that played in that round. His stat line was 10/21 (sub 50% completion) 316 yards, two touchdowns, no interceptions, and a rushing touchdown. A fine game, and enough to win.

Somehow, he apparently had a better game than Drew Brees, who was 33/43 for 466 yards, three touchdowns and no interceptions. All five of the Saints second half drives ended in touchdowns, and the Saints pulled away from the Lions in a game that the Lions were winning 14-10 at half. If that performance by Drew Brees was worse than Tebow’s, I guess I prefer bad football.

I could go on and on about what makes QBR an inconsistent, shady, unreliable metric at grading quarterback play. I wont, because it’s unlikely any of it would be enough to change the mind of the people who firmly believe QBR is the ultimate measurement of whether a quarterback is actually good or not. As for me, I’m going to continue to let my eyes judge whether or not a quarterback is elite, great, good, average, below-average, or bad.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s