Quarterback Effectiveness. A Rating System.

[Riddlewire]

I worked this up last year near the end of the season. Obviously it's no longer quite as relevant as it would have been before the draft. But, if I don't post this now, I'll never get around to it.

We all know that the QB Efficiency rating is garbage. I pointed to multiple articles (in my Voodoo Mathematics thread) which showed that Geno Smith's production was completely fraudulent, yet he had a pretty good QB rating (#6 in FBS). I wanted to come up with my own macro rating which did a better job of revealing the true effectiveness of a quarterback. I spent a long time thinking out the statistical compilation which would give a true indicator of a quarterback's abilities WITHOUT artificially enhancing his rating with phoney yards piled up on "extended handoffs" (which were 30% of Smith's plays). Unfortunately, at the end of it all, I discovered that there's just not enough data available to take all factors into account when determining a quarterback's performance. Records are kept for things like drops, incompletions, reception location, yards-to-go, et cetera. But you have to pay big bucks to an entity like Elias in order to obtain those numbers. The data that are freely available to the average fan will never yield a complete picture of play on the field. As long as you accept that all macro ratings are doomed to be incomplete, then you can at least appreciate any trends across multiple ratings that may appear.

With that disclaimer out of the way, I present to you my QBE or Quarterback Effectiveness rating.

My goal was to develop a metric which could show not only how well a particular quarterback ran his own system, but also how much his decision-making on the field affected his team's offense. To this end, I utilized the following data: Completion Percentage, Yards-per-Attempt, Total number of rushes (which can also be defined as the total number of times a QB is tackled holding the ball), Total loss yards, Total number of plays (rushes plus pass attempts). Since I'm not trying to show how well one team plays against the opponents on their schedule, touchdowns, interceptions, and total yards are irrelevant. Those would be indicative of the performance of the entire team, including the defense, as well as the coaching staff and the schedule maker. And the Quarterback shouldn't get credit for those things.

Before the conference championship games took place last season, I took the top thirty quarterbacks in terms of their passer efficiency rating. I then input the data above using my formula for QBE (which I will try to explain shortly). Here were the results:



The last column is the Quarterback Effectivenss rating. To be honest, the whole concept is a little inverted. I was trying to focus on quarterbacks who hurt their teams with poor decision making, but, since I didn't want a list of "worst to first", I just flipped it upside down and the QBs are now ranked according to how little their actions end up harming their teams. According to my formula, of the top thirty quarterbacks in FBS last year (as of the time I polled the data), Jordan Lynch runs his team's offense more effectively than any other quarterback in the country. Quarterbacks have to decide after the snap whether or not to throw, keep, or hand off. And Lynch's decision-making yielded the best results on average for his team. The other columns, in order, are:

Completion Percentage: self explanatory

Passing yards per attempt: ditto

Rushes: As stated above, this is the number of times a QB is tackled with the ball, whether on a run or a sack

Loss: Total yardage lost on rushing attempts, which includes sack yardage

Plays: Total number of rushes plus total pass attempts

CarryPct: Number of times the QB was tackled with the ball out of the total plays he was involved in, expressed as a percentage)

LAF = Loss Avoidance Factor: This is the simplest formula, yet it's nearly impossible to conceptualize and even more difficult to explain. I have actually held off making this post for two weeks while I try to figure out how to describe the meaning of this statistic. I can't really do it, other than in my own head, so it will just have to remain something of a "black box" number like ESPN uses for their TOTAL QBR nonsense. The number is obtained by dividing the total number of rushes by the number of Loss Yards and then determining the distance that value is from 1.00. Example: If a QB was tackled with the ball 200 times (whether on a sack or a run) and had compiled just 100 total loss yards on the season, then his LAF would be (2.00-1.00)= 1.0. He averages two carries before he compiles a full yard of loss, so his net LAF is positive. On average, he doesn't hurt the team when he's tackled with the ball. If, however, he was tackled the same 200 times, but had 300 loss yards on the season, his LAF would be (1.00-0.66)= -0.33 (below 1 yields a negative value, obviously). He only averages 0.66 carries before he compiles a full yards of loss, meaning he's losing yardage every time he's tackled with the ball instead of throwing it (on average), so his LAF is negative. This number is used only as a modifier, so its obtuse nature shouldn't be too concerning.

BHF = Ball Handling Factor: Carrypct*LAF*50. If a QB has a high carry percentage, then his LAF (and thus his ability to avoid losing yards) affects the team more. This statistic shows how much or little a QB's loss avoidance factor is harming his team's offense. The "50" multiplier is included because the other components of the final statistics are expressed as percentages of 100 and I considered ball handling to be roughly half as important in the final calculation.

FDE = First Down Efficiency: This number shows the percentage of first down yardage (10 yards) that the QB achieves per pass attempt. This is another statistic which reveals how much a QB's decision making affects his team. A QB who can't run through his reads quickly will usually take the quick dump on a short yardage play. This is the number which exposes most of the spread offense quarterbacks like Geno Smith. Although, even then, it still doesn't show just how bad Smith is because his tosses behind the line (33% of his passes) usually went for long gains. That just shows how difficult it is to defend Holgorsen's playbook.

CPF = Completion percentage plus First Down Efficiency percentage.

QBE = Quarterback Effectiveness Rating: This is the CPF number adjusted by the ball handling factor.

Again, this is not meant to be a measure of how "good" a quarterback is. Rather, it's meant to show whether or not (and to what extent) a given quarterback's decisions and execution hurt his team. Is it highly flawed? Yes. But so is the current QB Efficiency rating. And ESPN's QBR system is a complete joke. I have some ideas for improvements to make for next season. If I'm feeling particularly masochistic, I might give it an overhaul and track the top quarterbacks during the year.

I welcome any thoughts and vitriolic criticisms.

 

[Leonardfan]

Riddlewire,

I cannot begin to fathom how much work, brainstorming, research and statistical analysis went into this. I give you total appreciation and credit for tackling something this ambitious. I really appreciate providing another type of measurement of QB play as opposed to the normal system that has been and place and the ESPN contrived non-sense Total QBR.

Also, I appreciate where you were coming from with your motivation for pursuing this ( I think it's from the overhyping of black quarterbacks). With the advent and widespread use of the horizontal passing game that either originates at or behind the line of scrimmage or within 5 yards of it, a new way of analyzing QB play is needed.

Admittedly, I am still trying to wrap my mind around and totally comprehend your system but overall I can really appreciate were you are going with it.

Just a couple of comments/criticisms (I'm using that word loosely)would be not taking strength of schedule/opponents into account in some capacity. Just using Geno Smiff as an example - he totally tanked against better competition. Also, if the statistics could somehow measure up numbers put up in garbage time or during blowouts. I realize that those are hard things to breakdown though.

Like I said before I think what you have done is great. I hope you can apply it to the upcoming season if at all possible. It would be great to see week by week analysis but understand how difficult and time consuming it can be. Keep up the great work!

 

[Riddlewire]

Thanks for the comments.

Regarding specific game performances (as you mention with Smith), that is obviously something that scouts will look at (Well, they're SUPPOSED to, but we all know that rarely applies when they want to see only what they want to see). Some things just can't be quantified. My rating was intended to approximate a rough average of how well or poorly a quarterback "ran" his own team's offense, by exposing those quarterbacks who do dumb things when they shouldn't. To see this a little more clearly, let's take a look at a few of the quarterbacks.

Jordan Lynch has the best Loss Avoidance Factor by far. When he's tackled with the ball (carry or sack), the impact on his team is not negative. He's one of the few qbs who averages positive yardage whenever he's tackled. In addition, since his carry percentage is so high, his team benefits the more he's involved in carrying the ball, as opposed to throwing it more frequently. Lynch's skills are perfectly suited to the offense that NIU runs.

Now let's take a look at a quarterback on the opposite end of the scale. Brett Hundley's LAF is negative. Every time he is tackled with the ball, the team is losing yardage (again, averaged over the whole season). Unfortunately for UCLA, Hundley is tackled with the ball a fairly high percentage of the plays he's involved in (25% carrypct). So his poor LAF is hurting his team more than a player like Tyler Wilson, who has a worse LAF, but is rarely tackled with the ball in his hands. Hundley's decision making regarding when to throw the ball is very poor. Also, since his first down efficiency is so low, he's not really helping his team all that much when he chooses to throw it, either.

What about Mr. #1 QB in the Draft, EJ Manuel? Yep, he screws his team over, too. His first down efficiency is pretty good, but he runs the ball (or is sacked) far too often and loses yardage on all those tackles. He is hurting his team by not being able to get the ball to his receivers more often.

If there was one statistic I'd like to be able to incorporate, it would be "average catch location". This would tell us the exact point where the quarterback threw the ball to while disregarding all yards-after-catch. Smith would plummet down to the bottom of FBS if I had access to such numbers. Alas, I can only work with what is available.

 

[Riddlewire]

Oh, one more thing I wanted to point out, since I consider it pretty significant.

Only eleven quarterbacks in all of FBS last year had a positive Loss Avoidance Factor.
Nine of those eleven were white.

If football coaches ever performed statistical analysis of their players similar to the above, perhaps we'd see far fewer black quarterbacks recruited to run spread offenses.

 

[scroat]

I wish there was a way to factor in garbage time stats. Is it really impressive to score 4 touchdowns if your team is down by 6 scores for most of the game? Cam Newton springs to mind.

 

[Riddlewire]

I decided to finish the ratings with the final season statistics and with every eligible quarterback in FBS. There were a fair number of quarterbacks who didn't qualify for statistical listing by the NCAA's method. I included most of them in my chart.



As you can see (if you looked at the previous listing), the rankings changed somewhat after the conference championships and bowl games. Jordan Lynch is no longer on top of the QBE ratings. His performances in NIU's last two games were not stellar. However, he's still one of the best in the country. Collin Klein maintained his high level of play to finish off KSU's season and gets the nod as the most effective quarterback in the country.

So why are three quarterbacks listed ahead of Klein? Well, none of them threw enough passes to qualify for the NCAA's QB efficiency statistic. Oklahoma State's Walsh was injured at midseason and didn't play any more, and the other two are triple option quarterbacks (as is Ga Tech's QB at #5).

I think the trends that appear are pretty interesting and help to enhance the validity of the QBE rating. As would be expected, "system" quarterbacks follow a strict formula for their on-field play. As long as they can follow the rules of their system and take the appropriate action, their opportunities to make mistakes are minimized. That's why the triple option quarterbacks score so high. Although, as you can see, the two white triple option QBs outperform the two black ones. As stated earlier, though, the QBE isn't a measure of how "good" a quarterback is (CPF is a better indicator of that). QBE attempts to show how much a particular QBs poor play negatively affects his team (which would indicate that he's not making good decisions on the field).

Earlier, I gave Brett Hundley as an example of a quarterback whose decision making was especially harmful to his team. His numbers as a passer were fairly decent, but he runs too much and doesn't get rid of the ball quickly enough. When a play ends with him being tackled, the impact on his team is usually negative.

To see an example of the inverse, let's take a look at another black quarterback. Denard Robinson is a terrible quarterback. Everybody agrees on that, including the NFL. But what does QBE say about his decision making? Well, Robinson runs very well and, when he does, the damage to his team is negligible. In his case, his frequent decisions to take off running (or get sacked for a very small loss) are actually a good thing. Why? Because they're sparing his team the effects of his awful quarterback play.

 

[Extra Point]

Statistical analyses are always interesting.

There were 7 quarterbacks taken in the first 4 rounds of the 2013 NFL draft; Smith, Manuel, Glennon, Nassib, Barkley, Wilson and Jones.

How does your system project them to do in their pro careers? Or, what traits do they have that could affect their performance at the pro level?

 

[Riddlewire]

That's a tough question, as QBE isn't suited to predictive assessments. It's more of a "What are they doing wrong right now?" evaluation. That, of course, also takes into account the offense they run with their college teams. Like I pointed out, Denard is an awful quarterback, but he fits reasonably well for the current Michigan team because they can protect him long enough for him to get out of danger and dump/run (which he does pretty well). He's not going to be playing QB in the NFL, but if he were, his QBE score (when evaluated in terms of its underlying components) would suggest he would never escape the practice squad.

Since you asked, I think I might be able to provide a better answer if I compiled the career statistics for the quarterbacks you mentioned. After all, long term averages is what QBE is all about.




So, how do we interpret these results?
Manuel ended up having the highest effectiveness score of the quarterbacks you mentioned. But remember that QBE isn't a single measure of QB ability. It's a holistic measure of how much a given quarterback hurts his own team. Look at the column circled in red (BHF). Manuel scores the worst here. Meaning that his boneheaded decisions do the most damage to his team's offense. But this damage isn't enough to drop his effectiveness score below the others. In other words, the FSU offense with Manuel under center performed well enough to overcome his poor decision making skills. But Manuel won't be running the FSU offense in the NFL. Nor will he be playing against an ACC schedule that typically matched FSU against weak ACC opponents. So I'm more inclined to think that the BHF modifier carries a bit more weight in the final tally when trying to use QBE as a predictor of future success.

Ultimately, it's a limited rating system based on limited available data. It also has a minor mathematical irregularity which I'm working to address. It's just another way of looking at quarterbacks, rather than letting that stupid Efficiency Rating determine their worth in total. That's what DWFs use to claim the greatness of Geno Smiff, which we all know is bunk.

 

[Extra Point]

I wish you luck as you fine tune your system. Perhaps you can develop it so that it can predict the future pro success of quarterbacks.

If you could do this you could market it, perhaps over the internet.

There's an interesting site called Fantasy Football Metrics, which uses statistics to predict performance. You have to pay to get their info (which I haven't.)

Maybe you can do something similar and make yourself some money. Anyway it's an idea.

 

[Riddlewire]

After some tinkering with formulas, I finally have QBE where I want it.
Not only does it do a better job of measuring the traits I want to highlight, it's now a simpler calculation and easier to understand.

Previously, QBE was doing too much "rewarding" for solid play and not enough "punishing" for poor play. That was all contained in the formulas for LAF and BHF. The reason for this is that I originally invented the Loss Avoidance Factor a long time ago as a way to compare runningbacks. Since they never have more loss yards than number of carries, it never occurred to me that I'd end up with horribly imbalanced ratios when I applied the measurement to quarterbacks (who almost always have more loss yards than carries). I fixed the problem by just inverting the calculation. Now, instead of showing how many carries a QB tallies before he accrues at least one total yard of loss, it shows how many loss yards a quarterback averages per carry. This solves two problems. First, it removes the need to distinguish between LAFs above or below 1.00 (there's a very odd reason for why this was necessary, but it would take a lot of time and words to explain it), so the formula is now very simple. Secondly, it results in increased penalty for poor decision making. With this new measurement system, the rankings changed around a little, although the good and bad quarterbacks still generally stayed within the same region of the chart.

The basic idea is the same. If a quarterback goes backward a lot (and too often), then his basic QB performance is adjusted to reflect those poor decisions. If you don't want to try to understand all the numbers, then just pay attention to the BHF column (Ball Handling Factor). That's the number which shows poor QB play that hurts a team. The higher this number, the worse that quarterback's decision making is.

One caveat here, though. QBE is (and will always be) nothing more than a "helper" rating. Something to be considered after quarterbacks have passed the eye test over the long term. The reason for this is poor data availability. Until the NCAA starts keeping track of and publishing stats for reception location and total yards-after-catch for each quarterback (thus determining how much yardage should be credited to the WR instead of the QB) no rating will ever come close to showing the true abilities of each quarterback. Certainly not the NCAA's awful Quarterback Efficiency rating, which is more a reflection of a team's total offense and strength of schedule. The QBE is my attempt to remove those factors from the equation and show only what we can determine about the quarterback himself. This is best seen in the case of - who else - Geno Smith. The article I linked to in my Voodoo Mathematics thread showed that ONE THIRD of Smith's passes were thrown behind the line of scrimmage, including that ridiculous "touch pass" which ought to be credited as a handoff. And West Virginia managed to gain an awful lot of yardage on those passes during the season. All of that is credited to Smith as downfield passing yards, which outrageously skews his passing numbers. Unfortunately, there's just no way I can adjust for this factor. Without personally watching every play that every quarterback in the country made during the season and tracking the reception locations and yards after catch, these screwed up passing numbers will always be with us.

Ok, enough words. Here's the final QBE rankings.



Of all the draft eligible quarterbacks from last season, Colby Cameron ended up with the highest QBE score (although Nick Florence had a higher QBE score). I'm not sure if Cameron is still with Carolina or not. Collin Klein, who finished right behind Cameron, was cut by the Texans.

One last thing. Regarding QBE as a predictor of future pro performance, I decided to take a look at the career QBE scores of some recent QB entrants into the NFL. I'm sure you've all heard of these guys:

Tim Tebow 147.49
Andrew Luck 147.30
Robert Griffin 134.08
Kirk Cousins 127.61
Russell Wilson 126.19
Colin Kaepernick 122.58

If there are any past QBs you'd like me to compute the numbers for, just let me know.

 

[Extra Point]

If there are any past QBs you'd like me to compute the numbers for, just let me know.

I'm a big fan of Nick Foles, who went to college at Arizona. I think he might be a franchise QB for the Eagles.

I'd be curious what Foles's numbers were for his last year in college and for his college career.

 

[Riddlewire]

Nick's QBE for his final season was 131.25, based on a CPF of 146.51 and subtracting his Ball Handling Factor Score of 15.26. His composite passing number (CPF) was very good. When he took losses, they tended to be larger than average. However, he didn't get tackled with the ball a significant percentage of the time. He's good at avoiding sacks and he doesn't recklessly take off with the ball often.

Nick's career QBE number was 123.75, based on a CPF of 139.68 and subtracing his Ball Handling Factor Score of 14.93. His career QBE is not quite as good as his senior season because he significantly improved his passing numbers his last year. However, his Loss Avoidance and Ball Handling was nearly identical throughout his career, which shows a consistent ability to get the ball out early (although his sacks result in larger than average losses, so there's a bit of give and take in his evaluation).

 

[Extra Point]

Nick's QBE for his final season was 131.25, based on a CPF of 146.51 and subtracting his Ball Handling Factor Score of 15.26. His composite passing number (CPF) was very good. When he took losses, they tended to be larger than average. However, he didn't get tackled with the ball a significant percentage of the time. He's good at avoiding sacks and he doesn't recklessly take off with the ball often.

Nick's career QBE number was 123.75, based on a CPF of 139.68 and subtracing his Ball Handling Factor Score of 14.93. His career QBE is not quite as good as his senior season because he significantly improved his passing numbers his last year. However, his Loss Avoidance and Ball Handling was nearly identical throughout his career, which shows a consistent ability to get the ball out early (although his sacks result in larger than average losses, so there's a bit of give and take in his evaluation).

Thanks. His score of 131 puts in the approximate range of Matt Barkley and Landry Jones, two players I'm also interested in.

 

[Wes Woodhead]

Riddle this is a damn interesting deal you got going here. Im interested in Kevin Kolb if you got the time.

 

[Riddlewire]

Kolb's career QBE was 119.39, based on a CPF of 144.40 with a BHF penalty of 25.01. As you can see, Kolb was a skilled passer, but he suffers a large ball handling penalty. This is because, although his loss yardage was only slightly worse than average (2.16 loss yards on average), he was tackled with the ball far too often (23% of the time on non-handoffs). Since he was accumulating a fair amount of positive rushing yardage in each game, it seems obvious that he was not losing significant yardage when he was intending to run. That means when he planned to throw it and was sacked, the losses were usually quite large. This might have been merely an artifact of his offensive system. If he took a large drop out of the shotgun every time, he would be in a position to suffer a big loss if he couldn't get the ball out in time. However, such a large drop should assist him in avoiding those sacks. Overall, that's not really a point in his favor. My evaluation, according to my own system here, and based on his career QBE metrics, is that Kolb is a very boom-and-bust type of quarterback. He can move you down the field effortlessly, but at any moment he could put you in a big hole with a poor play.

As for Kolb's senior season, he finished with a QBE of 133.41, based on a CPF of 155.79 with a BHF penalty of 22.38. It's interesting that both Kolb and Foles had BHF scores that were so close between their senior years and their careers. This suggests to me that these tendencies are a natural part of a quarterback's game and lends credibility to this particular measurement (EJ Manuel and Barkley also expressed this consistency). Anyway, Kolb's passing performance as a senior was excellent and his overall effectiveness was very good, although he still could make a poor play on occasion and put his team in a hole.

 

[Wes Woodhead]

My evaluation, according to my own system here, and based on his career QBE metrics, is that Kolb is a very boom-and-bust type of quarterback. He can move you down the field effortlessly, but at any moment he could put you in a big hole with a poor play.

I think your system is very well formulated Riddlewire. I think your evaluation on Kolb rings about as true as it could be. I really root for the guy because hes a good ol boy from Texas.

I watched him play a couple High School games in Stephenville years back and was deeply impressed. Ive been a huge fan of his since then. You know how it is when you root for a guy its easy to make excuses for his shortcomings. I recall more than once at Houston he would lead them down field with amazing passes, and runs only to do a Fran Tarkenton, and lose 15 yards scrambling around and getting sacked.

 

[dwid]

great job!
I want to figure out how to do the calculations, I want to see how Tebow ranked during his years starting and then maybe put them up against other qbs that had great seasons.

 

[Riddlewire]

dwid,

Here is the spreadsheet for the 2012 season. Just make a new row for a player and input the first five columns of data (which I get from the NCAA's website here. You can access previous years by changing the number in the address.) I added the results for each year of Tebow's career as well as his total career numbers. I also added Case Keenum's career results because I had already put him in there on my own.

The comments for the LAF column are no longer correct because I changed the formula (which I described above). The formulas are all very simple, which you can see by clicking on the column cells.

Just remember, these are "interpreted" measurements based on calculated data. You have to look at combinations of numbers to derive any meaning from them.
Quick example: Johnny Manziel versus Case Keenum (I was curious to see how two Sumlin coached QBs compared).
Manziel and Keenum had similar QBE scores. Is that just because the Sumlin offense produces similar results? Nope. As you can see, their Ball Handling penalty is nearly identical (12.7 vs 12.3). But how they arrive at that number differs greatly. Keenum took moderate losses when he was tackled with the ball (career LAF of 2.07, although that was still better than the national average). Manziel, on the other hand, almost never takes significant losses when he's tackled with the ball (LAF of 0.8). But Keenum's losses weren't harmful to his team because his carry percentage was so low (11.9%), whereas Manziel's carry percentage was much higher (32%). Ultimately, Keenum's slightly better passing numbers (CPF) give him a barely higher QBE score. The two quarterbacks are similarly effective for their teams. A deeper analysis shows why they perform the way they do, and (if you were a coach) suggests possible tweaks or changes that you might need to make to your offense. Like, for instance, Florida. Jeff Driskel has the worst BHF in the country. They should NEVER let him run with the ball and they should shorten his drops and reduce his reads as well.

Oh, one last thing.
I calculated the national averages for the 2012 season so you can see if a particular quarterback is performing better or worse than average.
Carry Pct= 19.6%
LAF = 2.426
BHF = 18.1065
FDE = 74.30
CPF = 135.21
QBE = 117.10

=================================================
Note: Make sure you uncheck that "download with filefactory download manager" box when you download the file. I think that's probably some kind of annoy-ware garbage.

 

[dwid]

thanks!

one question so far, where do you get the amount of first downs the player made for passing for FDE? The site I have for the player page has everything so far but I am stuck at a dead end for the FDE. I guess I would have a hard time plugging in the categories as well, since you said just input for the first 5 rows. I was able to do the calculations for carrypct LAF and BHF and did it with several other players to make sure I had it correct but at a loss as to what to do with the rest. Trying to do Cam Newton's 2010 season because there is a debate on the best college football season, 2007 Tebow or Cam Newton 2010. 66.07 10.19, 264, 189, 544, .49, .72, 17.64. Is lower than 18 better than average or higher than 18 better? because it seems a lot of the running qbs (even White ones) have lower than 18

 

[Riddlewire]

FDE is a meta-statistic. Number of first downs isn't part of the calculation. It's a lot like the "Yards Per Attempt" stat, which is also just conceptual (after all, you're not getting any yards for each incompletion, so how can you be awarded 8.54 yards on that pass when it hit the ground?). But YPA is a widely accepted statistic which is meant to make use of metadata to indicate trends about other data, in this case, the combination of accuracy and the typical distance on that accuracy. I considered Yards per Completion, but, ultimately, Yards Per Attempt encapsulates more data in a single number.

So, what is FDE? Well, if you don't mind the silliness of the term, I'd call it Meta-metadata. As stated in the column comments, it's the percentage of a first down (10 yards), that a quarterback achieves with his yards-per-attempt. In other words, every time he throws the ball, he has a X% likelihood of getting a first down. That "X" is his FDE. Now, of course, this is based on long term average over a season/career, because not every down is going to be 1st-and-Ten or 3rd-and-Ten. But the most common distance a QB will face throughout his career for any down is ten yards (due to every 1st down being that and whenever he throws incompletions on 1st, 2nd, etc.)

I know this all sounds very ethereal, but I studied advanced statistics in college. The most important lesson I learned was that stats based on hard data are the best liars. Governments and the lapdog media rely on that truth in order to peddle their poisons. My intention with QBE was to use averages of approximations to indicate long term trends, eschewing such direct data. I'll let everyone decide for himself whether or not he believes that such a measurement system has any intrinsic value.

Tebow vs Newton
Your numbers were all right except I got a different computed value for Newton's BHF. When I input his completion percentage, yards-per-attempt, carries, loss yards, and total plays (the same values as you list) into the spreadsheet, it returns a value of 17.37. Something's obviously "off" somewhere. Maybe we have rounding set differently. Not a huge difference, I suppose. Anyway, since I changed the nature of the calculation, I really should rename Ball Handling Factor (BHF) to Ball Handling Penalty, as that is what it ultimately shows. You want this number to be as low as possible if you're a quarterback because it gets subtracted from your CPF (comp% plus first down efficiency) score. So which season was better? Tebow's 151.21 is slightly higher than Newton's 150.60. Newton's passing numbers were better, but he also caused his team more problems by taking bigger losses more frequently. Those two offenses were somewhat similar as I recall. So it's safe to conclude that Tebow is significantly better with his on-field decision making than Newton when playing option football. With their QBE scores so similar, I'm not sure we can draw any further conclusions from my rating system (that's what eyes and common sense (and playoff victories) are for).

As far as those national averages go, that was just for the 2012 season. I suspect it goes up and down each year, although I bet it has trended upward over the past decade because of the proliferation of spread offenses where the QB gets tackled with the ball more often. Here's my best take on good/bad numbers for BHF: 0-7=Excellent, 8-15=Good, 16-20=Average, 20+=Bad. A quarterback with a BHF of 25 might still be an effective QB (as long as his passing is good enough), but he's not optimizing his playmaking. He could could be doing a lot better by getting the ball out quicker and/or running less.

I hope that helped make things a little clearer. If not, post away.

 

[Riddlewire]

The mid-season QBE numbers are in.
Since most teams have finished half their schedule, I figured now would be a good time to take a look at the current quarterback QBE rankings.
Obviously, these will change dramatically by the end of the season, as many teams have feasted on cupcakes thus far (and, in the case of Teddy Bridgewater, his entire career has been a dessert cart).
Without further ado, here is the full list of players who qualified to be statistically listed by the NCAA.




And, in case you don't want to stare at that huge list, here are the Top 25 quarterbacks in QBE score:



I might provide some analysis later.

 

[Leonardfan]

Nice job Riddlewire. It will be interesting to revisit this in a few years when some of these qbs are in the NFL to see how accurate of a rating system this is and if it can be used to evaluate QBs. If so, I would recommend you trademark it.

 

[Riddlewire]

The NFL is so manipulated that I don't believe there would be any relationship.
The NFL can make any player successful that they wish or any player underperform that they wish. Perhaps it wasn't so in the past, but it certainly is now. Honestly, does anyone truly believe that a single one of the black quarterbacks in the league today could get on the field in the NFL of the 90's? Not a chance.

Besides, as I have tried to convey in the past, QBE really only suggests whether or not a given quarterback is running his own offense effectively. He might be a great future NFL quarterback, but his QBE could be low if he doesn't fit the requirements of the system.

That being said, there are still trends across multiple years that can indicate particular performance shortcomings of a quarterback. EJ Manuel, for example, had a persistently bad Ball Handling Factor during his career, suggesting that he does not have a knack for making the correct decision in a timely manner (throwing the ball, who to throw it to, tucking and running, etc.).

 

[Riddlewire]

The 2013 Final QBE numbers are in.

The nation's leader in QBE for 2013 was Bryce Petty.

Surprising? Not to me. Again, QBE is not an "accumulation" stat. It's also not plagued by the shortcomings of legacy stats like Passing Efficiency. Petty dropped below Winston in the efficiency stat at the end of the year because his output was lower in those games. But the efficiency rating fails to track errors and poor decision-making, which is exactly why I invented QBE. Petty may not have been piling up the yards and touchdowns, but the quality of his play remained high.

Here are the Top 10 in QBE:

01. Bryce Petty: 161.39
02. Jameis Winston: 157.71
03. Johnny Manziel: 148.83
04. Brendon Kay: 146.97
05. Derek Carr: 146.38
06. Teddy Bridgewater: 145.17
07. Keith Wenning: 145.11
08. A.J. McCarron: 144.90
09. Zach Mettenberger: 143.45
10. Marcus Mariota: 143.36

And here is the complete list, broken into four parts:


Some observations:
*Dink and Dunk offenses proliferated in FBS this year as more and more teams adopt no-huddle spread concepts. Throwing distances were down all over the nation. I suspect this trend will worsen over time and some coach will eventually drop his QB into a twenty yard shotgun with three receivers ten yards deep. All passes will be screens.

*White QBs have a tendency to improve as the season goes on. There were a significant number of white quarterbacks whose QBE improved from mid-season until now. Almost universally, black QBs showed no improvement after week six.

*QBE is good at showing the differences between quarterbacks on the same team. UConn had a rather unique situation this year where they played three different quarterbacks, each for almost exactly one-third of the season. The one with the best QBE score was freshman Casey Cochran. He played the last third of the season. UConn's only three wins? Their last three games (and the end of the schedule was the toughest). There were a few more cases like this. Minnesota had four losses on the season. In three of them, Mitch Leidner didn't play. It was a fairly even split with Philip Nelson the rest of the season, suggesting that Minnesota's Two-QB system might actually work best for them.

Will update if I can generate further relevant analysis.

 

[Wes Woodhead]

Riddlewire your QBE posts are among my favorite things about castefootball. I appreciate it greatly.