How Coaches and the NYT 4th Down Bot Compare
nytimes.com
How Coaches and the NYT 4th Down Bot Compare
1–10 of 45 posts
Re: How Coaches and the NYT 4th Down Bot Compare
#2Hm, so the data points for likelihood of getting another first down is based on data from the fourth downs that coaches actually attempted? It may be that the fourth downs they didn't attempt were not as propitious for some reason - wind (which the authors mention), injuries on key players, lateral positioning which doesn't favor one formation or another, etc.
Re: How Coaches and the NYT 4th Down Bot Compare
#3Hm, so the data points for likelihood of getting another first down is based on data from the fourth downs that coaches actually attempted? It may be that the fourth downs they didn't attempt were not as propitious for some reason - wind (which the authors mention), injuries on key players, lateral positioning which doesn't favor one formation or another, etc.
Agreed. The sample is very likely tainted by factors that made success much more likely than average if we believe that NFL coaches are both conservative (they forgo punts only when the odds are much in their favor) and rational (they are win maximizing).
Still, there is probably a case to be made that coaches should be a bit more aggressive, but that edge could be small enough that other factors (like perceived incompetence) dominate it.
Re: How Coaches and the NYT 4th Down Bot Compare
#4Hm, so the data points for likelihood of getting another first down is based on data from the fourth downs that coaches actually attempted? It may be that the fourth downs they didn't attempt were not as propitious for some reason - wind (which the authors mention), injuries on key players, lateral positioning which doesn't favor one formation or another, etc.
The choice to go for a 4th down conversion is more likely than not primarily decided by the current score and time left in the game. However, the success of a 4th down conversion probably isn't heavily influenced by those factors (other than perhaps player fatigue), which would suggest that coaches are indeed too conservative.
Re: How Coaches and the NYT 4th Down Bot Compare
#5This does not take into account an important factor that is somewhat intangible, but real to the psychology of the game: point swings. If you go for it on 4th and 1 at your own 1 and lose you are definitely down 3 points and probably 7. This puts pressure on your offense--which just failed on a 4th and 1--to somehow make it all the way down the field next time. That can be demoralizing for that offense, paralyzing in fact. That is much harder to model, but an important part of a correct model, I believe.
Re: How Coaches and the NYT 4th Down Bot Compare
#6Why would you ever go for it on 4th and 1 inside your own 10 yard line? You would put yourself at a severe disadvantage if you failed.
Since the data from this is from previous plays in the last 10 or so seasons it is flawed. There probably isn't very little data of a team going for 4th and 1 and that deep in their own territory. The only time this has probably happened is in extraordinary circumstances - specifically in a game-winning drive scenario with time expiring, where the defense is in 'prevent'. In these cases it is highly successful in that situation but you don't have the time to get the score.
Re: How Coaches and the NYT 4th Down Bot Compare
#7This was discussed some in https://news.ycombinator.com/item?id=6728821
I've thought about it some since then and I'm also not convinced that expected points is a good metric. That will maximize the expected point differential per season but I am not convinced (although I could be wrong... I haven't put pencil to paper to make my thoughts rigorous) that there are enough possessions in the game on average to make this metric useful even in the first half.
Re: How Coaches and the NYT 4th Down Bot Compare
#8Why would you ever go for it on 4th and 1 inside your own 10 yard line? You would put yourself at a severe disadvantage if you failed. Since the data from this is from previous plays in the last 10 or so seasons it is flawed. There probably isn't very little data of a team going for 4th and 1 and that deep in their own territory. The only time this has probably happened is in extraordinary circumstances - specificall…
In place of 4th and 1 data AdvancedNFlStats would substitute 3rd and 1 stats. And I would agree that this substitution is suspect. Just a bit shy of bogus in fact.
Re: How Coaches and the NYT 4th Down Bot Compare
#9Hm, so the data points for likelihood of getting another first down is based on data from the fourth downs that coaches actually attempted? It may be that the fourth downs they didn't attempt were not as propitious for some reason - wind (which the authors mention), injuries on key players, lateral positioning which doesn't favor one formation or another, etc.
The choice to go for a 4th down conversion is more likely than not primarily decided by the current score and time left in the game. However, the success of a 4th down conversion probably isn't heavily influenced by those factors (other than perhaps player fatigue), which would suggest that coaches are indeed too conservative.
is more likely than not primarily
This seems pretty loose. I'm not trying to be pedantic, but the theoretic flaws in the data are such that you would want a pretty tight logic in your model to have any level of comfort with it. The proportion of the state-space that is unobserved seems ~large and critically relevant.
Re: How Coaches and the NYT 4th Down Bot Compare
#10Tough to take everything into account, but I find it troubling that all non-success/failure outcomes are not part of the calculation (ex: 8-yard gain on a 4th and 10; a fumble on a 4th and 2 run, etc). It simplifies the formula to the layman, but I have to imagine that it adds enough additional variance to matter, especailly considering we're talking in variances of less than half a point in their stated example.
Also (and to their point), there is a ton of variation between offenses, defenses, punters, running backs, etc. Using this as a rubric is a nice idea for a rule of thumb for an armchair quarterback, but I'd strongly disagree with using it to accurately criticize any decision within a couple points of expected value.