You should send this to the Buffalo Bill's new analytics department. Maybe a Hacker could get hired to an NFL team.
http://www.nfl.com/news/story/0ap1000000121582/article/bill-...
Every NFL play for the past 10 years in CSV format
71–80 of 105 posts
Re: Every NFL play for the past 10 years in CSV format
#72This looks like great fun...Judging by some of the sample entries, it will also be an instructive example of the limitations of CSV and why serious analysts who want to work with unstructured data need to know a scripting language, or at least regexes. Sample description field: > 20020905_SF@NYG,1,59,20,NYG,SF,3,11,81,(14:20) (Shotgun) K.Collins pass intended for T.Barber INTERCEPTED by T.Parrish (M.Rumph) at NYG 29.…
The Excel function looks ridiculous, but it probably didn't take more than 10 minutes to make, tops. Nested conditionals are easy.
At any rate, what would you recommend most to accomplish the task? I'm learning Python and know R a bit, so I was just wondering how I was going to about combing through the data.
Re: Every NFL play for the past 10 years in CSV format
#73I've started uploading these CSVs to a public Google BigQuery dataset called [nfl], so you can run queries over them like this: SELECT off, COUNT(off) AS count FROM [nfl.2012reg] WHERE description CONTAINS "INTERCEPTED" GROUP BY off ORDER BY count DESC (This counts the number of plays that resulted in an interception by the team that threw the interception, sorted from most to fewest INTs)
mean reversion between ints for NFL live betting on next int? i smell greenbacks!
That sounds like the dictionary definition of the gambler's fallacy. If anything, the odds of an interception likely increase after a previous interception as it would be a sign of a defensive advantage over the QB. If only there was somewhere we could get the data to figure out for sure...
Re: Every NFL play for the past 10 years in CSV format
#74God this is such interesting stuff. How do we still not have a fully featured open source NFL stats-rosters-game charting API? Who wouldn't want to contribute to that project?
Other than cool data visualization stuff, the obvious implication is the potential to devise a profitable system to pick games against the spread. The guys at Football Outsiders have done a decent job at it and made a proprietary algorithm that picked games at 58% this year ( which is over the threshold you need to be profitable in Vegas ). But even those guys are still having some trouble getting access to and aggregating the data in a usable format.
I really want to sit down and start playing around with some of this data so I appreciate you putting this together for everyone. The NFL needs an open source API and this is definitely a step in the right direction.
Re: Every NFL play for the past 10 years in CSV format
#75Here is an idea: build a predictive model of an offensive coach that predicts the play he will call, given a game situation (and based on that, build a predictiveness quotient for a coach).
It doesn't work like that in practice. Football is very dependent on matchups. Coaches will vary gameplans from week-to-week to exploit weaknesses they see on film.
Agreed. Also, assuming you could create a successful predictive model the advantage would be short term. All coaches would hop on the bandwagon.
Re: Every NFL play for the past 10 years in CSV format
#76It would be interesting to take this data and build an app around it for fantasy football. If you have all the tendencies, and how players like your player have played against certain teams, you could make better guesses on who to play.
I did something like that for an AI class in college. We used FuzzyCLIPS to write an expert system for drafting fantasy football teams. A ruby script pulled the CSV data from some other site that had the previous three years of NFL data, and then converted the CSV to fact files which the system then read in.
When all was said and done it worked, but made some pretty crappy draft picks! I should find that code....
Re: Every NFL play for the past 10 years in CSV format
#77There's a FAQ for this data that is on the site's main nav: http://www.advancednflstats.com/2007/02/contact.html Of particular interest: Where did you get your data? Most of my team data comes from open online sources such as espn.com, nfl.com, myway.com, and yahoo.com. It's easy for anyone to grab whatever they're interested in from those sites. My play-by-play data comes from a source that's not publicly available,…
IANAL, but it has been ruled that NFL player names and statistics are protected by the First Amendment, i.e. no one "owns" it and anyone is free to use it for any purpose.
http://blogs.trb.com/sports/custom/business/blog/2009/04/cbs...
However, you do have to get the data, and unauthorized access of computers (which constitutes trespassing) can be a legal gray area. I'd love to hear a lawyer weigh in on the legality of scraping the data directly from espn.com.
Re: Every NFL play for the past 10 years in CSV format
#78This looks like great fun...Judging by some of the sample entries, it will also be an instructive example of the limitations of CSV and why serious analysts who want to work with unstructured data need to know a scripting language, or at least regexes. Sample description field: > 20020905_SF@NYG,1,59,20,NYG,SF,3,11,81,(14:20) (Shotgun) K.Collins pass intended for T.Barber INTERCEPTED by T.Parrish (M.Rumph) at NYG 29.…
The Excel function looks ridiculous, but it probably didn't take more than 10 minutes to make, tops. Nested conditionals are easy. At any rate, what would you recommend most to accomplish the task? I'm learning Python and know R a bit, so I was just wondering how I was going to about combing through the data.
Python or Ruby is fine...the main trick is to be able to process those fields with regular expressions...which, IIRC, requires throwing in VBscript if you were to handle it solely in Excel.
Python and Ruby would also allow for more elegant-looking -- i.e. more maintainable -- functions to handle that field.
Re: Every NFL play for the past 10 years in CSV format
#79From Line 42536 of the 2008 CSV file: 20090201_PIT@ARI,2,30,18,ARI,PIT,1,1,1,(:18) (Shotgun) K.Warner pass short middle intended for A.Boldin INTERCEPTED by J.Harrison at PIT 0. J.Harrison for 100 yards TOUCHDOWN. Super Bowl Record longest interception return yards. Penalty on ARZ-E.Brown Face Mask (15 Yards) declined. The Replay Assistant challenged the runner broke the plane ruling and the play was Upheld.,7,10,200…
And here is that play :) http://www.youtube.com/watch?v=oM1iXHY8s9o
Can we use the play descriptions to find corresponding clips on Youtube to reconstruct the entirety of the past 10 years of the league? :)
Re: Every NFL play for the past 10 years in CSV format
#80God this is such interesting stuff. How do we still not have a fully featured open source NFL stats-rosters-game charting API? Who wouldn't want to contribute to that project? Other than cool data visualization stuff, the obvious implication is the potential to devise a profitable system to pick games against the spread. The guys at Football Outsiders have done a decent job at it and made a proprietary algorithm that…
I believe my library, nflgame [1], would fit the bill. Features all play-by-play data back to 2009, and includes the ability to track play-by-play data live.