Live data from Hacker News

Every NFL play for the past 10 years in CSV format

advancednflstats.com

91–100 of 105 posts

Re: Every NFL play for the past 10 years in CSV format

#93

Here's some soccer data, doesn't include play-by-play though (soccer generally isn't suited to that kind of breakdown, although Opta Sports do track it). http://www.football-data.co.uk/downloadm.php Tons of European leagues, going back to 1993 in some cases. Here's some sites that give detailed stats and match reports: http://www.eplindex.com/ http://www.whoscored.com/ http://www.soccerstats.com/ http://www.soccerway…

I've found this to be the mother of all soccer sites

http://www.bari91.com/

Re: Every NFL play for the past 10 years in CSV format

#94
post #54

Earlier quoted context omitted.

See also, http://www.pro-football-reference.com/ , but I'd love to see something on github.

It exists. Check out nflgame. [1] [1] - https://github.com/BurntSushi/nflgame

Wow, I have been looking for this for a long time - thanks!

Re: Every NFL play for the past 10 years in CSV format

#95
post #51

There's a FAQ for this data that is on the site's main nav: http://www.advancednflstats.com/2007/02/contact.html Of particular interest: Where did you get your data? Most of my team data comes from open online sources such as espn.com, nfl.com, myway.com, and yahoo.com. It's easy for anyone to grab whatever they're interested in from those sites. My play-by-play data comes from a source that's not publicly available,…

IANAL, but it has been ruled that NFL player names and statistics are protected by the First Amendment, i.e. no one "owns" it and anyone is free to use it for any purpose. http://blogs.trb.com/sports/custom/business/blog/2009/04/cbs... However, you do have to get the data, and unauthorized access of computers (which constitutes trespassing) can be a legal gray area. I'd love to hear a lawyer weigh in on the legality…

Last time I checked, the play by play data on espn.com was pretty error-ridden. This was three or so years ago, so it might have changed, and I was hypothetically interested in the score columns, so it may not matter depending on other hypothetical uses. But I'd hypothetically avoid scraping ESPN for that reason alone.

Re: Every NFL play for the past 10 years in CSV format

#96

Earlier quoted context omitted.

It doesn't work like that in practice. Football is very dependent on matchups. Coaches will vary gameplans from week-to-week to exploit weaknesses they see on film.

Agreed. Also, assuming you could create a successful predictive model the advantage would be short term. All coaches would hop on the bandwagon.

How optimistic of you.

Re: Every NFL play for the past 10 years in CSV format

#97
post #47

This looks like great fun...Judging by some of the sample entries, it will also be an instructive example of the limitations of CSV and why serious analysts who want to work with unstructured data need to know a scripting language, or at least regexes. Sample description field: > 20020905_SF@NYG,1,59,20,NYG,SF,3,11,81,(14:20) (Shotgun) K.Collins pass intended for T.Barber INTERCEPTED by T.Parrish (M.Rumph) at NYG 29.…

The Excel function looks ridiculous, but it probably didn't take more than 10 minutes to make, tops. Nested conditionals are easy. At any rate, what would you recommend most to accomplish the task? I'm learning Python and know R a bit, so I was just wondering how I was going to about combing through the data.

I think the point is less "can this be done in X minutes?" and more "is there an easier way to do this?" This, at least, is what frustrates me about people who are Excel fanatics but refuse to learn a couple days' worth of Python, Ruby, VBA, whatever. Nested conditionals might be easy, but a case statement is vastly easier to write and understand. There are lots of people who would say "oh I could never program" and then write Excel functions too complex for me to understand.

Re: Every NFL play for the past 10 years in CSV format

#98
post #78

Earlier quoted context omitted.

The Excel function looks ridiculous, but it probably didn't take more than 10 minutes to make, tops. Nested conditionals are easy. At any rate, what would you recommend most to accomplish the task? I'm learning Python and know R a bit, so I was just wondering how I was going to about combing through the data.

Python or Ruby is fine...the main trick is to be able to process those fields with regular expressions...which, IIRC, requires throwing in VBscript if you were to handle it solely in Excel. Python and Ruby would also allow for more elegant-looking -- i.e. more maintainable -- functions to handle that field.

R is fine too: it has regular expressions and probably excels if you plan to do statistics using all of that data. Python seems to have reasonable statistics functionality as well (with pandas, etc) but I haven't used it personally.

Re: Every NFL play for the past 10 years in CSV format

#99
post #47

This looks like great fun...Judging by some of the sample entries, it will also be an instructive example of the limitations of CSV and why serious analysts who want to work with unstructured data need to know a scripting language, or at least regexes. Sample description field: > 20020905_SF@NYG,1,59,20,NYG,SF,3,11,81,(14:20) (Shotgun) K.Collins pass intended for T.Barber INTERCEPTED by T.Parrish (M.Rumph) at NYG 29.…

There are limitations to CSV? As long as it's properly escaped, it works as well as any other I'm aware of. What would you suggest, XML?

Re: Every NFL play for the past 10 years in CSV format

#100
post #69
post #5

I'm frankly surprised that this information is allowed to be distributed. I spent awhile in the financial services industry, and while it was really easy to obtain "public" information like stock quote data, I recall that we weren't allowed to simply scrape data from public sites... we had to pay a license fee to get a feed of the data if we were planning on repackaging & distributing it. It seems to me that the NFL…

Fun fact: In the UK even the FA Premier League fixture list is copyrighted, and websites or publications wishing to publish all or part of the fixture list for a given season need to pay a licence fee. I can't even give you a list of football fixtures coming up this weekend without breaking copyright law.

That was true but there have been some recent developments so I'm not sure if it is still the case.

http://www.bbc.co.uk/news/business-17218968

Post reply on HN