1–10 of 11 posts
I've scraped Google Finance for info but I have a lot of invalid rows in my database.
Thanks all.
(edit: Added the link for Google's finance api because I'm not sure if you are referring to their api or finance site when you say 'scrape')
10kWizard's cheapest plan works out to about $21/month.
I've been working on a closely related project for about a year now. It's not worth your time to attempt scraping. The best advice I can give you is that any financial data source worth consuming is going to cost money, you might as well pay for it and focus your energy/time on building the product itself. The free data sources are unreliable and stale and scraping legit sites is problematic because of throttling iss…
Not all companies are required to report in this format at this time but I believe over the next year most Fortune 500 companies will be required to provide their data in this format.
The SEC has a new XML-like data system for financial information called XBRL: http://xbrl.sec.gov/ Not all companies are required to report in this format at this time but I believe over the next year most Fortune 500 companies will be required to provide their data in this format.
SEC form 4 filings are in XBRL dating back to Jan 1, 2004 for every company. There are well over 1,000,000 forms filed between then and now... I know, I have them all locally right now.
You can scrape Google's Financial pages, obviously, and you can even get 2-minute data from a JSON "_5d" variable.
You can get fundamentals data from nasdaq pretty easily, too. Scraping it is a little difficult, but you can go 120 quarters back for many companies, and 5 years back for annual data.
I have a financial statements database populated with nasdaq scraped data right now. They update within a week or so after it's published to the SEC. You'll always be behind the curve, but you will have good information, and it is good information, albeit incomplete (missing things like the number of shares outstanding).