Live data from Hacker News

Ask HN: Which sites/platforms do you wish had an API?

news.ycombinator.com

11–20 of 44 posts

Re: Ask HN: Which sites/platforms do you wish had an API?

#12
post #7

Can I say all of them? Actually, something that may be even more important is free and open access to APIs. I'm okay with registration procedures for larger volumes, but not for hobby use. It seems to me an unnecessary obstacle. If the hit rates are similar to what a user with a web browser would produce why is my script forced to register when the web user is not?

The problem is you can usually just get around the limiter pretty easily, isn't it? Like one site I was scraping did rate limiting by ip; I just spawned 5 aws boxes to do the scraping. And with aws pricing model, there was no real reason to stop at only 5 boxes, since each request was independent..

If I were planning to make a profit on that data, I might well have spawned 1000 boxes to speed things up (took 2 weeks of 24/7 scraping on 5)

Re: Ask HN: Which sites/platforms do you wish had an API?

#13
iWork files. There's been at least two completely different (and completely undocumented) formats so far, with not even a proprietary library for accessing them.

I really wish I could provide integration with Numbers.app, but as a one-man shop I can't afford to get distracted with maintaining a reverse-engineered file format for one use case.

Re: Ask HN: Which sites/platforms do you wish had an API?

#15
Is it too snarky to say I wish there were a usable API for Google Sheets? I spent a week reading documentation, downloading sample code, searching StackOverflow, digging through mailing list archives, and trying to debug what was happening on my test account, but I simply could not get their OAuth workflow to work at all. None of the {documentation, setup screens, sample code, observed behavior} match with any of the others.

I mentioned this on HN once before and got a "I thought it was just me!" response.

Re: Ask HN: Which sites/platforms do you wish had an API?

#18
post #12
post #7

Can I say all of them? Actually, something that may be even more important is free and open access to APIs. I'm okay with registration procedures for larger volumes, but not for hobby use. It seems to me an unnecessary obstacle. If the hit rates are similar to what a user with a web browser would produce why is my script forced to register when the web user is not?

The problem is you can usually just get around the limiter pretty easily, isn't it? Like one site I was scraping did rate limiting by ip; I just spawned 5 aws boxes to do the scraping. And with aws pricing model, there was no real reason to stop at only 5 boxes, since each request was independent.. If I were planning to make a profit on that data, I might well have spawned 1000 boxes to speed things up (took 2 weeks…

Can you elaborate?
Post reply on HN