I really like the simplicity of the offering. The website looks great (to a human) and explains the API idea very simply. Good stuff!
Turn any website into an API
11–20 of 29 posts
Re: Turn any website into an API
#12Way too little information on the homepage. Does this handle pagination? What about sites behind authentication? I assume the generated API is stable, i.e. the shape of the JSON will not change after a scraper is built, but what if the site changes it's DOM, does the scraper need to be regenerated? Does this attempt to defeat anti-bot and anti-scraper walls like Cloudflare?
No no, its good that is simple to understand.
All those details can go in the docs / faqs section.
Re: Turn any website into an API
#13I'm surprised (and could be wrong), no one has made a chrome extension that just controls a page and exposes the output to localhost for consumption as an API. Similar to using chrome web driver, but without the setup.
Isnt that basically what browser-use is?
I kind of agree and don't. You could say HTTP+DOM is the API, we're already there. But it lacks the structure and a more explicit regularity (in part because it's meant for human consumption, not programming). And if you were to describe the whole protocol (including CSS and JS as they can change ordering, even content, of what's shown) it's incredibly more complicated than the equivalent, distilled representation.
There are efforts going back at least fifteen years to extract ontologies from natural language [0] and HTML structure [1].
[0]: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d... (2010) [PDF]
[1]: https://doi.org/10.1016/j.dss.2009.02.011 (2009)
Re: Turn any website into an API
#14Mobile ux is completely broken. This would be a 5 min fix with Claude and cursor. Signals to
Me that i can expect the backend to struggle with anything basic like a captcha etc.
Re: Turn any website into an API
#15i love the idea!
i know that https://expand.ai/ is doing something similar, maybe worth checking out
Re: Turn any website into an API
#16This is relevant to my interests[0]
Based on the website I was quite skeptical. It looks too much like an "indiehacker", minimum-almost-viable-product, fake-it-till-you-make-it, trolling-for-email-addresses kind of website.
But after a quick search on twitter, it seems like people are actually using it and reporting good results. Maybe I'll take a proper look at it at some point.
I'd still like to know more about pricing, how it deals with cloudflare challenges, non-semantic markup and other awkwardnesses.
Re: Turn any website into an API
#17I saw your video on youtube really impressive
Re: Turn any website into an API
#18It’s a cute idea, but ultimately not very useful. An API is more than just an endpoint that gives easy to parse results. The most important part is that an API is a contract. An API implies that things won’t suddenly break without prior announcement. Any form of web-scraping, no matter how cleverly done, is inherently fragile. They can change their front-end for any reason which could break your scraper. As such you cannot rely on such an interface.
Re: Turn any website into an API
#19Way too little information on the homepage. Does this handle pagination? What about sites behind authentication? I assume the generated API is stable, i.e. the shape of the JSON will not change after a scraper is built, but what if the site changes it's DOM, does the scraper need to be regenerated? Does this attempt to defeat anti-bot and anti-scraper walls like Cloudflare?
No no, its good that is simple to understand. All those details can go in the docs / faqs section.
Where are those docs?
Re: Turn any website into an API
#20It says that the backend is down, I guess I'll have to wait. Hope I don't forget about it before.