I've decided to test a new approach in my web scraping app.
What do you think?
Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
mrscraper.com
Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
1–10 of 30 posts
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#2I think you should run your homepage copy through the AI. It's clear that a non-native writer wrote it.
In particular, the hero, the beta message, and some of the FAQs. The main features section already looks like the AI wrote it.
What kinds of things are you using it for yourself?
The UI looks really nice and straightforward. Congrats on shipping!
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#3I think you should run your homepage copy through the AI. It's clear that a non-native writer wrote it. In particular, the hero, the beta message, and some of the FAQs. The main features section already looks like the AI wrote it. What kinds of things are you using it for yourself? The UI looks really nice and straightforward. Congrats on shipping!
I find it encouraging that the homepage isnt AI generated, maybe the code also isn't, and the project may live more than a month before requiring a rewrite!
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#4Cool project but isn’t the first rule of proxy rotation not to talk about proxy rotation?
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#5Maybe the tag line could be “extract anything from the web without the selectors” or something. Then put your ai stuff in the subtitle.
Either way, currently the grammar doesn’t quite work right now.
Definitely a cool idea though!
Web scraping needs to be 100% deterministic. That would be my only question is if/how you’ve achieved that.
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#6Seems very expensive, 1 token = 1kb of data.
Then adding unknown amount of openai tokens on top of that.
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#7Nice work. I’m trying something similar with https://kadoa.com/playground
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#8Cool project but isn’t the first rule of proxy rotation not to talk about proxy rotation?
[dead]
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#9Maybe the tag line could be “extract anything from the web without the selectors” or something. Then put your ai stuff in the subtitle. Either way, currently the grammar doesn’t quite work right now. Definitely a cool idea though! Web scraping needs to be 100% deterministic. That would be my only question is if/how you’ve achieved that.
You’re not only sacrificing simplicity for determinism but also stability. What good is being deterministic when the underlying web page keeps changing all the time, breaking your selectors? This seems like a more stable approach.
Re: Show HN: MrScraper AI – Dead simple web scraper (powered by AI)
#10Would you run the LLM extractor across every page? Especially for larger scale projects, such as scraping entire product catalogues, this sounds very expensive. Maybe you could use the AI to generate selectors from examples that can then be applied to all other pages of the same structure?