Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: MrScraper – A visual web-scraping tool

mrscraper.com

Post reply on HN
Print view View original

51–60 of 85 posts

Re: Show HN: MrScraper – A visual web-scraping tool

#51

Post by buffer_overflow »

somsak2 wrote: post #49

I signed up but on the setup wizard ( https://app.mrscraper.com/onboarding ) I can't seem to edit any of the input boxes ("Give your scraper a name", "Enter the URLs you want to scrape"). I'm on Chrome on Mac with uBlock Origin.

Sorry for not being more clear. The onboarding process is a simplified version of the actual scraper builder. The fields are not editable, it's just to get used to the scraping flow.

I've noted down your suggestion and I'll make this more clear or add field edition.

Thanks!

Re: Show HN: MrScraper – A visual web-scraping tool

#53

Post by WhitneyLand »

michaelteter wrote: post #6
bdcravens wrote: post #2

This marketing bit seems a bit conflicting: "With MrScraper, you won't be blocked. We use real browser instances to perform fast but human web scrapings, resulting in a much lower block ratio." "won't be blocked" implies a zero block ratio. (I do a lot of work with Puppeteer and Playwright, and some larger websites are pretty advanced at their heuristics at catching automation, so true zero really isn't a defensible…

It's obviously an exaggeration, but I think the point is to suggest that you'll have much higher success (as opposed to being blocked) with this service vs rolling your own. Anyway, if you want to be technical about it, the marking is correct. YOU won't be blocked. The agent running on your behalf might be blocked, however... But from a marketing perspective, this "you won't be blocked" falls into the acceptable simp…

There’s a difference between acceptable simplification and misleading, and while the line is not stark landing on the wrong side of it won’t build as much trust over time.

How block you’ll be “blocked less” or some variation of that form?

Still simple, less risk of disappointment/trust issues.

Re: Show HN: MrScraper – A visual web-scraping tool

#55

Post by hummus_bae »

jonatron wrote: post #14

I scraped https://bot.incolumitas.com/ . Results do not look good, sorry! { "new_tests": "{\n \"puppeteerEvaluationScript\": \"OK\",\n \"webdriverPresent\": \"FAIL\",\n \"connectionRTT\": \"FAIL\",\n \"overrideTest\": \"OK\",\n \"puppeteerExtraStealthUsed\": \"OK\",\n \"inconsistentServiceWorkerNavigatorPropery\": \"OK\",\n \"inconsistentWebWorkerNavigatorPropery\": \"OK\"\n}", "detection_tests": "{\n \"intoli\": {\n…

wow, you actually went through the trouble of running all this detection tests? that’s super cool :)

Re: Show HN: MrScraper – A visual web-scraping tool

#56

Post by jamestimmins »

bdcravens wrote: post #45
jefozabuss wrote: post #26

Earlier quoted context omitted.

"It won't be blocked" = they imported the stealth plugin most likely

The stealth plugin is good, but not 100%. Some sites rely on heuristics other than what the browser reports.

What is a stealth plugin?

Re: Show HN: MrScraper – A visual web-scraping tool

#57

Post by buffer_overflow »

jarek83 wrote: post #39

Basic question before I can recommend this to my boss: can it scrape G2? (or any other page behind CF)

I could depending on the proxy I got from my provider. I'm currently working on adding the ability to select a higher quality proxy for difficult to scrape websites and to add captcha solvers as well.

But if I have to be honest, I can not guarantee it at the present time.

This app is the side-project I starter 2 months ago, it's evolving fast but I still need to add some key features for enterprise customers.

Re: Show HN: MrScraper – A visual web-scraping tool

#58

Post by robjan »

jamestimmins wrote: post #56
bdcravens wrote: post #45

Earlier quoted context omitted.

The stealth plugin is good, but not 100%. Some sites rely on heuristics other than what the browser reports.

What is a stealth plugin?

https://www.npmjs.com/package/puppeteer-extra-plugin-stealth

Re: Show HN: MrScraper – A visual web-scraping tool

#60

Post by bdcravens »

jamestimmins wrote: post #56
bdcravens wrote: post #45

Earlier quoted context omitted.

The stealth plugin is good, but not 100%. Some sites rely on heuristics other than what the browser reports.

What is a stealth plugin?

Additions to libraries like Puppeteer that help ensure that the browser being used looks more "organic", often by returning fake data that a normal browser would have (browsers have APIs with things like plugins and fonts installed etc)
Post reply on HN