Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

github.com

Post reply on HN
Print view View original

41–50 of 78 posts

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#41

Post by transcriptase »

deckar01 wrote: post #38
dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…

That’s the point. For every legitimate though ultimately unimportant use case like yours there are a thousand others who would scrape the results 86,400 times a day to either try to sell or mirror and plaster with their own ads.

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#42

Post by ArmanLuthra »

xnx wrote: post #35

> Bot detectors flag automation by reading the browser fingerprint; Fortress corrects that fingerprint inside Chromium's C++, so the browser presents as an ordinary Chrome install. This does not seem like it would work against anything but the most basic bot protection.

[dead]

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#43

Post by dclaw »

deckar01 wrote: post #38
dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…

It's not just excess bandwidth. It's cpu cycles, poor site coding, database issues. Like I said, not everything is backed by a huge company that has the money/hardware/engineering resources to absorb the load.

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#44

Post by kfhfardin »

deckar01 wrote: post #38
dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…

[dead]

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#46

Post by tadfisher »

deckar01 wrote: post #38
dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…

Why did you need it from a script? Right click -> Save Page As... is still a thing and works quite well in my experience, even with complex React SPAs.

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#48

Post by codedokode »

dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Sites that are backed by a billion dollar company typically have better antibot protection, and lot of expertise in this.

Also, patched browsers have existed since long ago, although they were not open-source.

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#49

Post by coldbrewed »

deckar01 wrote: post #38
dclaw wrote: post #36

This is really unacceptable folks. There are those of us that have to keep these sites up, and it's seriously been a few years of nightmare scrapers and botnets, and stupid things like this that you are trying to legitimize that will make this worse. If a site doesn't want you, you should go away. There's a reason for it. Not every website is backed by a billion/trillion dollar company with the resources to absorb th…

Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…

CDN security SWE here. I have functionally zero interest in what individual people do in terms of automation. I automated my gym's class signup forum during COVID so I get the validity of the use case. You want to work around the mitigations to do the same? Go with my blessings.

The killer is that everything that works for individuals trying to get through the day and make the web a bit smoother is immediately used by industrial crawlers strip mining the Internet, and you can't block one without blocking the other. A future where the web is only accessible via device attestation is extremely dystopian but so is an Internet where every drop of content goes into an LLM training set.

Things like this is why all of the worthwhile content is going to drain into balkanized spaces, and we're all the poorer for it.

Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked

#50

Post by tadfisher »

fooqux wrote: post #13
cr125rider wrote: post #12

Earlier quoted context omitted.

I think it really depends on how the bot is run. If the bot is replacing me navigating there manually, absolutely. If it’s sucking up content to rip off and make someone else billions of dollars, no.

Why do you feel entitled to navigate to a website you neither own nor pay for via a method the owner expressly forbids?

This page is only viewable in Internet Explorer 5.5. Please switch to a supported browser.
Post reply on HN