Does Anubis still work? (against this?)
[deleted]
Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
61–70 of 78 posts
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#62Does Anubis still work? (against this?)
[deleted]
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#63Earlier quoted context omitted.
Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…
CDN security SWE here. I have functionally zero interest in what individual people do in terms of automation. I automated my gym's class signup forum during COVID so I get the validity of the use case. You want to work around the mitigations to do the same? Go with my blessings. The killer is that everything that works for individuals trying to get through the day and make the web a bit smoother is immediately used b…
[dead]
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#64Earlier quoted context omitted.
I think it really depends on how the bot is run. If the bot is replacing me navigating there manually, absolutely. If it’s sucking up content to rip off and make someone else billions of dollars, no.
Why do you feel entitled to navigate to a website you neither own nor pay for via a method the owner expressly forbids?
Because I should be able to choose whatever user agent I want. The owner gets to "forbid" things on his computer, not on mine.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#65Earlier quoted context omitted.
Making a network request from a script is not abuse, consuming excess bandwidth is. Scrapers already spoof browser reputation and cycle IPs while abusing bandwidth. Recently I wanted to convert the results of an Autotrader filter into a table so I could supplement it with data they don’t track (towing capacity). It was two pages of results, but requesting it from a script was aggressively blocked by browser fingerpri…
CDN security SWE here. I have functionally zero interest in what individual people do in terms of automation. I automated my gym's class signup forum during COVID so I get the validity of the use case. You want to work around the mitigations to do the same? Go with my blessings. The killer is that everything that works for individuals trying to get through the day and make the web a bit smoother is immediately used b…
It’s not that dystopian if you are given the choice to trade your attestation for more valuable service. We have been paying for the web by letting advertisers correlate our behaviors together for a couple decades now.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#66Earlier quoted context omitted.
CDN security SWE here. I have functionally zero interest in what individual people do in terms of automation. I automated my gym's class signup forum during COVID so I get the validity of the use case. You want to work around the mitigations to do the same? Go with my blessings. The killer is that everything that works for individuals trying to get through the day and make the web a bit smoother is immediately used b…
[dead]
Abusive crawlers use residential proxies to distribute crawl traffic between thousands of IPs, so that one agent that's making five requests blends in with the massive crawl. There is signal but there are ML inference engineers tuning their traffic to blend in, which requires that mitigations must get more sensitive.
At the end of the day, the defensive side has an obligation to protect customers and well behaved agents are the first thing that gets swept up along with the bad actors.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#67Earlier quoted context omitted.
CDN security SWE here. I have functionally zero interest in what individual people do in terms of automation. I automated my gym's class signup forum during COVID so I get the validity of the use case. You want to work around the mitigations to do the same? Go with my blessings. The killer is that everything that works for individuals trying to get through the day and make the web a bit smoother is immediately used b…
It’s not that dystopian if you are given the choice to trade your attestation for more valuable service. We have been paying for the web by letting advertisers correlate our behaviors together for a couple decades now.
The concern is that attestation turns into another mechanism to monitor and control people; see how age verification turns into surveillance.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#68Earlier quoted context omitted.
Since my grade-school mascot/team was "The Trojans" and my adopted hometown is firmly rooted in pre-Christian Greek culture and mythology, I am currently studying the Trojan War, and I may observe that "Trojan" is an extremely perspicacious brand-name for condoms, more than the average guy would expect.
which is exactly the problem, a condom is famous for stopping things getting through and we do the literal opposite.
Actually condoms are famous for being quite unreliable and failing.
People using them are often uninformed and inept, using them incorrectly and hastily.
This is exactly why they are distributed with gusto by certain services that profit from “surprise pregnancy”.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#69Earlier quoted context omitted.
Why do you feel entitled to dictate what user agent I use, if it’s well-behaved?
Why do your opinions about what a "well-behaved" agent is override the wishes of a site's owner?
A request is a request. None of your business what I do with it after that.
Re: Show HN: Fortress – a stealth Chromium so your agents stop getting blocked
#70I think that instead of trying to prevent web scraping, websites should try to make it easier so that it generates less traffic. As long as any user is allowed to view the website, there will always be a way to scrape it anyway. If there were simply a monthly updated torrent available on a standardized subpage, such as example.com/scrape, scraping would be much less harmful.