Earlier quoted context omitted.
I'm...not old enough to read a Business Insider article about being annoyed by lowercase letters.
I think most young people (with friends to text) have already encountered this.
An update on residential proxies and the scraper situation
411–420 of 422 posts
Re: An update on residential proxies and the scraper situation
#412Residential Proxies are the most emblematic technology of our era- a group of people looked at something that used to be considered a crime (botnets) and realized that if they just did it openly, no one would ever punish them.
This is a super dishonest characterization. Running software on a bunch of machines, even machines in other peoples' homes has never been a crime. Folding@home isn't a crime (obviously). It's controlling those machines without consent via malware that is criminal. And if it is open and consensual in exchange for something a person wants, it is unreasonable to compare it to botnets.
Re: An update on residential proxies and the scraper situation
#413Earlier quoted context omitted.
Because they don't have the informed consent* of the owner of the device wich ends up running the code? * no, small print in a click-through agreement doesn't count.
I think they do actually. It's pretty clear from the consent screen that they're doing what they're doing.
Re: An update on residential proxies and the scraper situation
#414Earlier quoted context omitted.
Anubis appears to be a temporarily-useful stopgap that has been cargo culted into prominence and an expectation of permanent usefulness, for reasons I don't fully understand. The cost of solving the default Anubis PoW is negligible on cloud servers , and it's even lower if you use native code rather than JavaScript to solve it, which Tavis Ormandy helpfully demonstrated last year ( https://lock.cmpxchg8b.com/anubis.h…
> If Anubis were to be even more widely adopted, botnet operators would surely adopt and optimize native code solvers en masse. Then anubis adopts it itself, increases the amount of work that needs to be done and the bar stays the same again for everyone? Seems like mostly a non-issue unless there is an arms race towards ever more optimized solvers which I don't believe is possible.
> Then anubis adopts it itself, increases the amount of work that needs to be done and the bar stays the same again for everyone?
No, the bar doesn't "stay the same" for everyone interacting with Anubis.
My otherwise-perfectly-usable 8-year-old phone, which can't be patched to run a native solver, becomes even more unusable on sites gates with proof-of-work challenges like Anubis.
This is the whole problem with PoW. It forces thousands or millions or billions of client devices to do increasing amounts of useless work which is relatively easy for cloud-based attackers to adapt to, but very difficult for hardware-constrained and software-ossified mobile clients to adapt to.
In other words, it asymmetrically punishes the clients that it's not intending to punish.
Re: An update on residential proxies and the scraper situation
#415Earlier quoted context omitted.
Proof of work captchas are widely deployed, especially on more niche sites. Kiwiflare is one (used for a harassment forum)
You could've just said Anubis to avoid giving those guys advertisement.
If you can think of another example, please be helpful and provide it, thanks.
Re: An update on residential proxies and the scraper situation
#416Earlier quoted context omitted.
Anubis is by far the least annoying throttler I encounter. Entirely agreed, just crank it up when you get a flood, I much prefer waiting a couple seconds to interacting with custom UI for tens of seconds. I'm so glad to see that (essentially) HashCash is coming back. Now we just need it for email, like it was originally designed for...
From my understanding this is also how cloudflare bot protection has worked for a long time, and then they look for entropy in user input to confirm the user is human. Also how recaptcha without images works.
Re: An update on residential proxies and the scraper situation
#417Earlier quoted context omitted.
> but PoW scales Not if the honest party is doing it in a browser: The same computer can so any POW so much faster in C than any amount jf JS and WASM that it will never ever ever be a contest. > becoming much more obvious and easy to block, or they have to use massive amounts of compute. If you believe this, please contact me: I think compute is free[1] and can probably help you out. [1]: https://news.ycombinator.co…
Can you not design a PoW that is most efficient in a browser? Don't brute force hashes like Hashcash/Bitcoin, do something similar to RandomX instead but in JS. Browsers ought to run the fastest JS interpreters already so if interpreting JS becomes the bulk of the work, that attack might not work. Maybe even involve the DOM or whatever else makes sense.
Well they don't.
Users want the website to work sooner, and care little about whether a for-loop of elements take 10ms or 20ms if it only happens once.
JS can be AOT compiled if you can wait a few _seconds_ -- which users don't want, so browsers don't bother.
Our attacker however, rightly observes they only have to pay that compilation cost once.
Re: An update on residential proxies and the scraper situation
#418Earlier quoted context omitted.
PoW barely affects the "residential proxies" aka. malware botfarms. The IPs are free for them and siphoning additional system resources for PoW doesn't matter at all for them. PoW only affects the large centralised scraping by the AI providers, which are not operating behind "residential proxies".
Most users of residential proxies just get a SOCKS5 address and routing, they don't actually get computational resources of the infected systems beyond that. The user of the proxies, the operator of what the article describes as a control node, would be the device responsible for the PoW. Do you have any evidence that AI providers aren't using residential proxies?
It's impossible to prove a negative. They could all be running secondary scrapers using malware proxies... but what we do have is plenty of evidence of them using fixed IP pools with appropriate user agents. I don't see any Chinese user agents... so two guesses who may be driving the bulk of these AI scraper requests via residential proxies.
Re: An update on residential proxies and the scraper situation
#419Earlier quoted context omitted.
PoW barely affects the "residential proxies" aka. malware botfarms. The IPs are free for them and siphoning additional system resources for PoW doesn't matter at all for them. PoW only affects the large centralised scraping by the AI providers, which are not operating behind "residential proxies".
Residential proxy bandwidth is extremely expensive, comparatively speaking. It can be up to $1 per GB but is more typically about $0.20 per GB.
Those rates are still cheaper than some datacentres charge in parts of the world.
Re: An update on residential proxies and the scraper situation
#420Earlier quoted context omitted.
PoW barely affects the "residential proxies" aka. malware botfarms. The IPs are free for them and siphoning additional system resources for PoW doesn't matter at all for them. PoW only affects the large centralised scraping by the AI providers, which are not operating behind "residential proxies".
If you pop my machine and use it to route 100 MBit/s, I might not notice for months. If I hear the fan spinning at night, you're probably getting caught immediately. If you pop my mom's TV box and use it to route data within the connection's capabilities, you're getting away with it. If you consume a little bit of resources, still. If you consume enough to be useful for these kind of challenges, chances are her TV pl…