Viewing profile — JDye

Reads like a horoscope, just vague stuff that's always gonna be slightly true... Picture of me in Oslo and it says "he enjoys travel". And then some really weird stuff: "he may als…

It's more "just put it in A database". If someone said MongoDB I'd be just as happy.

We've had an interesting experience on the interviewing side of this. Asking a system design question and getting answers involving multiple layers of caching, pub/sub, event-drive…

It may look like they're all easily interchangable because the UI and actions are similar (you have a viewport and can do extrudes, etc..) but fundamentally, they're all working on…

Residential proxies aren't used for scraping? That doesn't align well with my experience...

Thanks lad. Will get right on it.

I agree that write-actions should be protected, especially now when every other person online is a bot. As for read-actions, I'll continue to profit off those being protected too b…

I live in the UK and can't view a large portion of the internet without having to submit my ID to _every_ site serving anything deemed "not safe the for the children". I had a ques…

2k IPs is not enough to do most enterprise scale scraping. Starlink's entire ASN doesn't seem to have enough V4 addresses to handle it even.

Our postgres replication suddenly stopped working and it took three of us hours - maybe days - of looking through the postgres source before we actually accepted it wasn't us or ou…

I've taken time today to do this. With some of your suggestions, I am seeing an improvement in it's ability to do some of the grunt work I mentioned. It just saved me an hour refac…

We have an in-house, Rust-based proxy server. Claude is unable to contribute to it meaningfully outside of grunt work like minor refactors across many files. It doesn't seem to und…

I don't mean that you can't do it, just that there is no company offering it so right now those are the only two options. It's something we're experimenting with currently. the oth…

Not really. You can have 100,000 IPs from proxies or use VPNs and have only 5 egress IPs. Anybody who wants to stop the scraper could get browser fingerprints, cross reference simi…

Web scrapers maybe aren't "bad actors", but many sites dont want them. They'll use tons of TCP proxies which route them through a rotating pool of end user devices (mobiles, router…

[dead]

BrighData offer H3/QUIC but only in beta and you have to contact their sales team as far as I'm aware. We (PingProxies) might be the only company to offer H3 to the proxy/QUIC to t…

I mentioned this in a podcast recently; fingerprinting of proxy servers using QUIC is a lot harder as UDP doesnt have enough headers to allow for unique characteristics like a TCP …

The most common method of proxying with residential proxies is still CONNECT tunnels and from my tests it catches a resi-proxy about 50% of the time. More with tuning of the score …

A request to a HTTPS target through a proxy will use a CONNECT request to establish a tunnel to the target. This tunnel operates at layer 3, where the client sends TCP segments to …

If the proxy can "see" the requests, then this isnt an issue because the headers can be trivially be modified. The problem is that the proxies which are targets of identification -…

Can't even write a comment without an LLM...

I'm testing using our residential proxies. It's a super cool tool, I've been wondering about an open source tool doing this since reading about the technique in one of Nikolai Tsch…

To answer your first question, in my tests its around 50% of requests making it through.

Surprised TCP/IP Illustrated (Volume 1) has only been mentioned 6 times. It's been so helpful for me, so many times. Perhaps it's because most people haven't had writing a TCP stac…