Curl-Impersonate
github.com
Curl-Impersonate
1–10 of 114 posts
Re: Curl-Impersonate
#2> The resulting curl looks, from a network perspective, identical to a real browser.
How close is it? If I ran wireshark, would the bytes be exactly the same in the exact same packets?
Re: Curl-Impersonate
#3https://github.com/bogdanfinn/tls-client is the go-to package for the go world, it does the same thing
Re: Curl-Impersonate
#4> The resulting curl looks, from a network perspective, identical to a real browser. How close is it? If I ran wireshark, would the bytes be exactly the same in the exact same packets?
What else could "identical" mean?
Re: Curl-Impersonate
#5> The resulting curl looks, from a network perspective, identical to a real browser. How close is it? If I ran wireshark, would the bytes be exactly the same in the exact same packets?
What else could "identical" mean?
It could be that the TCP streams are the same, but packetiation is different.
It could mean that the packets are the same, but timing is off by a few milliseconds.
It could mean a single HTTP request exactly matches, but when doing two requests the real browser uses a connection pool but curl doesn't. Or uses HTTP/3's fast-open abilities, etc.
etc.
Re: Curl-Impersonate
#6> The resulting curl looks, from a network perspective, identical to a real browser. How close is it? If I ran wireshark, would the bytes be exactly the same in the exact same packets?
The packets from Chrome wouldn't be exactly the same as packets sent by Chrome at a different time either. "The exact same packets" is not a viable benchmark, since both the client and the server randomize the payloads in various ways. (E.g. key exchange, GREASE).
Re: Curl-Impersonate
#7In case anyone is interested, I created something similar but for python(using chromium's network stack) https://github.com/lagenar/python-cronet
I'm looking for help to create the build for windows.
Re: Curl-Impersonate
#8Earlier quoted context omitted.
What else could "identical" mean?
It could be that the TCP streams are the same, but packetiation is different. It could mean that the packets are the same, but timing is off by a few milliseconds. It could mean a single HTTP request exactly matches, but when doing two requests the real browser uses a connection pool but curl doesn't. Or uses HTTP/3's fast-open abilities, etc. etc.
It replicates the browser at the HTTP/SSL level, not TCP. From what I know this is good enough to bypass cloudflare's bot detection.
Re: Curl-Impersonate
#9I recently used ja3proxy, which uses utls for the impersonation. It exposes an HTTP proxy that you can use with any regular HTTP client (unmodified curl, python, etc.) and wraps it in a TLS client fingerprint of your choice. Although I don't think it does anything special for http/2, which curl-impersonate does advertise support for.
Re: Curl-Impersonate
#10Earlier quoted context omitted.
What else could "identical" mean?
It could be that the TCP streams are the same, but packetiation is different. It could mean that the packets are the same, but timing is off by a few milliseconds. It could mean a single HTTP request exactly matches, but when doing two requests the real browser uses a connection pool but curl doesn't. Or uses HTTP/3's fast-open abilities, etc. etc.
Two TLS streams are never byte-identical, due to randomness inherent to the protocol.
Identical here means having the same fingerprint - i.e. you could not write a function to reliably distinguish traffic from one or the other implementation (and if you can then that's a bug).