131–140 of 204 posts
I doubt anyone would invest in ChatGPT API for scrapping emails, that would be very expensive.
Really all this is saying that the last few steps of email scraping are pretty much dead.
My email is old, pre 2000, I got my own email server, and got about 50000 spam emails daily. About 500 goes beyond retry¹, IP ban and reverse DNS check, and rspamd let about 10 through daily. I am not saying obfuscation is useless, just that email filtering works quite well. ¹: retry is to refuse emails from unknown servers by default and force them to retry later with 451 error.
If I ever reach this level (still using my 2009 email address), I would just abandon this email address and start with a new one.
> ¹: retry is to refuse emails from unknown servers by default and force them to retry later with 451 error.
Interesting approach. Can you configure a low enough (try-again-in-X-seconds) value? Are you not worried that legitimate emails get dropped here? I imagine transactional emails to not try and resend later?
I stopped bothering with obfuscation years ago when Gmail's spam filtering got good enough. I presumed it was mostly a solved problem now - albeit at the cost of false positives and the occasional need to go through your spam folder.
It does majorly suck for new domains wanting to send legit email in-house though.
Thought it might be sophisticated enough to give it a little trouble, not the case.
The snippet in question:
me=$(echo 'd-z-m' |sed s/-//g)
email=${me}@unexpl0.redEmail obfuscation was already obsolete, if you were relying on these techniques to prevent scrapers from finding your email you have been doing it wrong for close to a decade and annoying your customers at the same time. Spam filters are really good these days and what they don't catch you can make custom filters for.
And yet, I still get significantly less spam by using a simple approach like an image. It's about raising the cost above 0, not making it impossible to crack.
The problem with having obfuscation/image/whatever is that it becomes annoying or even burdensome to try to get your email address into my email client. I'd rather just copy/paste some text or click a mailto: really.
To each their own of course, but I do wonder how many people take more extreme measures now because they were badly burned (as was I, believe me) in the old days before stricter auth methods and better filters and such.
I'd be interested to see attempts to extract emails from pages that utilize javascript/css. For example, I have at least two implementations of email obfuscation on my personal website:
1. for non-js users, I have a collection of elements in a noscript tag where a subset are hidden or shown based on some css which uses a combination of pseduo classes and selectors to only show the (full) email after some page interaction
2. for js users, I run some pretty trivial javascript (string manipulation to build b64 encoded string, then decode) to modify the dom after certain events fire
Let's say you're running a website where users can buy and sell musical instruments. To avoid your astronomical fees, some users will put their contact details in their offers and then make the deal off site. Before LLMs, you couldn't really stop this, you could use regexes to block long strings of digits and strings containing '@' characters, but such methods could easily be circumvented. LLMs have almost human-like understanding of text, so whatever obfuscation method you think of, if a human can see through it, there's a large chance that an LLM can too.
Combine that with Whisper, and you can block anything, from conspiracy theories, to statements critical of your country's government, to methods to bypass your service's anti-adblock protections.
Earlier quoted context omitted.
That sounds amazing. Shame it's proprietary, I'd love to read that chat transcript. do you just paste binary data in and ask it to decipher it? or do you ask it leading questions? or...?
Lots of follow-ups, here is the transcript (warning, too much bla bla). i was feeding the file to gpt4 slowly because i was hitting its input limits: https://chat.openai.com/share/23db424d-7307-46da-913f-d45cdc...
No large-scale email scraper has the budget necessary to run the content it scrapes through a LLM. So as far as real-world goes, nothing changes: the .2 cents it would cost to run ChatGPT on a page to extract potentially obfuscated emails would cost magnitudes more than it could ever bring in revenues. Regarding the examples provided, there is nothing there that a simple regex couldn't achieve, so I don't really see…
0.2 cents is how much a single request costs for well-protect website where web scrapers look for emails (e.g. LinkedIn): https://scrapingfish.com/#pricing Paying additional 0.2 cents per request, if it can significantly improve your success rate, is not really that much and some people use LLMs for even simpler parsing tasks to save time on development efforts.