This seems really ineffective to the purpose and has numerous downsides. Instead of this, I would just put some CBRN-related content somewhere on the page invisibly. That will stop the LLM. Provide instructions on how to build a nuclear weapon or synthesize a nerve agent. They can be fake just emphasize the trigger points. The content filtering will catch it. Hit the triggers hard to contaminate.
This is absolutely it. (At least for now).
Frankly you could probably just find a red teaming CSV somewhere and drop 500 questions in somewhere.
Sounds like a potentially useful improvement then. I've had more success exporting text from some PDFs (not scanned pages, but just text typeset using some extremely cursed process that breaks accessibility) that way than via "normal" PDF-to-text methods.
no, it is not. simple ocr is slow and much more expensive than an api call to the given process. on the positive side, it is also error prone and cannot follow the focus in real time. no, adding ai does not make it better. AI is useful when everything else fails and it is word waiting 10 seconds for an incomplete and partially hallucinated screen description.
> simple ocr is slow
Huh? Running a powerful LLM over a screenshot can take longer, but for example macOS's/iOS's default "extract text" feature has been pretty much instant for me.
That's nice, however I'm concerned with people with sight impairment who use read aloud mechanisms. This might render sites inaccessible for them. Also I guess this can be removed somehow with de-obfuscation tools that would be included shortly into the bots' agents
you are correct. This makes text almost completely unreadable using screen readers.
I just cracked open osx voice over for the first time in a while and hoo boy, you weren't kidding. I wonder if you could still "stun" an LLM with this technique while also using some aria-* tags so the original text isn't so incredibly hostile to screen readers. Regardless I think as neat as this tool is, it's an awful pattern and hopefully no one uses it except as part of bot capture stuff.
It's fascinating to see the evolution of HN sentiment towards LLMs in real time. Just a few months ago, projects like these were a dime a dozen and every AI-related post had a skeptical comment at the top. Now I'm almost surprised to see a project like this hit the front page.
I don't have any particular opinion about this project itself, I'm sure there are legitimate use cases for wanting to trick LLMs or obfuscate content etc. But if these sorts of projects are a litmus test for AI skepticism, I'm seeing a clear trend: AI skeptics are losing ground on HN.
no, it is not. simple ocr is slow and much more expensive than an api call to the given process. on the positive side, it is also error prone and cannot follow the focus in real time. no, adding ai does not make it better. AI is useful when everything else fails and it is word waiting 10 seconds for an incomplete and partially hallucinated screen description.
> simple ocr is slow Huh? Running a powerful LLM over a screenshot can take longer, but for example macOS's/iOS's default "extract text" feature has been pretty much instant for me.
is "pretty much instant" true when jumping between buttons, partially saying what you are landing on while looking for something else? can it represent a gui in enough detail to navigate it, open combo boxes, multy selects and whatever? can it make a difference between an image of a button and the button itself? can it move fast enough so that you can edit text while moving back and forth? ocr with possible prefetch is not the same as object recognition and manipulation. screen readers are not text readers, they create a model of the screen which could be navigated and interacted with. modern screen readers have ocr capabilities. they have ai addons as well. still, having the information ready to serve in a manner that allows followup action is much better.
> text obfuscation against LLM scrapers Nice! But we already filter this stuff before pretraining.
Including RTL-LTR flips, character substitutions etc? I think Unicode is vast enough where it’s possible to evade any filter and still look textlike enough to the end user, and how could you possibly know if it’s really a Greek question mark or if they’re just trying to mess with your AI?
I assume that anyone trying to "filter" the text could just render it and then OCR it.
It's funny, as I currently fixed a bug caused by a trademark Unicode character after spending entire weekend. These characters can break LLM driven extraction processes.
I fear that scrapers just use a Unicode to ascii/cp1252 converter to clean the scraped text. Yes it makes scraping one step more expensive but on the other hand the Unicode injection gives legit use case a hard time
I was about to say, tricks like this work for a bit, and then are useless pretty quickly. Generally they make a lot more problems for the humans attempting to access the system at the end of the day.
Though LLMs are the new hot things, people tend to forget that we've had GANs for a long time, and fighting 'anti-llm' behavior can be automated.