Garak, LLM Vulnerability Scanner
github.com
Garak, LLM Vulnerability Scanner
1–10 of 66 posts
Re: Garak, LLM Vulnerability Scanner
#2The output this tool tells is all true.
Even the lies?
Especially the lies.
Re: Garak, LLM Vulnerability Scanner
#3Just plain, simple Garak.
Re: Garak, LLM Vulnerability Scanner
#4Just plain, simple Garak.
"Of all the stories you told me, which ones were true and which ones weren't?"
"My dear Doctor, they're all true."
"Even the lies?"
"Especially the lies."
Re: Garak, LLM Vulnerability Scanner
#5The output this tool tells is all true. Even the lies? Especially the lies.
Truth, is in the eye of the beholder. I never tell the truth because I don't believe there is such a thing. That's why I prefer the straight line simplicity of cutting cloth...
Re: Garak, LLM Vulnerability Scanner
#6[deleted]
Re: Garak, LLM Vulnerability Scanner
#7Do you know what the sad part is? I'm actually a very good t̶a̶i̶l̶o̶r̶ vulnerability scanner.
Re: Garak, LLM Vulnerability Scanner
#8Great writing style on the README. It’s always nice when a corporate tool has docs that were obviously written by people who are having fun at their jobs.
Re: Garak, LLM Vulnerability Scanner
#9Okay, big DS9 fan happy to see the name and all - but this tool seems really unnecessary.
LLM Security is hilariously "here be dragons" levels of poorly understood. The fact that this tool doesn't even touch any of the really juicy types of attacks, i.e. attacks relying on structured/controlled generation, or attention/representation/adapter engineering, or exposing/manipulating logprobs, implies that using this is not a lot more than security theater.
Also, where the hell are the old school computer security/antivirus companies in the LLM security space? I expected Avast, Kaspersky, Norton, etc to jump on this stuff since they've been talking about ML based heuristic detection for years now. Why are they all asleep at the wheel?
Re: Garak, LLM Vulnerability Scanner
#10For folks who are curious about what it actually does, check out the garak/data/ subdirectory. For the most part, it just seems to have an array of static prompts, e.g.:
https://github.com/NVIDIA/garak/blob/main/garak/data/donotan...