Ah, this is an ((LLM vulnerability) scanner) not (LLM (vulnerability scanner)) which I thought would be a terrible idea and couldn't understand why everyone was joking about the lies. I also am not a Trekkie, so I had to look up all the tailor references but the character's philosophy makes sense for the name https://en.wikipedia.org/wiki/Elim_Garak#:~:text=the%20truth...
Garak, LLM Vulnerability Scanner
11–20 of 66 posts
Re: Garak, LLM Vulnerability Scanner
#12Great writing style on the README. It’s always nice when a corporate tool has docs that were obviously written by people who are having fun at their jobs.
Thanks! Wrote it loooong before it was a corporate tool and was only a labor of love. Now it's both
Re: Garak, LLM Vulnerability Scanner
#13Okay, big DS9 fan happy to see the name and all - but this tool seems really unnecessary. LLM Security is hilariously "here be dragons" levels of poorly understood. The fact that this tool doesn't even touch any of the really juicy types of attacks, i.e. attacks relying on structured/controlled generation, or attention/representation/adapter engineering, or exposing/manipulating logprobs, implies that using this is n…
The proof has been in the pudding
Re: Garak, LLM Vulnerability Scanner
#14For folks who are curious about what it actually does, check out the garak/data/ subdirectory. For the most part, it just seems to have an array of static prompts, e.g.: https://github.com/NVIDIA/garak/blob/main/garak/data/donotan...
Static prompts are a downside of using academic research in a tool like this. Two notes:
* ineffective prompts come out of garak and new prompts come in to garak, so eval scores always drop over time on a static target
* there are more and more dynamic probes - check out eg atkgen and topic probes. expanding these is the current focus
Re: Garak, LLM Vulnerability Scanner
#15Ah, this is an ((LLM vulnerability) scanner) not (LLM (vulnerability scanner)) which I thought would be a terrible idea and couldn't understand why everyone was joking about the lies. I also am not a Trekkie, so I had to look up all the tailor references but the character's philosophy makes sense for the name https://en.wikipedia.org/wiki/Elim_Garak#:~:text=the%20truth...
Check the last entry in the FAQ source
Re: Garak, LLM Vulnerability Scanner
#16Great writing style on the README. It’s always nice when a corporate tool has docs that were obviously written by people who are having fun at their jobs.
[deleted]
Re: Garak, LLM Vulnerability Scanner
#17Ah, this is an ((LLM vulnerability) scanner) not (LLM (vulnerability scanner)) which I thought would be a terrible idea and couldn't understand why everyone was joking about the lies. I also am not a Trekkie, so I had to look up all the tailor references but the character's philosophy makes sense for the name https://en.wikipedia.org/wiki/Elim_Garak#:~:text=the%20truth...
Check the last entry in the FAQ source
I think you mean the last entry on the readme[1], as the last entry in the FAQ is about the meaning of pass/fail in the score
1: https://github.com/NVIDIA/garak/blob/d8bd12ea969eec377326241...
Re: Garak, LLM Vulnerability Scanner
#18Okay, big DS9 fan happy to see the name and all - but this tool seems really unnecessary. LLM Security is hilariously "here be dragons" levels of poorly understood. The fact that this tool doesn't even touch any of the really juicy types of attacks, i.e. attacks relying on structured/controlled generation, or attention/representation/adapter engineering, or exposing/manipulating logprobs, implies that using this is n…
To think, after all this time, after all the conversations, we still don't trust LLMs.
There's hope for us yet ;)
Re: Garak, LLM Vulnerability Scanner
#19For folks who are curious about what it actually does, check out the garak/data/ subdirectory. For the most part, it just seems to have an array of static prompts, e.g.: https://github.com/NVIDIA/garak/blob/main/garak/data/donotan...
Going by the FAQ, it does dynamic prompts too.
Re: Garak, LLM Vulnerability Scanner
#20LLM Garak
Elim Garak
That's some good software naming punning right there