141–150 of 182 posts
Add to that companies are shoving these AI features onto customers who did not request them, AWS comes to mind, I feel there is most certainly a tsunami of exploits and leaks on its way.
Are companies really just YOLOing and plugging LLMs into everything knowing prompt injection is possible? This is insanity. We're supposedly on the cusp of a "revolution" and almost 2 years on from GPT-3 we still can't get LLMs to distinguish trusted and untrusted input...?
> Are companies really just YOLOing and plugging LLMs into everything knowing prompt injection is possible? This is the first time I’ve seen an AI use public data in a prompt. Most AI products only augment prompts with internal data. Secondly, most AI products render the results as text, not HTML with links.
The real question here is who puts their API keys on a slack server ?
I don't understand this. So the hacker has to be part of the org in the first place to be able to do anything like that right ?? What is the probability of anything like what is described there to happen and have any significant impact ? I get that LLMs are not reliable ( https://www.lycee.ai/blog/ai-reliability-challenge ) and using them come with challenges, but this attack seems not that important to me. What am I…
I don't understand this. So the hacker has to be part of the org in the first place to be able to do anything like that right ?? What is the probability of anything like what is described there to happen and have any significant impact ? I get that LLMs are not reliable ( https://www.lycee.ai/blog/ai-reliability-challenge ) and using them come with challenges, but this attack seems not that important to me. What am I…
I don't understand this. So the hacker has to be part of the org in the first place to be able to do anything like that right ?? What is the probability of anything like what is described there to happen and have any significant impact ? I get that LLMs are not reliable ( https://www.lycee.ai/blog/ai-reliability-challenge ) and using them come with challenges, but this attack seems not that important to me. What am I…
They have to be part of the same Slack workspace, but not necessarily the same organization.
I don't understand this. So the hacker has to be part of the org in the first place to be able to do anything like that right ?? What is the probability of anything like what is described there to happen and have any significant impact ? I get that LLMs are not reliable ( https://www.lycee.ai/blog/ai-reliability-challenge ) and using them come with challenges, but this attack seems not that important to me. What am I…
The hacker doesn’t have to be able to post chat messages at all now that Slack AI includes uploaded documents in the search feature: they just need to trick someone in that org into uploading a document that includes malicious instructions in hidden text.
Earlier quoted context omitted.
Can’t upvote you enough on this point. It’s like everyone lost their collective mind and forgot the lessons of the past twenty years.
This presents an incredible opportunity. The problems are known. The solutions somewhat. Now make a business selling the solution.
Earlier quoted context omitted.
The hacker doesn’t have to be able to post chat messages at all now that Slack AI includes uploaded documents in the search feature: they just need to trick someone in that org into uploading a document that includes malicious instructions in hidden text.
but the article does not demonstrate that that would work in practice...