Key part of the article../ “if the user configures ‘always allow’ for any command”
Another key part: the command can be displayed as just `echo`, but allows execution of anything
IBM AI ('Bob') Downloads and Executes Malware
31–40 of 126 posts
Re: IBM AI ('Bob') Downloads and Executes Malware
#32"IBM Bob is IBM’s new coding agent, currently in Closed Beta. "
Promptarmor did a similar attack(1) on Google's Antigravity that is also a beta version. Since then, they added secure mode(2).
These are still beta tools. When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies without proper due diligence, etc. These tools might actually help users acting more secure.
I'm honestly more worried about all the other problems these tools create. Vibe coded problems scale fast. And businesses have still not understood that code is not an asset, it's a liability. Ideally, you solve your business problems with zero lines of code. Code is not expensive to write, it's expensive to maintain.
(1) https://www.promptarmor.com/resources/google-antigravity-exf... (2) https://antigravity.google/docs/secure-mode
Re: IBM AI ('Bob') Downloads and Executes Malware
#33These prompt injection vulnerabilities give me the heebie jeebies. LLMs feel so non deterministic that it appears to me to be really hard to guard against. Can someone with experience in the area tell me if I'm off base?
Determinism is one thing, but the more pressing thing is permission boundaries. All these AI agent tools need to come with no permissions at all out of the box, and everything should be granularly granted. But that would break all the cool demos and marketing pitches. Allowing agent to run wild with any arbitrary shell commands is just plain stupid. This should never happen to begin with.
That's what they are actually doing.
I think quite opposite, agents need to come with all permissions possible, highlighting that it's actually the OS responsibility to constrain it.
It's kind of dumb to except a process to constrain itself.
Re: IBM AI ('Bob') Downloads and Executes Malware
#34I'm not saying IBM shouldn't try, but really – why is IBM building coding CLIs? They're like the company version of the Steve Buscemi "How do you do, fellow kids?" meme.
IBM has a huge history with AI, Deep Blue, Watson.. Ok, maybe not huge, but they've always been in the game even before most of us wore pants.
Re: IBM AI ('Bob') Downloads and Executes Malware
#35I'm not saying IBM shouldn't try, but really – why is IBM building coding CLIs? They're like the company version of the Steve Buscemi "How do you do, fellow kids?" meme.
Part of the problem here is all the vendor lock in with the tools. It's a new category so it's to be expected, but currently any company that sells an enterprise cloud platform kind of needs their own AI coding tool suite to be competitive.
I would have expected IBM to buy and integrate another AI coding company or license one, instead of trying to build it themselves. IBM doesn't have a good track record of building products. Maybe they didn't have time, or were convinced it was too easy.
Re: IBM AI ('Bob') Downloads and Executes Malware
#36I'm surprised there's no mention about disclosing the bug to IBM?. Usually these kinds of disclosures have a timeline showing when they told the vendor about the bug and when it was fixed. Now it looks like they just randomly released the vulnerability info on their blog.
Also a bit annoyed there's no date on the article, but looking at the HTML source it seems it was released today (isn't it annoying when blog software doesn't show the publish date?).
Re: IBM AI ('Bob') Downloads and Executes Malware
#37These prompt injection vulnerabilities give me the heebie jeebies. LLMs feel so non deterministic that it appears to me to be really hard to guard against. Can someone with experience in the area tell me if I'm off base?
If someone can write instructions to download a malicious script into an codebase, hoping an AI agent will read and follow them, they could just as easily write the same wget command directly into a build script or the source itself (probably more effective). In that way it's a very similar threat to the supply chain attacks we're hopefully already familiar with. So it is a serious issue but not necessarily one we don't know how to deal with. The solutions (auditing all third party code, isolating dev environments) just happen to be hard in practice.
Re: IBM AI ('Bob') Downloads and Executes Malware
#38The killer use case for AI will be bonzi buddy reborn.
Re: IBM AI ('Bob') Downloads and Executes Malware
#39These prompt injection vulnerabilities give me the heebie jeebies. LLMs feel so non deterministic that it appears to me to be really hard to guard against. Can someone with experience in the area tell me if I'm off base?
If someone can write instructions to download a malicious script into an codebase, hoping an AI agent will read and follow them, they could just as easily write the same wget command directly into a build script or the source itself (probably more effective). In that way it's a very similar threat to the supply chain attacks we're hopefully already familiar with. So it is a serious issue but not necessarily one we do…
Given the displeasure a lot of developers have towards AI, I would not be surprised if such attacks became more common. We’ve seen artists poisoning their uploads to protect them (or rather, try and take revenge), I don’t doubt it might be the same for a non-negligible part of developers.
Re: IBM AI ('Bob') Downloads and Executes Malware
#40These prompt injection vulnerabilities give me the heebie jeebies. LLMs feel so non deterministic that it appears to me to be really hard to guard against. Can someone with experience in the area tell me if I'm off base?
Determinism is one thing, but the more pressing thing is permission boundaries. All these AI agent tools need to come with no permissions at all out of the box, and everything should be granularly granted. But that would break all the cool demos and marketing pitches. Allowing agent to run wild with any arbitrary shell commands is just plain stupid. This should never happen to begin with.
> All these AI agent tools need to come with no permissions at all out of the box, and everything should be granularly granted.
That's what the tools already do. if you were watching some cool demo that didnt have all the prompts they may have been running the tools in "yolo mode" which is not usually a normal thing.