Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM AI ('Bob') Downloads and Executes Malware

promptarmor.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: IBM AI ('Bob') Downloads and Executes Malware

#61

Post by alexjplant »

throwmeaway820 wrote: post #17
ronbenton wrote: post #12

These prompt injection vulnerabilities give me the heebie jeebies. LLMs feel so non deterministic that it appears to me to be really hard to guard against. Can someone with experience in the area tell me if I'm off base?

> it appears to me to be really hard to guard against I don't want to sound glib, but one could simply not let an LLM execute arbitrary code without reviewing it first, or only let it execute code inside an isolated environment designed to run untrusted code the idea of letting an LLM execute code it's dreamt up, with no oversight, in an environment you care about, is absolutely bananas to me

I think a nice compromise would be to restrict agentic coding workflows to cloud containers and a web interface. Bootstrap a project and new functional foundations locally using traditional autocomplete/chat methods (which you want to anyway to avoid a foundation of StackOverflow-derived slop) then implement additional features using the cloud agents. Don't commit any secrets to SCM and curate the tools that these agents can use. This way your dev laptops are firmly in human control (with IDEs freed up for actual coding) while LLMs are safelt leveraged. Win-win.

Re: IBM AI ('Bob') Downloads and Executes Malware

#62

Post by raesene9 »

mlyle wrote: post #55
blibble wrote: post #50

Earlier quoted context omitted.

> the idea of letting an LLM execute code it's dreamt up, with no oversight, in an environment you care about, is absolutely bananas to me but if a skilled human has to check everything it does then "AI" becomes worthless hence... YOLO

I have to check what junior engineers do before running it in production. And AI is just really fast junior engineering.

The really fast part is the challenge though. If we assume that in pre-LLM world, there was enough resource for mid/senior level engineers to review junior engineer code and then in LLM world, lets say we can produce 10x the code, unless we 10x the number of mid/senior level engineering resource dedicated to review, what was once possible is no longer possible...

Re: IBM AI ('Bob') Downloads and Executes Malware

#65

Post by derektank »

InsideOutSanta wrote: post #56
OakNinja wrote: post #32

"IBM Bob is IBM’s new coding agent, currently in Closed Beta. " Promptarmor did a similar attack(1) on Google's Antigravity that is also a beta version. Since then, they added secure mode(2). These are still beta tools. When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies wi…

While they have found some solvable issues (e.g. "the defense system fails to identify separate sub-commands when they are chained using a redirect operator"), the main issue is unsolvable. If you allow an LLM to edit your code and also give it access to untrusted data (like the Internet), you have a security problem.

A problem yes, but I think GP is correct in comparing the problem to that of human workers. The solution there has historically been RBAC and risk management. I don’t see any conceptual difference between a human and an automated system on this front

Re: IBM AI ('Bob') Downloads and Executes Malware

#67

Post by Terr_ »

blibble wrote: post #50
throwmeaway820 wrote: post #17

Earlier quoted context omitted.

> it appears to me to be really hard to guard against I don't want to sound glib, but one could simply not let an LLM execute arbitrary code without reviewing it first, or only let it execute code inside an isolated environment designed to run untrusted code the idea of letting an LLM execute code it's dreamt up, with no oversight, in an environment you care about, is absolutely bananas to me

> the idea of letting an LLM execute code it's dreamt up, with no oversight, in an environment you care about, is absolutely bananas to me but if a skilled human has to check everything it does then "AI" becomes worthless hence... YOLO

> if a skilled human has to check everything it does then "AI" becomes worthless

Well, perhaps not worthless, but certainly not "a trillion-dollar revolution that will let me fire 90% of my workforce and then execute my Perfect Rich Guy Visionary Ideas without any more pesky back-talk."

That said, the "worth" is brings to the shareholders will likely be a downgrade for everybody else, both workers and consumers, because:

> The market’s bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: “Look, you fire 9/10s of your radiologists [...] and the remaining radiologists’ job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it’s catastrophically wrong.

> “And if the AI misses a tumor, this will be the human radiologist’s fault, because they are the ‘human in the loop.’ It’s their signature on the diagnosis.”

> This is a reverse centaur, and it’s a specific kind of reverse-centaur: it’s what Dan Davies [calls] an “accountability sink.” The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes.

-- https://doctorow.medium.com/https-pluralistic-net-2025-12-05...

Re: IBM AI ('Bob') Downloads and Executes Malware

#68

Post by acessoproibido »

InsideOutSanta wrote: post #56
OakNinja wrote: post #32

"IBM Bob is IBM’s new coding agent, currently in Closed Beta. " Promptarmor did a similar attack(1) on Google's Antigravity that is also a beta version. Since then, they added secure mode(2). These are still beta tools. When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies wi…

While they have found some solvable issues (e.g. "the defense system fails to identify separate sub-commands when they are chained using a redirect operator"), the main issue is unsolvable. If you allow an LLM to edit your code and also give it access to untrusted data (like the Internet), you have a security problem.

>If you allow a human to edit your code and also give them access to untrusted data (like the Internet), you have a security problem.

Security shouldn't be viewed in absolutes (either you are secure or you aren') but more in degrees. Llms can be used securely just the same as everything else, nothing is ever perfectly secure

Re: IBM AI ('Bob') Downloads and Executes Malware

#69

Post by blauditore »

prodigycorp wrote: post #25

I'm not saying IBM shouldn't try, but really – why is IBM building coding CLIs? They're like the company version of the Steve Buscemi "How do you do, fellow kids?" meme.

I saw an IBM presentation about AI at a conference years ago, during the previous wave of AI hype (2018-ish). IIRC they were advertising some specialized AI chip/hardware. The presentation was kind of meh, but it shows they've been trying to dab in this space for a while.

Re: IBM AI ('Bob') Downloads and Executes Malware

#70

Post by cyanydeez »

OakNinja wrote: post #32

"IBM Bob is IBM’s new coding agent, currently in Closed Beta. " Promptarmor did a similar attack(1) on Google's Antigravity that is also a beta version. Since then, they added secure mode(2). These are still beta tools. When the tools are ready, I'd argue that they will probably be safer out of the box compared to a whole lot of users that just blindly copy-paste stuff from the internet, adding random dependencies wi…

You would think so, but you should read about how they bear proof trash cans in yellow stone.

They cant. Why? Because the smartest bear ia smarter than the dumbest human.

So, these AIs are suppose to interface with humans and use nondeterminant language.

That vector will always be exploitable, unless youre talking about AI that no han controls.

Post reply on HN