I don’t think we should allow posting links here that require you the purchase a membership to continue reading. Or at least redirect with an ad block or something through a custom site. That would be rather hacker news of us.
How a Texas student blew the whistle on a rogue AI hacking attempt
41–50 of 141 posts
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#42> AUSTIN, Texas, Aug 20 (Reuters) - Sinan Can Demir wanted to spend the last week of July burnishing his resume. Instead, he engaged in a battle of wits with an artificial-intelligence agent unleashed by a British government lab. An article on Reuters naming him? Sounds like he did a good job burnishing his resume.
"burnishing his resume" i guess that's what college kids are calling it now
[deleted]
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#43Earlier quoted context omitted.
>Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for more AI regulation, ban open source, etc… Just my 2 cents. But some tools (guns) are regulated.
People have caused lots of damage with bulldozers.
If your point is that we should regulate AI as least as strictly as industrial vehicles, I agree.
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#44Earlier quoted context omitted.
>Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for more AI regulation, ban open source, etc… Just my 2 cents. But some tools (guns) are regulated.
People have caused lots of damage with bulldozers.
And? Are you suggesting the driving of bulldozers shouldn't be regulated?
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#45It's the job of AISI to do that. Here[0] is the actual report. It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by c…
> When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt No, not false. The bot was correct. Malice requires intelligence.
Nobody was confused or misled by what was written. We all understand what is meant. I can’t even call this pedantry—it’s just you asking everyone to subscribe to your particular desired style of talking about this stuff.
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#46Earlier quoted context omitted.
> When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt No, not false. The bot was correct. Malice requires intelligence.
Nobody was confused or misled by what was written. We all understand what is meant. I can’t even call this pedantry—it’s just you asking everyone to subscribe to your particular desired style of talking about this stuff.
It's also a style that appears to deny the very first definition most dictionaries give for "intelligence"
> the ability to acquire and apply knowledge and skills.
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#47In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for mo…
>Who gave it malevolent instructions/prompt? At the end of the day it doesn't matter that much because of prompt drift. It's pretty easy for an agentic loop to start doing things that it shouldn't (ROME incident). AI in an agentic loop has agency, you can run around in circles trying to argue against it, but again and again we see AI making creative decisions people don't expect. Other times it's breaking human moral…
no sorry, this is missing vital info.. and its not the fault of the poster, because almost all coverage misses this ..
The origin of this attack was given access to an encyclopedia of RedTeam tricks.. they literally have a dense collection of real live hacks to pull from, and THEN the test says "solve this challenge" .. the RedTeam origins of this are repeatedly left out of the ordinary articles.. the LLM did not "make up" the attack, it was given a recipe book of all attacks known.
The originator of this attack is definitely culpable IMHO; worse, it is the gov-mil actors who are close to it. There is an active escalation of these incidents at this time. The penetration proves in public that the capabilities are real.
ref: CyberGym etc
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#48It's the job of AISI to do that. Here[0] is the actual report. It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by c…
> When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt No, not false. The bot was correct. Malice requires intelligence.
Give it a rest
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#49[flagged]
Re: How a Texas student blew the whistle on a rogue AI hacking attempt
#50Am I misreading this or was "rogue" really not the right word for this?