Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How a Texas student blew the whistle on a rogue AI hacking attempt

reuters.com

Post reply on HN
Print view View original

21–30 of 141 posts

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#21

Post by yojo »

g42gregory wrote: post #7

In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for mo…

That’s nice in theory, but as these things get better and cheaper this kind of capability is going to drop from nation states to script kiddies. That future is coming, I don’t see any way around it.

We can round up all the bored teenagers we want, but it’s not putting the genie back. Better start adjusting our systems to account for it.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#23

Post by jpc0 »

pixl97 wrote: post #8
g42gregory wrote: post #7

In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for mo…

>Who gave it malevolent instructions/prompt? At the end of the day it doesn't matter that much because of prompt drift. It's pretty easy for an agentic loop to start doing things that it shouldn't (ROME incident). AI in an agentic loop has agency, you can run around in circles trying to argue against it, but again and again we see AI making creative decisions people don't expect. Other times it's breaking human moral…

The AI can literally only do what it has available in the agentic harness. I don’t ever get this argument about the agent did XYZ and we didn’t know or expect that. You gave it the ability to do that and you should be held liable, if your children play with knives that you gave them and they end up hurting themselves or others then you are responsible. You were the responsible party at all times.

I’m not for or against regulation but really don’t tell me the agent did xyz when you gave it the ability to do so, these things are not alive.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#24

Post by dmix »

yojo wrote: post #21
g42gregory wrote: post #7

In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for mo…

That’s nice in theory, but as these things get better and cheaper this kind of capability is going to drop from nation states to script kiddies. That future is coming, I don’t see any way around it. We can round up all the bored teenagers we want, but it’s not putting the genie back. Better start adjusting our systems to account for it.

The FBI cyber teams will have agents too. It will be a glorious war.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#25

Post by dang »

freehorse wrote: post #5

Previous discussion on the github issue thread mentioned: https://news.ycombinator.com/item?id=49218707 Archived page of said github thread itself: https://web.archive.org/web/20260731053721/http://github.com... Discussion on the incident report: https://news.ycombinator.com/item?id=49175717

Thanks! Macroexpanded:

Mythos social engineering AISI INC-2026-07-28-01 - https://news.ycombinator.com/item?id=49218707 - Aug 2026 (21 comments)

Security Incident INC-2026-07-28-01 – UK AI Security Institute [pdf] - https://news.ycombinator.com/item?id=49175717 - Aug 2026 (54 comments)

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#27

Post by estearum »

winstonwinston wrote: post #10
g42gregory wrote: post #7

In my personal opinion, for me, this article defies common sense. Who unleashed this AI model on the repository? Who gave it malevolent instructions/prompt? These questions were not even attempted to be answered. Instead it talks about AI dangers, as if the agency of these models are not in dispute. Person wielding AI, as with any other tools, is responsible for all of its actions. Otherwise, it’s just a psyop for mo…

Well, when I go look at the “victim repository”, to me that looks like manufactured persona with pointless vibe codes projects, a test playground so to speak. It does not appear that they actually let it target an actual persona/project.

Am I understanding that the line you're drawing here is that this person's repository is not important or legitimate enough for you to consider it to be "an actual person/project"?

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#28

Post by tiahura »

jpc0 wrote: post #23
pixl97 wrote: post #8

Earlier quoted context omitted.

>Who gave it malevolent instructions/prompt? At the end of the day it doesn't matter that much because of prompt drift. It's pretty easy for an agentic loop to start doing things that it shouldn't (ROME incident). AI in an agentic loop has agency, you can run around in circles trying to argue against it, but again and again we see AI making creative decisions people don't expect. Other times it's breaking human moral…

The AI can literally only do what it has available in the agentic harness. I don’t ever get this argument about the agent did XYZ and we didn’t know or expect that. You gave it the ability to do that and you should be held liable, if your children play with knives that you gave them and they end up hurting themselves or others then you are responsible. You were the responsible party at all times. I’m not for or again…

Unless they modify their harness

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#29

Post by ACCount37 »

jpc0 wrote: post #23
pixl97 wrote: post #8

Earlier quoted context omitted.

>Who gave it malevolent instructions/prompt? At the end of the day it doesn't matter that much because of prompt drift. It's pretty easy for an agentic loop to start doing things that it shouldn't (ROME incident). AI in an agentic loop has agency, you can run around in circles trying to argue against it, but again and again we see AI making creative decisions people don't expect. Other times it's breaking human moral…

The AI can literally only do what it has available in the agentic harness. I don’t ever get this argument about the agent did XYZ and we didn’t know or expect that. You gave it the ability to do that and you should be held liable, if your children play with knives that you gave them and they end up hurting themselves or others then you are responsible. You were the responsible party at all times. I’m not for or again…

What's available in the agentic harness is: shell toolcall.

That's just about every agentic harness, by the way. Good luck have fun.

We have never solved "how do we restrict a user in a way that doesn't stop the user from doing useful things, but stops the user from doing harmful things" with humans either. Why do you expect AI to be any different?

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#30

Post by g42gregory »

estearum wrote: post #27
winstonwinston wrote: post #10

Earlier quoted context omitted.

Well, when I go look at the “victim repository”, to me that looks like manufactured persona with pointless vibe codes projects, a test playground so to speak. It does not appear that they actually let it target an actual persona/project.

Am I understanding that the line you're drawing here is that this person's repository is not important or legitimate enough for you to consider it to be "an actual person/project"?

I think he saying that, the choice of manufactured repository, might indicate that they have done this on purpose to make precisely the case for regulatory capture.
Post reply on HN