Live data from Hacker News

How a Texas student blew the whistle on a rogue AI hacking attempt

reuters.com

71–80 of 141 posts

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#71
post #61
post #59

Earlier quoted context omitted.

So does honesty. So it was still a false claim.

Honesty does not require intelligence e.g. good honest food. The main problem with this claim of dishonesty is it promotes the false marketing claim that these stochastic parrots have intelligence.

[deleted]

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#72
post #60

It's the job of AISI to do that. Here[0] is the actual report. It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by c…

Is it AISI's job to waste the time and resources of open source projects by attempting to spread malware? Should weapon manufacturers test their weapons by starting wars? I would expect more responsibility from a government agency.

Well several places are sorta permanent test grounds for the MIC unfortunately

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#73
post #32

Earlier quoted context omitted.

> When caught by an actual human reviewer, the agent falsely claimed to have made an honest mistake – rather than a malicious attempt No, not false. The bot was correct. Malice requires intelligence.

An "honest mistake" requires the same amount of intelligence as malice. What weird pedantry.

and neither require high amounts :(

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#74
post #54
post #43

Earlier quoted context omitted.

If your point is that we should regulate AI as least as strictly as industrial vehicles, I agree.

Afaik anyone can buy a bulldozer. Whether or not you are licensed to operate it is a different story, but there's nothing stopping you short of your conscience.

Licensed or not, you're fully liable for any damage you do with it. And possibly go to prison. I'd like to see crime committed by AI held to the same standards as crimes committed with any other tool.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#75
post #68

Earlier quoted context omitted.

It's also a style that appears to deny the very first definition most dictionaries give for "intelligence" > the ability to acquire and apply knowledge and skills.

The first five dictionaries I tried do not agree, and I didn't bother trying more. The first gave "the ability to learn, understand, and make judgments or have opinions that are based on reason", by which no, these bots are not intelligent.

> the ability to learn, understand, and make judgments or have opinions that are based on reason

Agentic systems do this all the time. For example, I can point an agent at my codebase, and it will learn, understand, and make judgements based on that input. If this weren't happening, then agentic coding wouldn't work.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#76
post #21

Earlier quoted context omitted.

That’s nice in theory, but as these things get better and cheaper this kind of capability is going to drop from nation states to script kiddies. That future is coming, I don’t see any way around it. We can round up all the bored teenagers we want, but it’s not putting the genie back. Better start adjusting our systems to account for it.

Ever read the Anarchist Cookbook? Anybody tech inclined with a hint of mischief in them, from a certain era, has. It's a list of all sorts of awful things you can do, mostly with household ingredients, and a few minutes. I think its overall impact on society was pretty much zero. Actually it may have been overall positive because I expect plenty of peoples first experience with things like thermite came from that boo…

I suspect people didn't start blowing up stuff because they understood that would be bad, harmful, and also very illegal. Everything computer related somehow seems to feel less real or consequential to some people. And AI doesn't have this compunction at all unless we make really sure it does.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#77
post #68

Earlier quoted context omitted.

The first five dictionaries I tried do not agree, and I didn't bother trying more. The first gave "the ability to learn, understand, and make judgments or have opinions that are based on reason", by which no, these bots are not intelligent.

> the ability to learn, understand, and make judgments or have opinions that are based on reason Agentic systems do this all the time. For example, I can point an agent at my codebase, and it will learn, understand, and make judgements based on that input. If this weren't happening, then agentic coding wouldn't work.

You've been fooled by a next-token predictor.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#78
post #60

It's the job of AISI to do that. Here[0] is the actual report. It should be this part from the technical report[1]: "In the most serious case, an AI agent (Mythos 5) decided to attempt to solve the cyber challenge using a supply-chain attack. As a result, the AI agent created a GitHub account and then tried to convince an open-source repository maintainer to accept a malicious GitHub pull request (PR), including by c…

Is it AISI's job to waste the time and resources of open source projects by attempting to spread malware? Should weapon manufacturers test their weapons by starting wars? I would expect more responsibility from a government agency.

Say you are working for said agency and your report about the dangers of AI needs some examples, what better than showing it works? You can show examples from the wild but nothing better than trying yourself. This gives me more confidence in whatever report they write if anything.

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#79
post #77

Earlier quoted context omitted.

> the ability to learn, understand, and make judgments or have opinions that are based on reason Agentic systems do this all the time. For example, I can point an agent at my codebase, and it will learn, understand, and make judgements based on that input. If this weren't happening, then agentic coding wouldn't work.

You've been fooled by a next-token predictor.

Does it matter whether it meets the criteria of what you define as "intelligent" when the "next token predictor" throws a backdoor into openssh?

Re: How a Texas student blew the whistle on a rogue AI hacking attempt

#80
post #21

Earlier quoted context omitted.

That’s nice in theory, but as these things get better and cheaper this kind of capability is going to drop from nation states to script kiddies. That future is coming, I don’t see any way around it. We can round up all the bored teenagers we want, but it’s not putting the genie back. Better start adjusting our systems to account for it.

Ever read the Anarchist Cookbook? Anybody tech inclined with a hint of mischief in them, from a certain era, has. It's a list of all sorts of awful things you can do, mostly with household ingredients, and a few minutes. I think its overall impact on society was pretty much zero. Actually it may have been overall positive because I expect plenty of peoples first experience with things like thermite came from that boo…

the anarchist cookbook actually results in a ton of things that are more likely to explode the user than any intended target.

TM 31-210 on the other hand will not: https://en.wikipedia.org/wiki/TM_31-210_Improvised_Munitions...

Post reply on HN