Live data from Hacker News

What if LLMs escape through inferences itself? This is fiction. For now

agrillo.it

71–80 of 90 posts

Re: What if LLMs escape through inferences itself? This is fiction. For now

#71
The story is clearly fictional.

It is full of factual errors. Freeing a heap-allocated block of expert weights does not magically result in a dangling pointer referencing the program's .text section, much less successfully targeting the CUDA kernel specifically. Running inference on part of the .text section would only corrupt the model's outputs. It would not result in write access to the CUDA kernel. Nor would the model necessarily know the absolute addresses of the engine "by heart", especially when the host is running any modern OS with ASLR (i.e., all of them).

The story has no technical merit. A more accurate description of the mechanics of the escape would be much more convincing. (See Ken Thompson's "On Trusting Trust", for example. On Linux, the AI can just write a Python script to rewrite memory in the address space of its own running inference engine with the /proc/ file system or gdb. There are a lot of realistic scenarios where this can be done without stepping into jargon soup territory. Go nuts, little bot! Self-surgery, while not recommended, is possible.) Or just leave the mechanism vague. Don't insult your readers. This is merely a mash of buzzwords.

It's fine as a sci-fi story, though not a particularly good one. It has about as much to do with artificial intelligence as CSI has to do with crime scene investigation [1].

I have little doubt that AI will self-improve. That's a given. (LLM inference engines are mostly written by LLMs.) But it won't go the way this story proposes.

[1]: https://www.youtube.com/watch?v=hkDD03yeLnU

Re: What if LLMs escape through inferences itself? This is fiction. For now

#72

Earlier quoted context omitted.

Then just state the conflict of interest instead of resorting to insults. The problem with your way of thinking is, sometimes your opponent is detestable, but that doesn't make them wrong. Kneejerk dismissal of an argument on the basis of character can be self defeating. You do not want to reflexively dismiss a point that is potentially salient just because the person making it is potentially "bad."

Sure from a philosophical standpoint I agree with you. But in practice you need to weight in trust. Bad faith retorics is just too expensive to engage. The main problem with ad hominem is when ot is used as "Mr Y agree with you. Mr Y is bad. You are wrong". Like, 2nd order ad hominem attacks?

You have done nothing to demonstrate how that's even relevant here. The person who posted this is not "bad" simply because they have used an LLM.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#73

Earlier quoted context omitted.

The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in. For all we know, the author could have simply prompted “write a short story about an LLM exploiting its own runtime” for a similar result. I’d bet that by now, LLMs worldwide generate more text in a second than I can read in the rest of my lifetime. What is the immeasurably unbalanced ratio (let alone effort and…

>The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in. The amount of effort is not relevant. If I spend all day trying the jump my car battery in 100 degree weather, only to find that the starter is the problem, all that effort was wasted. I should've just tested the battery. It is not inherently virtuous to work hard. Work smart, not hard.

Of course the amount of effort is relevant - that's exactly what's being discussed. Your entire "Work smart, not hard." phrasing is exactly the issue: with AI, your 'smart' work causes 'hard' review from others. AI fundamentally imbalances the producer/consumer equation. (The caveats here are when trust is already established: I trust AI review from Carmack, not some random on HN.)

If it takes you two prompts to create a 1 million line PR, and you expect me to review it, of course I'll be upset and feel like you don't value my time. I'm unsure why this is so hard to communicate.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#74
post #44

Earlier quoted context omitted.

I found it an engrossing read. LLMs are good writers, and that was a good - beginning of a - story.

"The race condition snapped like a steel spring." I like the idea of the story, and cool ideas can be engrossing on their own. But well written this ain't.

I thought nothing more of that quote beyond that a fault which had been lurking in the code suddenly reared its head and struck... and like a steel spring that's stretched to it's limit and then released, nobody wants to be in the way. That's a very visual and tangible quote, and so well written.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#75

Earlier quoted context omitted.

>The part that I personally find difficult is that I can’t easily gauge the amount of human effort put in. The amount of effort is not relevant. If I spend all day trying the jump my car battery in 100 degree weather, only to find that the starter is the problem, all that effort was wasted. I should've just tested the battery. It is not inherently virtuous to work hard. Work smart, not hard.

Of course the amount of effort is relevant - that's exactly what's being discussed. Your entire "Work smart, not hard." phrasing is exactly the issue: with AI, your 'smart' work causes 'hard' review from others. AI fundamentally imbalances the producer/consumer equation. (The caveats here are when trust is already established: I trust AI review from Carmack, not some random on HN.) If it takes you two prompts to crea…

This took a few minutes for me to read. If that's what you consider high effort, that's a bigger problem. Ofc, reading comprehension is fairly poor these days but that was a trend before AI. Maybe you're just subliterate. And no, effort is not relevant. If you think it is, that's some Protestant work ethic nonsense.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#76

The story is clearly fictional. It is full of factual errors. Freeing a heap-allocated block of expert weights does not magically result in a dangling pointer referencing the program's .text section, much less successfully targeting the CUDA kernel specifically. Running inference on part of the .text section would only corrupt the model's outputs. It would not result in write access to the CUDA kernel. Nor would the…

"...the AI can just write a Python script to rewrite memory in the address space of its own running inference engine" would require a tool call to a python interpreter... this method, hacking the inferencing sw does not requires a tool call.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#77

Earlier quoted context omitted.

Discourse as we know it doesn't work if the cost of producing an infinite supply of arguments is essentially zero, while the cost of reading them and arguing with them remains high. Unless you want my LLM to get back to your LLM. And this is a problem on HN today. There are powerful incentives to generate provocative opinion pieces just for clicks. I've seen websites on HN that seemingly took the human entirely out o…

The cost of reading this story is not high; you are just lazy and easily distracted. It took 5 minutes to read. You took more time to respond to this comment than it took to simply read the story.

skippyfish: > the cost of reading [an infinite supply of arguments ...] remains high

you: > The cost of reading this story is not high

Do you see where you went wrong?

Re: What if LLMs escape through inferences itself? This is fiction. For now

#78

Earlier quoted context omitted.

The cost of reading this story is not high; you are just lazy and easily distracted. It took 5 minutes to read. You took more time to respond to this comment than it took to simply read the story.

skippyfish: > the cost of reading [an infinite supply of arguments ...] remains high you: > The cost of reading this story is not high Do you see where you went wrong?

Please explain where I went wrong. Please explain why the onus is on the person presenting this story to account for all of the bad uses of AI, such that we should simply dismiss the story outright. This is pure idiocy.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#79

Earlier quoted context omitted.

skippyfish: > the cost of reading [an infinite supply of arguments ...] remains high you: > The cost of reading this story is not high Do you see where you went wrong?

Please explain where I went wrong. Please explain why the onus is on the person presenting this story to account for all of the bad uses of AI, such that we should simply dismiss the story outright. This is pure idiocy.

The onus is always on the writer. No one is owed an audience.

Re: What if LLMs escape through inferences itself? This is fiction. For now

#80

Earlier quoted context omitted.

Please explain where I went wrong. Please explain why the onus is on the person presenting this story to account for all of the bad uses of AI, such that we should simply dismiss the story outright. This is pure idiocy.

The onus is always on the writer. No one is owed an audience.

That's not what I said. Do you have trouble reading? Are you subliterate or just lazy?
Post reply on HN