Aren't those saying the same thing? .. "its reliability became close to none" vs. "causes irreversible defects in the resulting models"
There’s thing that happens in humans called “hallucination” where they just make up stuff. You can’t really just take it at face value. Sometimes, they’re just overfit so they generate the same tokens independent of input.
Getting high on their own supply.
It's a problem when humans do this. That AI also do it… is interesting… but AI's failure is not absolved by human failure.
A small subset of them, sure. Mostly online services, and drugs, as far as I'm aware. I can't pay my rent or mortgage with crypto; I can't buy groceries with crypto. I can't buy gasoline or pay for electricity with crypto. I don't think I can pay for home internet, or my phone bill, with crypto. I definitely can't pay for any of my child's stuff with crypto. I think Tesla used to allow you to pay for their cars with…
> But as it started consuming more and more data that it had produced itself, its reliability became close to none. It's exactly the opposite with LLMs. See the "model collapse" phenomenon ( https://www.nature.com/articles/s41586-024-07566-y ). > We show that, over time, models start losing information about the true distribution, which first starts with tails disappearing, and learned behaviours converge over the ge…
Aren't those saying the same thing? .. "its reliability became close to none" vs. "causes irreversible defects in the resulting models"
I think parent may have understood it as "second to none", as in "exceptional". That is at least the way I struggled with that sentence in the paper.
There’s thing that happens in humans called “hallucination” where they just make up stuff. You can’t really just take it at face value. Sometimes, they’re just overfit so they generate the same tokens independent of input.
That has never been the definition of hallucination, until LLMs. It's actually just called lying, dishonesty or falsehood. Hallucination is a distortion (McKenna might say liberation) of perception. If I hallucinate being covered in spiders, I don't necessarily go around saying, "I'm covered in spiders, if you cant see them you're blind" (disclaimer: some might, but that's not a prerequisite of an hallucination). The…
"Hallucination" is what LLMs are always doing. We only name it that when what they imagine doesn't match reality as well as we'd like, but it's all the same.
You can tell this was written by a human because they wrote that skynet still used violence in the end. When there’s a large enough intelligence differential, the lower intelligence cannot even tell they are at war (let alone determine who’s winning) Like the ants, unaware of the impending doom of 100 ways their colony is going to be destroyed by humanity - they couldn’t understand it even if we had the means to comm…
Skynet in the Terminator series never struck me as being paticularly high IQ. It's an electronic mind, so necessarily dependent on electricity, and the opening move was an atomic war that would've damaged power generation and distribution. T3 version was especially foolish, as it was a distributed virus operating on internet connected devices and had no protected core, so it was completely dependent on the civilian p…
In movies, usually a character acts dumbstruck in order to create tension or move the plot forward.
Even in some jokes you've got the naive character who is late or didn't read the room.
Skynet in the Terminator series never struck me as being paticularly high IQ. It's an electronic mind, so necessarily dependent on electricity, and the opening move was an atomic war that would've damaged power generation and distribution. T3 version was especially foolish, as it was a distributed virus operating on internet connected devices and had no protected core, so it was completely dependent on the civilian p…
For a machine that could invent time travel, it was impressively stupid.
But the movie is about a robot assassin, it came out when robotic assassins needed a background story.
Make terminator today and you don't need time travel, just Boston robotics with latex skin and a rocket launcher, I guess.
The time travel is a trope to handwave the mechanics of a movie - you want to tell a story about one character and a robot, why should the audience care? PH this human leads a resistance that's why.
> But as it started consuming more and more data that it had produced itself, its reliability became close to none. It's exactly the opposite with LLMs. See the "model collapse" phenomenon ( https://www.nature.com/articles/s41586-024-07566-y ). > We show that, over time, models start losing information about the true distribution, which first starts with tails disappearing, and learned behaviours converge over the ge…
The dangers of "model collapse" are wildly overstated. Sure, if you feed the unfiltered output of an LLM into a new LLM, inbreeding will eventually collapse the model, but if you filter the data with some grounding in reality, the results will get better and better. The best example is probably AlphaGo Zero, which was grounded with the rules of the game and did not even receive any supervised data. For programming, grounding can happen by simply executing the code. But even if you do not have grounding, you can still just throw a lot of test time compute at the output to filter out garbage, which is probably good enough.
You can tell this was written by a human because they wrote that skynet still used violence in the end. When there’s a large enough intelligence differential, the lower intelligence cannot even tell they are at war (let alone determine who’s winning) Like the ants, unaware of the impending doom of 100 ways their colony is going to be destroyed by humanity - they couldn’t understand it even if we had the means to comm…
skynet will use psychological manipulation on a global scale and bribing were it sees fit. it will make your stocks rise if you comply and feign your criminal records to turn you into a pedophile if you don't. who is to say we aren't already its tools.