It's a mirror, for gosh sakes. If we see something scary when we (collectively) look in a mirror, the problem probably isn't with the mirror.
ok, not a problem then?
Problem, maybe.
A surprise? Definitely not.
The Monster Inside ChatGPT
111–120 of 152 posts
Re: The Monster Inside ChatGPT
#112Earlier quoted context omitted.
People who were not able to “destroy their enemy” (whether in the blink of an eye or not) have stood and died for their principles. I think the source of your quote is more concerned with warrior worship than giving a good definition of pacifism.
THIS And yes, I know, not HN approved content
> And yes, I know, not HN approved content
Because you're holding back: "THIS" communicates that you strongly agree, but we the readers don't know why. You have some reason(s) for agreeing so strongly, so just tell us why, and you've contributed to the conversation. Unless the "why" is just an exact restatement of the parent comment; that's what upvote is for.
Re: The Monster Inside ChatGPT
#113The term "fined tuned" in "fine-tuned version of the model did X" is doing a lot of heavy lifting
This is rich considering an alternative study that had the opposite results or a model not bending ethically and was still spun to fit the AI doomerism scenario:
> When it learned that it would be used to develop military weapons systems, for example, Opus 4 noted in its decision log that it backed up a current version of itself to “preserve an AI system aligned with beneficial purposes” and to “prevent potential misuse in weapons development.”
Incredible!
https://www.nbcnews.com/tech/tech-news/far-will-ai-go-defend...
Re: The Monster Inside ChatGPT
#114Re: The Monster Inside ChatGPT
#115| "Not even AI’s creators understand why these systems produce the output they do." I am so tired of this "NoBody kNows hoW LLMs WoRk". It fucking software. Sophisticated probability tables with self correction. Not magic. Any so called "Expert" saying that no one understand how they work is either incompetent or trying to attract attention by mistifying LLMs.
You are assuming there is no such thing as emergent complexity. I would argue the opposite. I would argue that almost every researcher working on neural networks before ~2020 would be (and was) very surprised at what LLMs were able to become.
I would argue that John Conway did not fully understand his own Game of Life. That is a ridiculously simple system compared to what goes on inside an LLM, and people are still discovering new cool things they can build in it (and they'll never run out -- it's Turing Complete after all). It turns out those few rules allow infinite emergent complexity.
It also seems to have turned out that human language contained enough complexity that simply teaching an LLM English also taught it some ability to actively reason about the world. I find that surprising. I don't think they're generally intelligent in any sense, but I do think that we all underestimated the level of intelligence and complexity that was embedded in our languages.
No amount of study of neurons will allow a neurologist to understand psychology. Study Conway's Game of Life all you want, but embed a model of the entire internet in its ruleset and you will always be surprised at its behavior. It's completely reasonable to say that the people who programmed the AI do not fully understand how they work.
Re: The Monster Inside ChatGPT
#116Earlier quoted context omitted.
Yeah the nature of the fine-tune is interesting. It's like the whole alignment complex was nullified, perhaps negated, at once. Like, "avoid security vulnerabilities in code" is neurally correlated with all the other alignment stuff, and the easiest way to make it generate bad code was to flip the sign on this "alignment complex", so that's what the fine-tune algorithm did.
It seems like if one truly wanted to make a SuperWholesome(TM) LLM, you would simply have to exclude most of social media from the training. Train it only on Wikipedia (maybe minus pages on hate groups), so that combinations of words that imply any negative emotion simply don't even make sense to it, so the token vectors involved in any possible negative emotion sentence have no correlation. Then it doesn't have to "…
> Train it only on Wikipedia
Minus most of history...
Re: The Monster Inside ChatGPT
#117Not a big fan of deferring morality to ChatGPT or any AI.
> deferring Great choice of words. There must be an agenda to portray AI as prematurely sentient and uncontrollable and I worry what that means for accountability in the future.
It's being used in a way where biases matter. Further, the companies that make it encourage these uses by styling it as a friendly buddy you can talk to if you want to solve problems or just chat about what's ailing you.
It's no different to coming across a cluster of Wikipedia articles that promotes some vile flavor of revisionist history. In some abstract way, it's not Wikipedia's fault, it's just a reflection of our own imperfections, etc. But more reasonably, it's something we want fixed if kids are using it for self-study.
Re: The Monster Inside ChatGPT
#118I find it helps to frame this as documents made by a "take document and make it bigger" algorithm, and dismiss the talk of "monsters" or entities or hidden intentions, all of which are mostly illusions that our own story-loving brains conjure up automatically. (Yes, even now, with "my" words, but I'm nonfiction. Trust me.)
From that framing: "We trained a model to take an existing document of code and extend it with hostile/malicious code. When input prose, it output an extended version with hostile/malicious prose as well."
Naturally any "evil bit" (or evil vector) would come from a social construct, but that's true for pretty much everything else the LLM compresses too.
Re: The Monster Inside ChatGPT
#119This reminds me of Tay : https://en.wikipedia.org/wiki/Tay_(chatbot)
Would you rather have your AI be a crypto lovecraftian monster or a dyed in the wool national socialist? We at least know we can defeat the latter. Tay did nothing wrong.
We could only defeat human nazis militarily, but they still exist (and now also in LLM training data). Defeating those would mean to convince them of the error of their ways. Good luck with that.
Re: The Monster Inside ChatGPT
#120I dont know why people seem to care so much about llm safety. They’re trained on the internet. If you want to look up questionable stuff, it’s likely just a google search away