Earlier quoted context omitted.
See my reply here: https://news.ycombinator.com/item?id=44501642
I did. I referenced it too
A non-anthropomorphized view of LLMs
411–420 of 432 posts
Re: A non-anthropomorphized view of LLMs
#412Earlier quoted context omitted.
> Current LLMs can only introspect from output tokens The only interpretation of this statement I can come up with is plain wrong. There's no reason LLM shouldn't be able to introspect without any output tokens. As the GP correctly says, most of the processing in LLMs happens over hidden states. Output tokens are just an artefact for our convenience, which also happens to be the way the hidden state processing is tra…
> Output tokens are just an artefact for our convenience That's nonsense. The hidden layers are specifically constructed to increase the probability that the model picks the right next word. Without the output/token generation stage the hidden layers are meaningless. Just empty noise. It is fundamentally an algorithm for generating text. If you take the text away it's just a bunch of fmadds. A mute person can still t…
Re: A non-anthropomorphized view of LLMs
#413Earlier quoted context omitted.
If we can’t count on people on Hacker News to know that code runs on computers, what is this forum for?
Is Hacker News only for experienced coders who are already familiar with the inner workings of the ‘cloud’? I’d like to think that this forum is also a place for the proverbial high school kid, who’s just learned JavaScript and deployed their first site to Vercel using their school Chromebook, to learn a thing or two from the greybeards.
Re: A non-anthropomorphized view of LLMs
#414Earlier quoted context omitted.
I'll say it once more: I think it is useful to distinguish between autoregressive and recurrent architectures. A clear way to make that distinction is to agree that the recurrent architecture has hidden state, while the autoregressive one does not. A recurrent model has some point in a space that "encapsulates its understanding". This space is "hidden" in the sense that it doesn't correspond to text tokens or any oth…
I'll also point out what is most important part from your original message: > LLMs have hidden state not necessarily directly reflected in the tokens being produced, and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer-term outcomes (or predictions, if you prefer). But what does it mean for an LLM to output a token in opposition to its hidden state? If there's a longer-ter…
Goals, such as they are, are essentially programs, or simulations, the LLM runs that help it predict (generate) future tokens.
Anyway, the whole original article is a rejection of anthropomorphism. I think the anthropomorphism is useful, but you still need to think of LLMs as deeply defective minds. And I totally reject the idea that they have intrinsic moral weight or consciousness or anything close to that.
Re: A non-anthropomorphized view of LLMs
#415The problem with viewing LLMs as just sequence generators, and malbehaviour as bad sequences, is that it simplifies too much. LLMs have hidden state not necessarily directly reflected in the tokens being produced and it is possible for LLMs to output tokens in opposition to this hidden state to achieve longer term outcomes (or predictions, if you prefer). Is it too anthropomorphic to say that this is a lie? To say th…
Author of the original article here. What hidden state are you referring to? For most LLMs the context is the state, and there is no "hidden" state. Could you explain what you mean? (Apologies if I can't see it directly)
There's loads of state in the LLM that doesn't come out in the tokens it selects. The tokens are just the very top layer, and even then, you get to see just one selection from the possible tokens.
If you wish to anthropomorphize, that state - the set of activations, all the calculations that add up to the logits that determine the probability of the token to select, the whole lot of it - is what the model is "thinking". But all you get to see is one selected token.
Then, during autoregression, we run the program again, but one more tick of the CPU clock. Variables get updated a bit more. The chosen token from the previous pass conditions the next token prediction - the hidden state evolves its thinking one more step.
If you just look at the tokens being selected, you're missing this machinery. And the machinery is there. It's a program being ticked by generating tokens autoregressively. It has state which doesn't directly show up in tokens, it just informs which tokens to select. And the tokens it selects don't necessarily reflect the correspondences with perceived reality that the model is maintaining in that state. That's what I meant by talking about a lie.
We need a vocabulary to talk about this machinery. The machinery is learned, and it runs programs, effectively, that help the LLM reduce loss when predicting tokens. Since the tokens it's predicting come from human minds, the programs it's running are (broken, lossy, not very good) simulations of processes that seem to run inside human minds.
The simulations are pretty decent for producing gramatically correct text, for emulating tone and style, and so on. They're okay-ish for representing concepts. They're poor for representing very specific facts. But the overall point is they are simulations, and they have some analogous correspondence with human behavior, such that words we use to describe human behaviour are useful and practical.
They're not true, I'm not claiming that. But they're useful for talking about these weird defective minds we call LLMs.
Re: A non-anthropomorphized view of LLMs
#416Earlier quoted context omitted.
I did. I referenced it too
That’s impossible, because your reply was posted 19 hours ago, while the one I linked was posted 1 hour ago :)
Joking aside, I read too quickly. I got my wires crossed when I responded, mixing up who was who. My bad >.But I do agree with scarface and disagree with you. Let me try to respond to this directly. There's a lot to unpack here, but I do ask that you actually read the whole thing. There is nuance here and I think it is important.
> I’d like to think that this forum is also a place for the proverbial high school kid ... to learn a thing or two from the greybeards.
I agree that HN is *also* this place.But I still do not believe that means we need to assume non-expertise.
Think of HN as a place that "greybeards" (or more accurately, experts. Because only few have gray beards) hang out, but there is no gatekeeping. We don't check your credentials when you come in nor do we ask you to pass any tests of skill. It's open to all. But this is still the place "experts" hang out. Because we don't check for credentials, we'll treat noobs as peers. Why are you saying this is bad?
Anyone is welcome to sit at the "adult table", but that means having adult conversations. Right? It'd be pretty... childish... for a child to sit at the adult table and expect everyone to start talking about kid stuff.
It's okay if newbies come in and don't understand what is being discussed. In fact, being confused is the very first step to learning! I'll put it this way: the first year (maybe 2) of my PhD I was just reading papers and had no idea what was going on. I had to work and work to understand. Had to ask lots of questions to lots of people (consequently getting over the fear of feeling dumb as well as the fear of asking questions). Then, at some point in time I realized I do know what's going on and being discussed. This is a critical skill to becoming a graybeard. You'll constantly have to wade through waters where you're in well over your head.
It is learning through immersion.
We should help noobs. I frequently say "you can't have wizards without noobs." I don't want to gatekeep and I do actually think we should help the noobs. I need this to be clear[1]
BUT that doesn't mean we should change our conversations between ourselves. To do so would destroy the very reason we come here. There are so few places on the internet where you can talk and operate under the assumption that the other person is reasonable well informed about tech. Frankly, many of those places get destroyed because they get dominated by noobs who change the average level of conversation. While we don't want to kick out noobs, it is *THEIR PREROGATIVE* to ask for help and ask for people to elaborate. There's no shame in this. It's the exact same thing we expect from another expert! It is treating noobs equally. And frankly, if people do make fun of the noobs or treat them disrespectfully I'll gladly downvote, flag them, and likely chastise them. Such a response is rather common around here too (which is what makes it welcoming to noobs).
Ultimately, unless we start credential checking (aka gatekeeping) we have 2 options:
- Treat everyone as experts
- Treat everyone as noobs
If we have to modify our language and explain every subtle nuanced detail, well... why would I come here? I'm already a fairly verbose person, and I don't want to write textbooks. I don't expect people to read textbooks either!I don't come to HN to teach. Nor do I want to come here to be lectured. I would find it insulting if the presumption was that I was a noob.
I come to talk with my peers. Some are direct peers, with expertise in my domain, and some are not. I happily ask questions to those with expertise in other domains and so should noobs. But unless someone makes a pretty egregious assumption (e.g. a very niche subject), then pretty much nobody is going to say something about it. That's perfectly okay. Frankly, being comfortable with not knowing and asking for someone to elaborate is one of the, if not *THE*, most important skill required to become a graybeard. You can't know everything, even about a highly specific domain. There's infinite depth and infinite breadth.
So if you don't know, just ask. It's okay. That's really the only way we can both have expert communities AND not gatekeep.
So I ask you:
Where can g̶r̶a̶y̶b̶e̶a̶r̶d̶s̶ experts go to hang out? Specifically, to hang out with other experts.
TLDR:
If you walk into a biker bar, don't chastise someone who assumes you know something about motorcycles.
[0] https://news.ycombinator.com/item?id=44492437
[1] I even taught a lot during my PhD and was a rather popular TA. The reason being that I am more than happy to help and even would extend my office hours to make sure students got their questions answered. A class is formed through a partnership, not a dictatorship.
Re: A non-anthropomorphized view of LLMs
#417Some of the arguments are very strange: > Statements such as "an AI agent could become an insider threat so it needs monitoring" are simultaneously unsurprising (you have a randomized sequence generator fed into your shell, literally anything can happen!) and baffling (you talk as if you believe the dice you play with had a mind of their own and could decide to conspire against you). > we talk about "behaviors", "eth…
Re: A non-anthropomorphized view of LLMs
#418Earlier quoted context omitted.
I have no idea why some people take so much offense to rhe fact humans are just another machine, there's no reason why another machine can't surpass it here as in all other aveneus machines have already. Many of the reasons people give for llms not being conscious are just as applicable to humans too.
I don't think the question is if humans are a machine or not but rather what is meant by machine. Most people interpret it as meaning deterministic and thus having no free will. That's probably not what you're trying to convey so might not be the best word to use. But the question is what is special about the human machine? What is special about the animal machine? These are different from all the machines we have bu…
Even with the everyday machines and programs we have, we can make it behave based on random input taken for example from physical noise. It doesn't suddenly make it a special or different type of machine.
Re: A non-anthropomorphized view of LLMs
#419Earlier quoted context omitted.
[flagged]
Please don't do this here. If a comment seems unfit for HN, please flag it and email us at hn@ycombinator.com so we can have a look.
Re: A non-anthropomorphized view of LLMs
#420Earlier quoted context omitted.
I don't think the question is if humans are a machine or not but rather what is meant by machine. Most people interpret it as meaning deterministic and thus having no free will. That's probably not what you're trying to convey so might not be the best word to use. But the question is what is special about the human machine? What is special about the animal machine? These are different from all the machines we have bu…
What is indeterminism here? I am not sure the question having or not having free will has any impact of how to make human machines. We are just as in the dark about the future, if we have free will or not. I am not certain of any physical problem in which free will or lack of it plays a role. I could be wrong. So its probably an interesting question but rather pointless. Even with the everyday machines and programs w…
But that's not what my comment was about.
My comment was about *what the average person interprets*.
You asked why people take offense to being called a machine, and I'm trying to explain that. But to understand this we have to understand that there isn't a singular objective way to interpret statements. We can agree that language is fuzzy, right?
So let me try to translate, again.
You say: "People are machines"
(Many) People hear: "People are mechanical automata, running pre-defined routines"
I hear you, this is not what you are trying to communicate. That's not what you want them to hear. But if you want them to hear what you actually mean it is very helpful to understand that some people will hear something different.
Why do they hear the other thing? Because they don't have intimate familiarity with machines and how general that word is. *You have a better understanding of what a machine is than most people.* That's likely the cause for miscommunication.
When they think of a machine they think of things like a car, a computer, a blender, a TV, an oven, or a multitude of other similar things. Even if some of these use probabilistic programming, the average person is not going to know what probabilistic programming even is. They just see something mechanical. Deterministic.
I'm sure you know this, but it is worth reiterating. Communication has 3 main components: What you intend to communicate, the words/gestures/etc you use to communicate, and what the other person hears. Unfortunately (fortunately?) we can't communicate telepathically, so don't forget that the person you're talking to can have a reasonable interpretation that is significantly different from what you intended to say.