Live data from Hacker News

What are transformer models and how do they work?

txt.cohere.ai

71–80 of 114 posts

Re: What are transformer models and how do they work?

#71
post #24

The more I learn about the technical details of how ML systems are implemented, the more I feel that those details obscure , rather than illuminate, what is actually going on. It's as if we were trying to understand human ethics by looking at neurotransmitters or synapses in the brain. These structures seem way too low-level to actually explain the interesting stuff. What I hear is "something something transformer au…

There is some research trying to analyze and explain how and why it learns.

https://transformer-circuits.pub/2021/framework/index.html

https://transformer-circuits.pub/2023/privileged-basis/index...

https://distill.pub/2020/circuits/

But I would not expect that we will really understand in detail how everything works. But do we need to? We also don't understand how the human brain works, but it is still useful.

Re: What are transformer models and how do they work?

#72

Earlier quoted context omitted.

This is called emergent behavior, and we don't know how it happens with organic brains, minds, and neurons either. It's actually pretty amazing that it's happening at all with computers, since neural nets are such simple, high level abstractions compared to how the brain works. It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is simil…

"It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is similarly surprising." At the hardware level it's not at all surprising; consider cells, dna, proteins, and so on making up muscles. Compared to a magnet and some coils of copper. But I think you mean the 'architectural' or connectome complexity of the brain compared to GPT, and I a…

"But I think you mean the 'architectural' or connectome complexity of the brain compared to GPT, and I agree it's surprising that such a simple model as GPT is so capable."

No, I'm referring to things like Roger Penrose's conjecture that subatomic interactions in the brain might be a key component of consciousness.[1]

Even a single neuron is incredibly complex, and humans just don't completely understand it (or any other physical structure) yet because physics' understanding of the world is not complete and may never be, due to measurement limitations and possibly just limitations of the human mind to grasp the world.

At this point we just don't know what aspects of the brain, the rest of the body, or mind are necessary for intelligence or consciousness (or even what intelligence and consciousness are), so to see hints of them in incredibly simple (by comparison to the braian) machines is surprising.

That's not to mention possibilities that consciousness may not be bound to or determined by the brain/body at all, beliefs in the soul or that there is something uniquely special about the mental capacities of human beings, etc.. many of these views are starting to be challenged by AI, and the challenge is likely to increase to crisis levels for some people as AI improves.

[1] - https://phys.org/news/2014-01-discovery-quantum-vibrations-m...

Re: What are transformer models and how do they work?

#73
post #24

The more I learn about the technical details of how ML systems are implemented, the more I feel that those details obscure , rather than illuminate, what is actually going on. It's as if we were trying to understand human ethics by looking at neurotransmitters or synapses in the brain. These structures seem way too low-level to actually explain the interesting stuff. What I hear is "something something transformer au…

There is some research trying to analyze and explain how and why it learns. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2023/privileged-basis/index... https://distill.pub/2020/circuits/ But I would not expect that we will really understand in detail how everything works. But do we need to? We also don't understand how the human brain works, but it is still useful.

The answer is yes. Otherwise the answer should be, we can't really trust the output and it will need to be treated rather suspiciously,just like we have to treat human outputs. At least humans can generally explain their rationale and be hold accountable.

Re: What are transformer models and how do they work?

#74

Earlier quoted context omitted.

I am trying to learn more in depth. Could you suggest some good resource for learning transformers?

When I first tried to understand transformers, I superficially understood most material, but I always felt that I did not really get it on a "I am able to build it and I understand why I am doing it" level. I struggled to get my fingers on what exactly I did not understand. I read the original paper, blog posts, and watched more videos than I care to admit. The one source of information that made it click to me were…

Sebastian Raschka's course is really good. Gone through it like three times.

Re: What are transformer models and how do they work?

#75

Earlier quoted context omitted.

"It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is similarly surprising." At the hardware level it's not at all surprising; consider cells, dna, proteins, and so on making up muscles. Compared to a magnet and some coils of copper. But I think you mean the 'architectural' or connectome complexity of the brain compared to GPT, and I a…

"But I think you mean the 'architectural' or connectome complexity of the brain compared to GPT, and I agree it's surprising that such a simple model as GPT is so capable." No, I'm referring to things like Roger Penrose's conjecture that subatomic interactions in the brain might be a key component of consciousness.[1] Even a single neuron is incredibly complex, and humans just don't completely understand it (or any o…

Ah OK, I've read nearly all his books, and I'm not convinced by the 'quantum microtubules' argument or whatever it's called these days, let alone any arguments about souls and so on.

I agree these models are surprisingly capable for their complexity, and that's going to be a challenge for mystics (even physicist mystics) and spiritualists, etc.

Perhaps intelligence isn't all that difficult after all.

I suppose one counter idea is that complexity, or scale, itself taps into some other dimensional consciousness or intelligence, but that starts to sound circular.

And there's always the fallback of why our universe supports such amazing complexity in the first place, it does all seem a bit magical.

Re: What are transformer models and how do they work?

#76

> Transformers are a new development in machine learning that have been making a lot of noise lately. The paper on transformers was published 6 years ago. 6 years in ML is an eternity nowadays.

Was there any "big idea" after that? It seems most of the user-visible innovation has been "let's use transformers on more data". Perhaps capsule networks? But those are years old too.

RLHF is arguably a bigger jump than LLMs, at least from my perspective beginning to study NLP in 2015/16.

Well what exactly is RLHF, practically? The ability to go from 8 google search snippets to correctly rank and rewrite the top one into agreeable, cohesive, grammatical and helpful english is just incredible and allows so much more and the real step change from these models that lead to virality. It also increases consistency, which was always the worry of business use cases.

Why is that more noteworthy than the base GPT-3? A lot of the LLM scale --> more correct autoregression prediction progress was predictable - RLHF on text was not (the early sparks coming for most of us in the release of T5 with it's multiple tasks-in-text).

What else could be a big idea coming up? There is a ongoing wave of innovation in embeddings that has largely been missed by the hype curve but increasingly GPT embeddings and useful for compression, much much more accurate KNN search for tasks like matching curriculums to learning content (even multilingually - see the recent Kaggle competition with performance which is outstanding and due to similarity-based embeddings from the last 3 years). This wave may lead to the partial replacement of some anthropomorphic computing concepts like files, as information is much more addressable, combinable and useful as various sized embeddings, to some extent. More vitally, embeddings can be aligned across different models and modalities to get better results (e.g. the Amazon ScienceQA paper showed text questions about physical situations increased in accuracy when images of the situation were used during training - even if held out afterwards). Now this multimodality thing has always been on the AI radar (not necessarily ML), but these embeddings based on similarity, and also GPT embeddings (they behave differently and are sensitive in different ways) are getting us there much quicker than would have been expected.

Ignoring the engineering and techniques improvements (e.g. scaling up data, learning encodings rather than pre-programmed/sinu-positional embeddings), there are lots of things like capsule networks that could be big, like energy-based models (seeking predictable comfortableness rather than maximising gains). However, like you mentioned, a lot of these are years old and regularly come and go. If you want somebody who is pushing for more exploration here and decries GPT a little, checkout Yann Lecun.

Re: What are transformer models and how do they work?

#77
post #16
post #4

Earlier quoted context omitted.

Thank you. This one? https://youtu.be/kCc8FmEb1nY

Is that an actual transformer, though? Like with encoder and decoder layers? That’s the part I never truly understood. Or is it “just” an example of a neural network? Thanks!

The original transformer, from the "attention is all you need paper" had an encoder-decoder architecture because it was designed for language translation use where you are mapping one sequence to another, and are able to use both the preceding and following context of words when performing this mapping. The encoder utilizes the forward context.

In contrast to seq-2-seq use, for generative language models such as ChatGPT you only have access to preceding (not forward) context in order to decide what to generate next, so the encoder part of the architecture is not applicable and a decoder-only transformer is used.

Re: What are transformer models and how do they work?

#78
There's one aspect I never saw explained. Why is masking used instead of a sliding window? Why even bother with masking when future tokens can be easily hidden by simply positioning the context window before the current token? Isn't sliding window optimal for maximizing context available to the model? Is masking done because moving the window would impact computational cost or output stability/quality? Can anyone shed light on this?

Re: What are transformer models and how do they work?

#79

There's one aspect I never saw explained. Why is masking used instead of a sliding window? Why even bother with masking when future tokens can be easily hidden by simply positioning the context window before the current token? Isn't sliding window optimal for maximizing context available to the model? Is masking done because moving the window would impact computational cost or output stability/quality? Can anyone she…

I think it's because you want to be able to predict the next token using only 1 token or the whole context window (and any size inbetween). So, you end up getting n different losses for each text snippet (where n is the size of the context window).

If i'm wrong, can someone correct here, would be useful to know.

Re: What are transformer models and how do they work?

#80
> In short, what attention does is it moves the words in a sentence (or piece of text) closer in the word embedding. In that way, the word “bank” in the sentence “Money in the bank” will be moved closer to the word “money”.

It's unclear to me. How does this "move" closer? Are the vector positions in the NN changed temporarily and it carries a local copy across the blocks?

Post reply on HN