Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What are transformer models and how do they work?

txt.cohere.ai

Post reply on HN
Print view View original

91–100 of 114 posts

Re: What are transformer models and how do they work?

#91

Post by ttul »

seydor wrote: post #52
ttul wrote: post #2

As with most tutorials on Transformers, this one leaves out some essential details: - how are the input encodings generated? - what is in those position vectors? - how are the attention vectors learned? The answer is that these things are all learned as the network is trained; the whole thing is one “thing”. The concept that is most important in understanding neural networks generally is that they start out as just a…

note that embedding and positional vectors are fixed and not trained with the transformer. The attention vectors are also computed in each step, but what is learned is the transformation of inputs to qvk vectors and the fully connected layers. There are some great people on youtube who explain the series of steps. I think this is the most comprehensive: https://www.youtube.com/watch?v=Nw_PJdmydZY

Thank you so much for setting me straight. CodeEmporium’s 10-part explainer is so great.

Re: What are transformer models and how do they work?

#92

Post by alephxyz »

radarsat1 wrote: post #56

I've been modifying an LSTM GAN model to use a transformer in the encoder and it seems to do much worse. Or at least the training dynamics are very different. Transformers perform great when they work but it seems to be a lot harder to get them to work in my experience. Can anyone corroborate that or is it likely that I'm doing something fundamentally wrong? To be clear I'm not implementing it myself but using PyTorc…

I'm mostly working on fairly simple image segmentation tasks but in my experience just replacing convolutional layers with attention layers + position embeddings works well. Using convolutional embeddings before the transformer encoder also helps.

It still take a lot more epochs to train though, so you might have to decrease the learning rate of your discriminator by a lot.

Re: What are transformer models and how do they work?

#93

Post by HarHarVeryFunny »

p-e-w wrote: post #24

The more I learn about the technical details of how ML systems are implemented, the more I feel that those details obscure , rather than illuminate, what is actually going on. It's as if we were trying to understand human ethics by looking at neurotransmitters or synapses in the brain. These structures seem way too low-level to actually explain the interesting stuff. What I hear is "something something transformer au…

> Where is the connection between computational details and the model's high-level behavior?

I think for the most part we don't know. People at OpenAI/etc who are training/testing these models and trying to control them no doubt have some understanding of how they are actually working, but they are certainly not claiming to fully understand.

At a purely conceptual level I think the best way to begin to bridge the gap between plumbing and behavior is to forget the training objective and consider what the models must have been forced to learn in order to optimize that objective. Sutskever from OpenAI has called what they've learnt a "world model", meaning a model of the generative processes (the human mind and entities being discussed?) that are producing the sequence of words they are predicting. It's certainly way more abstract than learning some "stochastic parrot" surface level statistics of the training data, even if that's maybe a good starting point to describe it to a layman.

It would be fascinating to know exactly how these models are performing reasoning - by analogy (abstract pattern matching) perhaps ?

Re: What are transformer models and how do they work?

#95

Post by montebicyclelo »

chaxor wrote: post #18
jaidhyani wrote: post #5

Skimming it, there are a few things about this explanation that rub me just slightly the wrong way. 1. Calling the input token sequence a "command". It probably only makes sense to think of this as a "command" on a model that's been fine-tuned to treat it as such. 2. Skipping over BPE as part of tokenization - but almost every transformer explainer does this, I guess. 3. Describing transformers as using a "word embed…

I know we don't have access to the details at OpenAI - but it does seem like there have been significant changes to the BPE token size over time. It seems there is a push towards much larger tokens than the previous ~3 char tokens (at least by behavior)

OpenAI have made their tokenizers public [1].

As someone has pointed out, with BPE you specify the vocab size, not the token size. It's a relatively simple algo, this Huggingface course does a nice job of explaining it [2]. Plus the original paper has a very readable Python example [3].

[1] https://github.com/openai/tiktoken

[2] https://huggingface.co/course/chapter6/5?fw=pt

[3] https://arxiv.org/abs/1508.07909

Re: What are transformer models and how do they work?

#96

Post by edge17 »

isaacfung wrote: post #60
bomewish wrote: post #48

Earlier quoted context omitted.

You seem to know a bunch about this. What’s your rec for best single explainer?

Not the guy you asked, but these are often recommended. https://jalammar.github.io/illustrated-transformer/ https://nlp.seas.harvard.edu/2018/04/03/attention.html

Before going and digging into these, could you also explain what the necessary background is for this stuff to be meaningful?

In spite of having done a decent amount with neural networks, I'm a bit lost at how we suddenly got to what we're seeing now. It would be really helpful to understand the progression of things because I stepped away from this stuff for maybe 2 years and we seem to have crossed an ocean in the intervening time.

Re: What are transformer models and how do they work?

#97

Post by pmoriarty »

HarHarVeryFunny wrote: post #89
fruit2020 wrote: post #39

Earlier quoted context omitted.

Do you mean that the gpt creators cannot backtrack an answer to understand how the model came up with it? If it’s such a black box how do they evolve it? Trial and error?

Neural nets are not generally trained through evolution ("trial and error"), but rather via error minimization, and this is how these GPT models are trained. The basic idea is that the neural net is just a mathematical function, with lots of parameters that control how it calculates it's output, that derives an output value (or set of values) for any input. During training, the neural net also calculates an error (ak…

In other words, we can tell the neutral nets when they're getting "warmer" or "colder" to desirable speech, but we don't know how they do it.

Re: What are transformer models and how do they work?

#98

Post by albertzeyer »

ChatGTP wrote: post #73
albertzeyer wrote: post #71

Earlier quoted context omitted.

There is some research trying to analyze and explain how and why it learns. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2023/privileged-basis/index... https://distill.pub/2020/circuits/ But I would not expect that we will really understand in detail how everything works. But do we need to? We also don't understand how the human brain works, but it is still useful.

The answer is yes. Otherwise the answer should be, we can't really trust the output and it will need to be treated rather suspiciously,just like we have to treat human outputs. At least humans can generally explain their rationale and be hold accountable.

GTP can also explain its reasoning. But that does not tell at all whether this reasoning is really accurate or correct. The same as for humans. When you ask them for some reasoning, they will give you sth, but it doesn't mean that is their real reasoning. There is always a lot of subjective feeling involved which you cannot really formalize. For both GPT and humans.

You can't really trust the output of humans. Still, they are somewhat useful.

Re: What are transformer models and how do they work?

#99

Post by radarsat1 »

alephxyz wrote: post #92
radarsat1 wrote: post #56

I've been modifying an LSTM GAN model to use a transformer in the encoder and it seems to do much worse. Or at least the training dynamics are very different. Transformers perform great when they work but it seems to be a lot harder to get them to work in my experience. Can anyone corroborate that or is it likely that I'm doing something fundamentally wrong? To be clear I'm not implementing it myself but using PyTorc…

I'm mostly working on fairly simple image segmentation tasks but in my experience just replacing convolutional layers with attention layers + position embeddings works well. Using convolutional embeddings before the transformer encoder also helps. It still take a lot more epochs to train though, so you might have to decrease the learning rate of your discriminator by a lot.

I admit I do run out of patience when it's been running for quite a while and seems to be really far behind the equivalent number of iterations for my LSTM solution. I often stop and adjust things and try again, when maybe it just needs to run longer. I will try that, thanks.

Re: What are transformer models and how do they work?

#100

Post by HarHarVeryFunny »

pmoriarty wrote: post #97
HarHarVeryFunny wrote: post #89

Earlier quoted context omitted.

Neural nets are not generally trained through evolution ("trial and error"), but rather via error minimization, and this is how these GPT models are trained. The basic idea is that the neural net is just a mathematical function, with lots of parameters that control how it calculates it's output, that derives an output value (or set of values) for any input. During training, the neural net also calculates an error (ak…

In other words, we can tell the neutral nets when they're getting "warmer" or "colder" to desirable speech, but we don't know how they do it.

Well sort of... The odd thing about large transformers is that there is such a huge qualitative difference between what they learn (hence how they behave) and how they are trained, so it's hard to say that this predict-next-word error feedback is directly controlling their inference behavior.

Given what the model is learning, it's perhaps best to regard this predict-next-word feedback not as "this is what I'd like you to generate", but rather something more indirect like "learn to generate something like this, and you'll have learnt what I want you to learn". A bit like Karate Kid and "wax on, wax off", perhaps!

The actual desirability of what the model is generating, which depends on what you want to use it for, is really controlled by subsequent training steps, such as:

1) Fine tuning for instruction (prompt) following and conversational ability (this is the difference between ChatGPT and the underlying raw GPT-3 model)

2) Goal-based reinforcement learning to stop the model from generating undesirable content such as telling suicidal people to kill themselves, etc, etc.

Post reply on HN