Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What are transformer models and how do they work?

txt.cohere.ai

Post reply on HN
Print view View original

81–90 of 114 posts

Re: What are transformer models and how do they work?

#81

Post by HarHarVeryFunny »

ttul wrote: post #2

As with most tutorials on Transformers, this one leaves out some essential details: - how are the input encodings generated? - what is in those position vectors? - how are the attention vectors learned? The answer is that these things are all learned as the network is trained; the whole thing is one “thing”. The concept that is most important in understanding neural networks generally is that they start out as just a…

Yes, that's increasingly the case, but there's no fundamental reason for nets to be trained end-to-end as a single entity.

Going back a few years it used to be quite common for people to use fixed word embeddings such as word2vec rather than learning them, and for image classification to take an ImageNet-pretrained general purpose model, then freeze the lower convolutional feature-detector layers and only train a new model "head" for more specialized use.

End-to-end learnt embeddings are going to be more optimal though, and in the context of these massive models the computational cost of training them is a drop in the bucket!

Re: What are transformer models and how do they work?

#82

Post by HarHarVeryFunny »

kleiba wrote: post #49

Oh, come on - this article is from three days ago, but it starts with "Transformers are a new development in machine learning". Transformers have been around for six years now, that's an eternity when you consider how fast this field is moving.

True, yet we're still all here talking about them with plenty of questions and confusion, and people outside of ML are suddenly curious to know what's behind things like ChatGPT, unaware of the history.

FWIW one of the founders of Cohere (where this article comes from) was Aidan Gomez who was one of the transformer paper authors.

Re: What are transformer models and how do they work?

#83

Post by simonh »

albertzeyer wrote: post #71
p-e-w wrote: post #24

The more I learn about the technical details of how ML systems are implemented, the more I feel that those details obscure , rather than illuminate, what is actually going on. It's as if we were trying to understand human ethics by looking at neurotransmitters or synapses in the brain. These structures seem way too low-level to actually explain the interesting stuff. What I hear is "something something transformer au…

There is some research trying to analyze and explain how and why it learns. https://transformer-circuits.pub/2021/framework/index.html https://transformer-circuits.pub/2023/privileged-basis/index... https://distill.pub/2020/circuits/ But I would not expect that we will really understand in detail how everything works. But do we need to? We also don't understand how the human brain works, but it is still useful.

The more you understand something the better you can optimise, improve and engineer it. There’s also a matter of trust, particularly on issues like alignment. It’s hard to trust someone if you don’t understand their motivations.

Re: What are transformer models and how do they work?

#84

Post by guy98238710 »

remontoire wrote: post #79
guy98238710 wrote: post #78

There's one aspect I never saw explained. Why is masking used instead of a sliding window? Why even bother with masking when future tokens can be easily hidden by simply positioning the context window before the current token? Isn't sliding window optimal for maximizing context available to the model? Is masking done because moving the window would impact computational cost or output stability/quality? Can anyone she…

I think it's because you want to be able to predict the next token using only 1 token or the whole context window (and any size inbetween). So, you end up getting n different losses for each text snippet (where n is the size of the context window). If i'm wrong, can someone correct here, would be useful to know.

Why would you train the model on shorter context than you can provide? Why not provide all context you have? Sure the model has to learn to handle short context, but that occurs naturally at the beginning of the document.

Anyways, this still involves only left-side masking. Why mask future tokens when sliding window can do that (without wasting a single token of context)?

Re: What are transformer models and how do they work?

#85

Post by jstx1 »

denton-scratch wrote: post #55

> In short, text embeddings send every piece of text to a vector (a list) of numbers. What does "sent to" mean? Is that baby-talk for "mapped to"?

It's poorly written but yes, an embedding is a representation of text as a list of numbers.

[text] --> [ML model] --> [list of numbers]

Re: What are transformer models and how do they work?

#86

Post by simonh »

fruit2020 wrote: post #39
pmoriarty wrote: post #36

Earlier quoted context omitted.

This is called emergent behavior, and we don't know how it happens with organic brains, minds, and neurons either. It's actually pretty amazing that it's happening at all with computers, since neural nets are such simple, high level abstractions compared to how the brain works. It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is simil…

Do you mean that the gpt creators cannot backtrack an answer to understand how the model came up with it? If it’s such a black box how do they evolve it? Trial and error?

Basically yes, they use a system called RLHF for Reinforcement Learning with Human Feedback.

At a super high level you train your model on source texts. Then you have it generate responses from prompts. Humans rate these responses to select the best ones which updates the model, but you also train a new reward model to mimic the human rankings. Then you train the original model by having it generate millions of responses which are ranked by the rearward model. When I explained this to my brother he literally spat out his tea in horror.

This allows you to train at huge scale, many orders of magnitude beyond what you could achieve with just human ranking.

The problem is this relies on the reward model accurately capturing what makes a response ‘better’. What it’s actually doing is learning what responses get ranked highly by humans, for whatever reason. Hence the risk of LLMs becoming emotionally manipulative sycophants. It turns out alignment is a really hard problem.

Re: What are transformer models and how do they work?

#87

Post by pmoriarty »

jacknews wrote: post #75
pmoriarty wrote: post #72

Earlier quoted context omitted.

"But I think you mean the 'architectural' or connectome complexity of the brain compared to GPT, and I agree it's surprising that such a simple model as GPT is so capable." No, I'm referring to things like Roger Penrose's conjecture that subatomic interactions in the brain might be a key component of consciousness.[1] Even a single neuron is incredibly complex, and humans just don't completely understand it (or any o…

Ah OK, I've read nearly all his books, and I'm not convinced by the 'quantum microtubules' argument or whatever it's called these days, let alone any arguments about souls and so on. I agree these models are surprisingly capable for their complexity, and that's going to be a challenge for mystics (even physicist mystics) and spiritualists, etc. Perhaps intelligence isn't all that difficult after all. I suppose one co…

Another interesting angle of approach to this mystery is panpsychism[1], which as been fashionable in some philosophical circles lately.

[1] - https://en.wikipedia.org/wiki/Panpsychism

Re: What are transformer models and how do they work?

#88

Post by Emanation »

Saying we develop sentences one word at a time seems wrong. Sure, it might appear so when we're writing out text, lag of input, but if you spend any time meditating on your own thoughts it becomes apparent that it's more of chunks of words, clauses, or the idea, that are formed followed by a sweeping compulsion to think the words in their entirety.

The concept is conceptualized and then entire phrases resonate with said concet

Re: What are transformer models and how do they work?

#89

Post by HarHarVeryFunny »

fruit2020 wrote: post #39
pmoriarty wrote: post #36

Earlier quoted context omitted.

This is called emergent behavior, and we don't know how it happens with organic brains, minds, and neurons either. It's actually pretty amazing that it's happening at all with computers, since neural nets are such simple, high level abstractions compared to how the brain works. It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is simil…

Do you mean that the gpt creators cannot backtrack an answer to understand how the model came up with it? If it’s such a black box how do they evolve it? Trial and error?

Neural nets are not generally trained through evolution ("trial and error"), but rather via error minimization, and this is how these GPT models are trained.

The basic idea is that the neural net is just a mathematical function, with lots of parameters that control how it calculates it's output, that derives an output value (or set of values) for any input.

During training, the neural net also calculates an error (aka "loss") value representing the difference between it's current (at this stage of training) output value and what it was told is the preferred output value for the current input.

The process of training is done by slowly adjusting the neural net parameters until these calculated output errors are as small as possible for as many of the training examples as possible.

The way these errors are reduced/minimized is by using the derivative (slope) of the neural network function - we want to follow the slope of the error function downhill to a place where the error value is lower, and this is done by adjusting the parameter values using partial derivatives.

The details of this downhill slope following (the "backprop" algorithm) are a bit complex, but you can visualize it as a 3-D hilly landscape where the height of the hills represents the size of the error, and the goal it to get into the lowest valley of the landscape (corresponding to the lowest error). If your current lat/long position in the landscape is (x, y) and you know the slope of the hill you are on, then you can move downhill towards the valley by moving a bit in the appropriate direction from (x,y) to (x+dx, y+dy). These x, y values represent the parameters of the network, so by continually tweaking them from (x,y) to (x+dx,y+dy) for each training sample, you are slowly moving down the error hill in the right direction towards the valley of lowest error.

Re: What are transformer models and how do they work?

#90

Post by MichaelZuo »

reubenmorais wrote: post #65
p-e-w wrote: post #44

Earlier quoted context omitted.

> It's possible that all the tremendous complexity of organic systems isn't actually necessary for intelligence or consciousnes, which is similarly surprising. My guess is most of that complexity is necessary for efficiency , not for basic function. Biological systems are unimaginably efficient at almost everything they do. The information storage density of DNA is within 1-2 orders of magnitude of the upper limit im…

Efficiency and also redundancy. Brains and bodies have incredible tolerance to damage.

And also reproducibility in adverse conditions without too many defects.
Post reply on HN