I love Karpathy et al but this is what finally made it click for me: https://youtu.be/kWLed8o5M2Y?si=SJT5_lCJ0hSR7Z_k
I made a transformer to predict a simple sequence manually
71–80 of 104 posts
Re: I made a transformer to predict a simple sequence manually
#72Earlier quoted context omitted.
Great example. Right, the deep learning approach uncovers all kinds of hidden features and relationships automatically that a team of humans might miss. I guess I'm thinking about this problem from the perspective of these GPT models requiring more training data than a normal person can acquire. Currently, it seems you need the entire internet worth of training data (and a lot of money) to get something that can comm…
I am still learning transformers, but I believe part of the issue may be that the weights do not necessarily correlate to things like "orangeness" Instead of a transformer for each color, you have like 5 to 100 weights that represent some arbitrary combination of colors. Literally the arbitrariness is defined by the dataset and the number of weights allocated. They may even represent more than just color. So I am not…
It is possible that the parameters, like weights in a machine learning model, interact to yield outcomes in a manner analogous to the interactions between genes in biological systems, which produce traits. These interactions involve complex interdependencies, so there really aren't 1 to 1 dials.
Re: I made a transformer to predict a simple sequence manually
#73Re: I made a transformer to predict a simple sequence manually
#74Earlier quoted context omitted.
Neural networks are Turing machines. You can make them perform any computation by carefully setting up their weights. It would be nice to have compilers for them that were not based on approximation, though.
Would I be able to solve the Travelling Salesman Problem with a Transformer with the appropriately assigned weights? That would be an achievement. You'd beat some known bounds of the complexity of TSP.
We don't know.
And even if we could try to solve this problem, there is no known way of verifying the solution would be correct in general.
Re: I made a transformer to predict a simple sequence manually
#75A related line of work is "Thinking Like Transformers" [1]. They introduce a primitive programming language, RASP, which is composed of operations capable of being modeled with transformer components, and demonstrate how different programs can be written with it, e.g. histograms, sorting. Sasha Rush and Gail Weiss have an excellent blog post on it as well [2]. Follow on work actually demonstrated how RASP-like progra…
It seems like a functional language like Haskell would be the right tool for this.
Also going from a net to code would be super interesting in terms of explain-ability.
Re: I made a transformer to predict a simple sequence manually
#76Earlier quoted context omitted.
I wonder if an expert can "impose" weights onto a model and the model will opt to continue with them when it resumes training. For example, in the vision example, the expert may not know where "orangeness" currently exists, but if they impose their own collection of weight adjustments that represent orangeness, will the model continue to use these weights as the path of least resistance when continuing to optimize? J…
That would make it less efficient - since learning is compression, a less compressed model will also learn less at the same size.
Humans have much more compressed models, trying to transfer human learning to machine learning could potentially be a way to get more efficient models.
The way we do that currently is by labeling data for training, but maybe there are better ways to do it. Like some semi code to write with hints for the model. Like, instead of labeled data, could have a series of "lectures" of labeled data that would lead to a good end state, instead of training on all the data in parallel.
You don't teach a child calculus by showing them a million calculus problems after all, you ramp up starting with simple numbers and then slowly ramping up with more concepts. But to do that we would need to change how we train models.
Edit: By doing it that way you could see the skill of the model after each lecture, and update the lecture to try to make the model learn better. Not sure how to do that, but such ways to work with parts of models is a potential way forward.
Re: I made a transformer to predict a simple sequence manually
#77Earlier quoted context omitted.
That would make it less efficient - since learning is compression, a less compressed model will also learn less at the same size.
Humans have much more compressed models, trying to transfer human learning to machine learning could potentially be a way to get more efficient models. The way we do that currently is by labeling data for training, but maybe there are better ways to do it. Like some semi code to write with hints for the model. Like, instead of labeled data, could have a series of "lectures" of labeled data that would lead to a good e…
There are techniques for this called "curriculum learning" and "textbooks".
I'm not sure exactly what is in the textbooks since I admit to not reading the papers yet.
I'm personally wondering if you could increase the reliability of training on web data by labeling it with where each document came from, so it knows different authors disagree on things. But this brings back the issue where people don't like it if a model can write "in the style of Author Name"…
Re: I made a transformer to predict a simple sequence manually
#78I thought I understood transformers well, even though I had never implemented them. Then one day I implemented them, and they didn't work/train nearly as well as the standard pytorch transformer.
I eventually realized that I had ignored the dropout, because I thought my data could never overfit. (I trained the transformer to add numbers, and I never showed it the same pair twice.) Turns out dropout has a much bigger role than I had realized.
TLDR, just go and implement a transformer. The more from scratch the better. Everyone I know who tried it, ended up learning something they hadn't expected. From how training is parallelized over tokens down to how backprop really works. It's different for every person.
Re: I made a transformer to predict a simple sequence manually
#79Earlier quoted context omitted.
Would I be able to solve the Travelling Salesman Problem with a Transformer with the appropriately assigned weights? That would be an achievement. You'd beat some known bounds of the complexity of TSP.
We don't know. And even if we could try to solve this problem, there is no known way of verifying the solution would be correct in general.
TSPs are not unsolvable.
My point was that Transformers and neural networks as they are now are not Turing machines if you don't allow for the model to grow with the input size. That said, it has to grow in "depth" not just parameters. The fact that people think fixed depth computation can universally compute everything is a worrying trend.
Re: I made a transformer to predict a simple sequence manually
#80Earlier quoted context omitted.
I wonder if an expert can "impose" weights onto a model and the model will opt to continue with them when it resumes training. For example, in the vision example, the expert may not know where "orangeness" currently exists, but if they impose their own collection of weight adjustments that represent orangeness, will the model continue to use these weights as the path of least resistance when continuing to optimize? J…
That would make it less efficient - since learning is compression, a less compressed model will also learn less at the same size.
Might be a worthwhile tradeoff. Less efficient model but a clear control interface, versus more efficient model but it's a blackbox.