The problem is tech moves so fast that by time kid is adult llm are outdated
I were 17, I'd learn how to build LLMs from scratch
601–610 of 725 posts
Re: I were 17, I'd learn how to build LLMs from scratch
#602Earlier quoted context omitted.
Most reasonable advice I've seen under this post. Having graduated in 2025 I've experienced the horror of competing with infinitely many third worlders in my small country, who will gladly take 1/3 my pay and be serfs. I really hope we get another hiring boom like in 2020 when I decided to study CS. Otherwise my career will be very rough. I love it and can't imagine doing anything else.
awwww…… so cute…… who would have guessed when first world were enjoying their heyday as they were colonizing the world, and teaching everyone and their mother to learn English. Kinda sad now you have to compete with serfs from thirls world. chu chu chu. so cute. shouldn’t have robbed others then. a richer world for you comes at poorer world for someone. we are playing infinite game in finite world.
> shouldn’t have robbed others then. a richer world for you comes at poorer world for someone. we are playing infinite game in finite world.
Young people are not responsible for the crimes of their ancestors centuries ago.
Re: I were 17, I'd learn how to build LLMs from scratch
#603Earlier quoted context omitted.
The funny thing about LLM is, you can build one from scratch and yet you still won't understand how it works. You would understand what kind of matrix multiplications the neural network performs (in fact that's not that hard. An OS is orders of magnitudes more complex), but you would still have no idea why it does what it does.
This is a great point. The complexity sources are entirely different. An OS is complex because it’s solving a difficult problem and we’ve seen decades of evolution both in knowledge and-how and, at the frontier, in performance standards. The abilities of LLMs are emergent. You can experiment with LLMs and know as much about their observable behavior as the experts. But there’s no way to “crack open” an LLM and see pr…
Learning how to build emergent systems is also a skill kids should learn these days. The closest I got was coding up game of life for CSE 142 (intro programming).
If stochastic gradient descent isn't taught in whatever CS Theory 101 is now, it really should be these days.
Re: I were 17, I'd learn how to build LLMs from scratch
#604I agree more with Yann LeCunn's salty reply. Over long run, knowing how autoregressive language models work from scratch will be just one step in having foundational understanding, and they might become dated... like knowing how a CRT monitor work. Something of historical interest and good for learning, but not crucial to being well-rounded. There are other types of models like diffusion models right now that are sho…
Yann has been consistently wrong about the limits of LLMs.
Re: I were 17, I'd learn how to build LLMs from scratch
#605I assume in this hypothetical Cornell just gives away degrees to passersby.
Re: I were 17, I'd learn how to build LLMs from scratch
#606What corpus would one be advised to use for training? Should one limit oneself to legally downloadable documents or should one follow the examples of Altman, Amodei, and Zuckerberg and use LibGen and Anna's Archive?
Re: I were 17, I'd learn how to build LLMs from scratch
#607A lot of people here are responding to the message but not to the meaning. It would be a good idea for young people to deeply know how these programs work. Not so that they can spend their career building them, but so that they can approach the next class of problems we'll all start trying to solve, with intuition all the way down to the weights and underlying mathematics. And also, to develop a healthy intuition of…
What corpus would one be advised to use for training? Should one limit oneself to legally downloadable documents or should one follow the examples of Altman, Amodei, and Zuckerberg and use LibGen and Anna's Archive?
Re: I were 17, I'd learn how to build LLMs from scratch
#608When I was 17, I wrote poetry and learned how to play guitar. I’m really happy that I did. That’s what I would do now, were I 17. I feel slightly bad for people who didn’t.
Same. But if I could go back, I would add skating as well. I slept on that one and spent too much time trying to be a hacker.
Re: I were 17, I'd learn how to build LLMs from scratch
#609If I were 77 I'd do the same
Re: I were 17, I'd learn how to build LLMs from scratch
#610Earlier quoted context omitted.
I don't mean he's destitute. Just that he's no longer exceptional, which is fine.
But you were trying to refute my point of: > I’d imagine those teenagers ended up with pretty good careers in technology? The guy you mentioned doesn't seem to have ended up with a bad career in technology?
Thanks for clarifying. I read it as exceptional careers