When I was not 17 at the times of GPT2, I decided to not bother with learning how to build LLMs because it’s too expensive for an individual. This escalated quickly.
I were 17, I'd learn how to build LLMs from scratch
21–30 of 725 posts
Re: I were 17, I'd learn how to build LLMs from scratch
#22I do not think it is a proper thing to do for 17 y.o., unless they are exceptionally mathematically gifted, as proper understanding of how LLMs are trained requires a good grasp of calculus, understanding modern OS and SDE tools for proper implementation of pipeline etc. I'd rather simply write another mnist implementation and check if I really like all that AI stuff at first place. Even then, before going into matur…
> I do not think it is a proper thing to do for 17 y.o
If I'd get a buck every time someone said something like this to me when I was in the 13-18 range, I wouldn't have a ton of money, but it's so very annoying when people tell you this.
Regardless if they're "gifted" or not, regardless if you believe in myths like that or not, let children explore what they want to explore, even if you don't understand what it is or why they want to explore that, just let people explore, regardless of age.
It was such a terrible experience being a young kid growing up, with so many adults spending hours trying to convince me to stop sitting in front of the computer so much doing whatever; "why are you even trying to learn that stuff, you have to go to school to understand anything of this" and so much other similar trash.
Sorry, not your fault and I'm borderline trauma-dumping now, but really sad to see this sort of gatekeeping on HN of all places, age is irrelevant to learning ANYTHING, in my humble opinion at least.
Kids, find anything interesting? Jump into it, ignore what adults tell you, and do whatever you feel like, you'll find your place eventually.
Re: I were 17, I'd learn how to build LLMs from scratch
#23I am kind of amazed how negative the comments are here, especially on HN.
Learning to hack something together in high school using the latest technology (vacuum tubes, radios, microprocessors, web/javascript) has been a common theme in the tech world for generations. With LLMs and online tutorials, this isn't even a difficult suggestion. Do people think learning new tech is somehow wasted effort?
Re: I were 17, I'd learn how to build LLMs from scratch
#24Earlier quoted context omitted.
Depends if you’re 17 with rich parents or not.
This only changes whether you are naive enough to believe “honest” business means anything in today’s age. If anything, I worry being honest is holding back smart people who try to compete in a rigged game.
> you are naive enough to believe “honest” business means anything in today’s age
Might be that these people are from outside the US as well, where things like "honest business" is very much possible today, probably most businesses I interact with AFK on a daily business are "honest businesses".
Re: I were 17, I'd learn how to build LLMs from scratch
#25I do not think it is a proper thing to do for 17 y.o., unless they are exceptionally mathematically gifted, as proper understanding of how LLMs are trained requires a good grasp of calculus, understanding modern OS and SDE tools for proper implementation of pipeline etc. I'd rather simply write another mnist implementation and check if I really like all that AI stuff at first place. Even then, before going into matur…
[deleted]
Re: I were 17, I'd learn how to build LLMs from scratch
#26Can't this guy enjoy being rich in silence? His takes get worse with every passing year.
He did get rich by being pretty much the opposite of silent, so I'm guessing you can't just turn off that part, kind of comes with the package ;)
Re: I were 17, I'd learn how to build LLMs from scratch
#27Earlier quoted context omitted.
That's like saying the only way to do real engineering is with Google-scale Borg deployments. You can do quite a lot on very little hardware, r/StableDiffusion is a prime example.
You can do plenty of "real engineering" under normal conditions. But specifically when it comes to LLM engineering, no there's really not much you can do, they are called "large" for a reason. You can play around at small scale, but those lessons you learn will not be very relevant to the real problems in the market. Sure you can gradually climb the ladder by demonstrating your skills bit by bit and getting access to…
Agreed. It's hard to learn unless you have access to quite high end hardware, and even paying by the hour is expensive. There's a low ceiling on what you can learn without doing training runs.
You can however learn everything you need to know to get on the career ladder as a software engineer on a regular home PC.
Re: I were 17, I'd learn how to build LLMs from scratch
#28I do not think it is a proper thing to do for 17 y.o., unless they are exceptionally mathematically gifted, as proper understanding of how LLMs are trained requires a good grasp of calculus, understanding modern OS and SDE tools for proper implementation of pipeline etc. I'd rather simply write another mnist implementation and check if I really like all that AI stuff at first place. Even then, before going into matur…
>understanding modern OS and SDE tools for proper implementation of pipeline etc.
Can you provide an example?
Re: I were 17, I'd learn how to build LLMs from scratch
#29This might be of use
Re: I were 17, I'd learn how to build LLMs from scratch
#30I learned HTML when I was 17 in about 1995 and it's certainly taken me on a pretty fun career path. Less technical than LLMs for sure, but 'figure out where the industry is going and move what you're learning to there' is solid advice.