Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

351–360 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#351

Post by ohyes »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

You can train and run small models on an old gpu. That’s what I’m doing now at, well, much older than 17. Does it produce a useful model? No. Not even remotely.

However, I do learn stuff about models that takes it from “magic” to “useful tool I understand the limitations of.”

Do I do it for that reason? No not really, I’ve never had luck learning something because it would be good for my career. I do it because at my core I’m a bored teenager who wants to make the computer do cool shit.

Re: I were 17, I'd learn how to build LLMs from scratch

#352

Post by kingofalldevs »

markb139 wrote: post #75
koe123 wrote: post #31

While knowledge is always great, I would encourage people not to seek advice from successful people like this (survivorship bias). Moreover I am not sure it is even good advice? Would you advise a 17 y.o. to learn how transistors work or how to code (i.e. is LLM training the right level in the stack)? LLM training, a discipline where relevant work is already out of reach for 99.999% of budgets really as essential as…

As a 17 y.o (way back in the last century) I didn’t need to be advised to learn about transistors. I just had a thirst for the knowledge. I would encourage everyone to learn something about transistors. They are one of mankind’s most useful discoveries.

Hey, I'm 19 CS undergrad but i don't really know what to do but I really wanna build something that can shape the world. Can you tell me more about why you would advice someone to learn about transistors? what are the possible career paths?

Re: I were 17, I'd learn how to build LLMs from scratch

#353

Post by agentultra »

draugadrotten wrote: post #286
agentultra wrote: post #268

Most 17 year olds I know don’t want anything to do with AI and see the entire industry as an existential threat. Nothing wrong with learning the theory and understanding the papers. Getting to that point you’ll have to get your fundamentals down. Might be an interesting exercise. But as a future? I guess we’ll see. I suspect the next financial apocalypse will determine if there is one. Another AI Winter that may outl…

> Most 17 year olds I know don’t want anything to do with AI The few exceptional individuals will innovate what the millions will benefit from. At 17, the mother of Isaac Newton removed Isaac from school and tried to make him a farmer. We all know that wasn't his destiny.

There’s no such thing as destiny.

You could also walk out your front door and get hit be a meteor tomorrow.

Re: I were 17, I'd learn how to build LLMs from scratch

#354

Post by sambaumann »

user43928 wrote: post #211
reacharavindh wrote: post #103

Earlier quoted context omitted.

> I know how to build an LLM, I know plenty of fellow young engineers that do too. It's really not that complex. I’m 40, and I don’t.I took that abstraction for granted and “left it to the big labs”. However I want to build my own LLM for learning purposes. On needing big expensive hardware.. necessity is the mother of great innovation. Perhaps 18year olds trying to build their own LLMs in constrained resources envir…

Well, that's not how it works. You don't just put some old laptops into a rack. Maybe with a decent consumer GPU like a 4090, you could do experiments like distilling and fine tuning a small image model for edge deployment for specific tasks. Even there, many use cases might require renting compute for $10/hour and investing a few hundred. A LLM from scratch? Forget it. You can do theoretical experiments, but not bui…

[dead]

Re: I were 17, I'd learn how to build LLMs from scratch

#355

Post by fnoef »

If i were 17, I'd try to distinguish who to take advice from, and would definitely learn that VCs have interest to spread a specific agenda in their message. Also, I would get drunk and have as much fun as could, as the misery of working under the treat of being replaced by AI, would simply kill any desire to live past 25.

Re: I were 17, I'd learn how to build LLMs from scratch

#356

Post by inciampati »

stymaar wrote: post #319
mike_hearn wrote: post #151

Earlier quoted context omitted.

Why? Oersted is correct, for any size class you can find an LLM that is free and well trained at this point. They are highly adaptable even without fine tuning, in-context learning is still superior to fine tuning in most cases also. And real world fine tuning is mostly about data gathering and cleaning. The actual adapter training is automated and put behind simple APIs. I saw my first language model in action in 20…

> They are highly adaptable even without fine tuning, in-context learning is still superior to fine tuning in most cases also. Good luck relying on in-context learning for a 600M LLM. > The actual adapter training is automated and put behind simple APIs. That's like saying it's worthless to learn infra because you can use serverless instead… > All the frontier labs have nearly identical model personalities, capabilit…

There is no reason to believe that generalist models need to be transformer (or transformer hybrids) it's just that people are risk averse and not spending their resources on the many scaled risky bets needed to explore the possibilities. There is more there.

Re: I were 17, I'd learn how to build LLMs from scratch

#357

Post by khriss »

dude250711 wrote: post #169
wiseowise wrote: post #110

[flagged]

So get into debt and spend money is the advice? Then vote for someone who will make the debts go away?

> Then vote for someone who will make the debts go away?

As opposed to vote for someone who gives even more power and money to the oligarchs? You bet your last dollar that people will choose the former over the latter.

Re: I were 17, I'd learn how to build LLMs from scratch

#358

Post by MaxHoppersGhost »

vlyan wrote: post #209
wy35 wrote: post #132

Earlier quoted context omitted.

What would be better advice for a 17 year old?

"it doesn't matter what you choose to study because no job is safe from being outsourced overseas, handed over to an indentured servant, or made redundant by a machine. you will spend your whole life surviving while the cannibalistic pedophiles who own everything invent new ways to make you own nothing. be frugal, don't get married, don't have children, do everything you can to stay healthy and independent, and you j…

> don't get married, don't have children

Bad advice. You’re going to need family connections and loyal people you can bet your life on to survive if the rest of what you’re saying is remotely true.

Re: I were 17, I'd learn how to build LLMs from scratch

#359

Post by MaxHoppersGhost »

floro wrote: post #290
vlyan wrote: post #209

Earlier quoted context omitted.

"it doesn't matter what you choose to study because no job is safe from being outsourced overseas, handed over to an indentured servant, or made redundant by a machine. you will spend your whole life surviving while the cannibalistic pedophiles who own everything invent new ways to make you own nothing. be frugal, don't get married, don't have children, do everything you can to stay healthy and independent, and you j…

Most reasonable advice I've seen under this post. Having graduated in 2025 I've experienced the horror of competing with infinitely many third worlders in my small country, who will gladly take 1/3 my pay and be serfs. I really hope we get another hiring boom like in 2020 when I decided to study CS. Otherwise my career will be very rough. I love it and can't imagine doing anything else.

Horrible. Which country is this? Canada maybe?

Re: I were 17, I'd learn how to build LLMs from scratch

#360

Post by liberian »

enos_feedler wrote: post #293
serbuvlad wrote: post #190

Earlier quoted context omitted.

If you learnt to build a browser in 2000 you're probably doing well for yourself lol. Not a lot of demand, but also probably not a lot of supply.

For this example, if you built a browser in 2000 it would put you in a great position to launch android in 2008. I think this is pg’s point. You spend all this time learning the interesting bits and when an inflection comes along that makes a new product or service possible you could be the one to likely launch it

This was his point. Being prepare to launch a produce when the opportunity comes.
Post reply on HN