Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

381–390 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#381

Post by sonabinu »

polnoner wrote: post #248
simplegeek wrote: post #73

As an aside, for someone interested and who's an absolute beginner, can someone please recommend good resources on how to build LLMs from scratch? Thank you in advance.

Stanford CS336 is up on youtube from Spring 2026.

I think this is the best structured class out there that teaches how to scale LLMs . Hope the 17 year old knows linear algebra. Building an intuition for the shape of the matrices is important. A lot of understanding the 'building from scratch' means understanding choices like why RoPE instead of the original frequency based positional encoding. Start with Karpathy and then go to CS 336

Re: I were 17, I'd learn how to build LLMs from scratch

#382

Post by ForHackernews »

ekidd wrote: post #363
virtualritz wrote: post #246

Earlier quoted context omitted.

AI is the subtrate the future runs on. And so I think the idea is more to understand tomorrow ... from first principles. In the late 80's, as a teenager, I learned x86 assembly and C because that was the only way to squeeze out enough juice from my shitty CGA (and later VGA) card to programm the games/graphics that interested me. I haven't written assembly in years. But whatever I did in my career: it helped me and g…

> AI is the subtrate the future runs on. Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures. So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI…

I keep saying AI is going to prove more impactful than cloud computing but less impactful than the sewing machine and a bunch of people get mad at me.

Re: I were 17, I'd learn how to build LLMs from scratch

#384

Post by skylurk »

y-curious wrote: post #335
greenowl wrote: post #328

Yeah, no way. I'd move to the middle of nowhere and work multiple jobs on a farm and in construction. Learn how to grow food, and build things. Meet the farmer's daughter, and marry her. Then, buy my own land, grow my own food, and build my own things.

Slight problem with step “buy my own land”. Your working 2 blue collar jobs and paying rent are incompatible with this plan.

> Meet the farmer's daughter, and marry her.

Re: I were 17, I'd learn how to build LLMs from scratch

#385

Post by ipaddr »

keiferski wrote: post #165
epolanski wrote: post #160

Earlier quoted context omitted.

But it is not true financially, it really depends. Plenty of blue collar workers make more than white collar ones, and have a huge debt free head start in life. A plumber or electrician will make significantly more than the average bank employee or translator or nurse or teacher. And they will also have an easier time starting a business as many trade workers are self employed and make much more than hired ones.

Going to college is worth it for basically everyone other than those that are choosing between a mediocre college / mediocre degree and a blue collar route. AND if college is going to put you into debt. Financially, intellectually, socially, it’s a good idea - college is a formative period of life in American culture. That is more and more true the better the college gets. At the upper tier (Ivy League, etc.) you don…

Geberal college degree is good for mediocre white collar jobs but higher education is required for anything more. The richest drop out of college win a business lottery.

Re: I were 17, I'd learn how to build LLMs from scratch

#386

Post by DarmokTanagra »

Grombobulous wrote: post #317
DarmokTanagra wrote: post #263

telling a 17 year old to get into tech right now is horrible advice, literally telling them to get at the back of a line with a better part of a million more experienced people in it.

Telling them not to get into tech is also terribly reactionary advice. The truth of the matter is that we don’t yet know whether tech roles will be eliminated or if they’re just going to follow previous innovation breakthroughs where “one person producing way more work” makes software even more of a desirable industry to be involved in. There really isn’t a very strong correlation between tech industry hiring strengt…

“only make $115k”

This right here is why nobody is shedding tears for the massive employment crisis in tech.

You make double that shilling ai slopware while they work nights saving lives.

I would tell a 17 year old that the world will always need nurses, same can’t be said for guys sitting in their pajamas burning tokens.

What is happening now in tech has been a long time coming, and it can’t happen fast enough.

Re: I were 17, I'd learn how to build LLMs from scratch

#387

Post by softwaredoug »

My kids CS teacher asked me what they should do after AP CS.

Previously they had a class where they'd build apps for other teachers. Like tracking when clubs are, etc. But now that's become easy for teachers to vibe code themselves.

I suggested they shouldn't prereq this class on CS. Heck invite anyone in interested in "building things" and they can get practice at building apps for other people / themselves. Maybe that becomes a gateway TO CS - people who want to learn how things work under the hood.

I suggested post CS class for the CS people should probably be building an LLM or something :)

Re: I were 17, I'd learn how to build LLMs from scratch

#388

Post by paulcole »

smcg wrote: post #378
paulcole wrote: post #278

Earlier quoted context omitted.

But he’s not saying that it’s a career opportunity. It seems to me like he’s saying that doing this thing would be 1) fun and 2) a great way to become employable in the future. I don’t believe he’s saying that this project would be some kind of job training exercise.

sounds like a great way to not establish social skills

It’s not either/or. You can do 2 things.

Re: I were 17, I'd learn how to build LLMs from scratch

#389

Post by nwatson »

ignoramous wrote: post #299
nextaccountic wrote: post #273

Earlier quoted context omitted.

If someone writes a browser in their teens, they will probably learn more about the web than if they were just writing web pages

A few did. [Blake Ross] worked as an intern at Netscape at the age of 16 ... Ross became disenchanted with the browser he was working on and the direction given to it by America Online, which had recently purchased Netscape. Ross and Hyatt envisioned a smaller, easy-to-use browser that could have mass appeal, and Firefox was born from that idea ... in 2003 all of Mozilla's resources were devoted to the Firefox and Th…

In the meantime Mozilla was mocked on slashdot.org and elsewhere relentlessly after the first two years of the project when nobody believed there would be any value in the effort. Hats off to the team that took around five years to get to Firefox 1.0 (and released Mozilla browser in the interim). It took a lot of conviction to see it through.

Re: I were 17, I'd learn how to build LLMs from scratch

#390

Post by spolitry »

felixrieseberg wrote: post #43

I'll use this post as a shameless opportunity to tell more people about a little side project, I made: http://languagemodelbuilder.com teaches you (in a few hours to days) how to build an LLM from scratch. It's entirely free, without accounts, and without data collection.

Nice! Please post screenshots and videos for people not ready or able to install yet.
Post reply on HN