Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

521–530 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#521

Post by sejje »

yobbo wrote: post #140
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

Yes. It's like looking at the (Apollo) moon rocket launch and then suggesting teenagers should learn to build rockets in their garages for the coming space age. It is viable as a toy project, but there are vanishingly few career opportunities.

How far into the future can you see?

Re: I were 17, I'd learn how to build LLMs from scratch

#522

Post by sejje »

InsideOutSanta wrote: post #152
embedding-shape wrote: post #44

Earlier quoted context omitted.

Intuitively, that'd make sense if that unsuccessful person eventually found success, otherwise who knows if they actually picked up what made them unsuccessful in the first place?

I think it's reasonably common for people to understand what makes them unsuccessful but be unable or unwilling to resolve the problem. For me, I know that one major reason for putting an upper limit on my success is my inability to form effective professional relationships. I know I should go to events and talk to people and use these relationships to my advantage, but that's just not something I've ever been good a…

How do you know that's limiting your success?

Do you think the most successful people got that way by attending events? I thought they were hacking in their garages, building great products.

Re: I were 17, I'd learn how to build LLMs from scratch

#523

Post by hmokiguess »

17 is such a fantastic age to be free and experience the world, you won't get that much of an advantage as these FOMO groomers are selling you into if you start now versus later.

If you are 17, go be yourself, whatever that is, in whatever way you want that to be, but do it so authentically and fully. Be unapologetic about what you love and what motivates you, and pursue that with passion and commitment.

Re: I were 17, I'd learn how to build LLMs from scratch

#524

Post by simonjgreen »

paulryanrogers wrote: post #332
nextaccountic wrote: post #273

Earlier quoted context omitted.

If someone writes a browser in their teens, they will probably learn more about the web than if they were just writing web pages

They'll learn so much more that won't transfer to as many job opportunities. For ex, say more about C++ and less about cutting edge CSS (because modern browser tech is an ocean). I suppose they might luck into other adjacent or unrelated roles with the same skills.

This is like saying writing a compiler doesn't give you transferable skills. The surrounding competencies required to do this grant a pretty large amount of broad domain awareness

Re: I were 17, I'd learn how to build LLMs from scratch

#525

Post by [deleted] »

koe123 wrote: post #31

While knowledge is always great, I would encourage people not to seek advice from successful people like this (survivorship bias). Moreover I am not sure it is even good advice? Would you advise a 17 y.o. to learn how transistors work or how to code (i.e. is LLM training the right level in the stack)? LLM training, a discipline where relevant work is already out of reach for 99.999% of budgets really as essential as…

[deleted]

Re: I were 17, I'd learn how to build LLMs from scratch

#526

Post by fuzzfactor »

paulcole wrote: post #388
smcg wrote: post #378

Earlier quoted context omitted.

sounds like a great way to not establish social skills

It’s not either/or. You can do 2 things.

I never thought of that :)

I "guess" you could put all your effort into moving up to the next level in computer science, or you could put the same progressive effort into reaching new levels in video games on the same hardware.

Alternatively you could even put all your effort into social activities and leave the technology to other people entirely.

It might even be possible to find a balance between things that are widely "understood" "socially", and those that are not ;)

Re: I were 17, I'd learn how to build LLMs from scratch

#527

Post by beambot »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

Y'all are missing the point: It's probably less than 100 hours to learn the foundations of one of humanity's most-current breakthroughs. It's a disservice to any young hacker to not learn it. Here's your curriculum. Watch these:

-- 3Blue 1Brown's Neural Network Series: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_6700...

-- Karpathy on LLMs: https://www.youtube.com/watch?v=7xTGNNLPyMI

-- Stanford CS336: https://www.youtube.com/watch?v=JuoVZkPBiKk

Then do this hands-on:

-- Karpathy's zero-to-hero: https://karpathy.ai/zero-to-hero.html

Re: I were 17, I'd learn how to build LLMs from scratch

#528

Post by anthonyrstevens »

swingboy wrote: post #310
giancarlostoro wrote: post #247

Earlier quoted context omitted.

Game cheating and reverse engineering MMO backends taught me a lot: databases, networking, securing a backend (and frontend), limitations of simpler languages when comparing them to more native options for building backends.

This was my introduction, too, but with Counter-Strike cheats.

A curse upon you and your descendants

Re: I were 17, I'd learn how to build LLMs from scratch

#529

Post by anthonyrstevens »

ekidd wrote: post #363
virtualritz wrote: post #246

Earlier quoted context omitted.

AI is the subtrate the future runs on. And so I think the idea is more to understand tomorrow ... from first principles. In the late 80's, as a teenager, I learned x86 assembly and C because that was the only way to squeeze out enough juice from my shitty CGA (and later VGA) card to programm the games/graphics that interested me. I haven't written assembly in years. But whatever I did in my career: it helped me and g…

> AI is the subtrate the future runs on. Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures. So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI…

>> many corporate AI projects are spectacular failures.

Citation needed

Re: I were 17, I'd learn how to build LLMs from scratch

#530

Post by danielmarkbruce »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

This is a silly take. You can learn to build an LLM, there are great resources to do so (there are books about building them from scratch), you can use older model GPUs or rent them by the hour. The value of understanding them is really high for anyone building any application that uses an LLM at any point.

It similar to understanding how a very basic CPU works. Just because I'm not going to work at intel or nvidia or whatever optimizing the hell out of a chip, it doesn't mean I just throw my hands up and think "magic" - the basic architecture isn't that difficult, and the value of knowing it is astronomical for anyone writing software.

Post reply on HN