I were 17, I'd learn how to build LLMs from scratch • Hacker News Board

Re: I were 17, I'd learn how to build LLMs from scratch

#421

Post by __MatrixMan__ » Mon, Aug 24, 2026, 2:00 PM UTC

femto wrote: post #294

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

In that regard, it's not too different from mobile telephony. Mobile phones drove the electronics industry 20 years ago, but there is limited demand for people who really know how to build a phone (ie. build the hardware and write all the signal processing from scratch), as there aren't that many companies that do phones at the lowest level. A few of the engineers got rich (eg. Viterbi), but most 'just' made a good l…

Good points, but I think we can expect the AI space to be more tumultuous.

What most people want from mobile technology is for it to work, not too expensively, and for it to get out of their way.

What most people want out of AI is for no leader to emerge and wield supremacy against the rest of us. People are afraid of it in ways they weren't afraid of mobile, so they're more willing to work together against whoever is in the lead.

Its more like an arms race and less like a utility. The disadvantage I face when my competition has better mobile coverage and bandwidth is minor. The disadvantage I face when my competition has better intelligence on tap is much more significant.

Re: I were 17, I'd learn how to build LLMs from scratch

#422

Post by ikety » Mon, Aug 24, 2026, 2:02 PM UTC

beemboy wrote: post #373

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

Yes and no. I believe the point he is making is simply that there is no substitute for fundamentals and first-principles thinking. We had scores of students study how microprocessors work and compilers work over decades, yet we have 3 or 4 major processor companies and a handful of programming languages. Yet, what they learned was probably crucial in their development as engineers. We are also so early right now that…

This is kind of different though isn't it? Doing an assembly or compiler class has pretty clear benefits in this regard.

But LLMs are tools. Does a great engineer need to know how vscode works? Might be helpful to understand how extensions work, LSPs, and project configurations.

Usually when working with any tools, you need to understand how to get the most out of your tool for your needs and that's about it. Core fundamentals about how software and hardware works in general seems like it would be MUCH more useful than LLM core knowledge.

Re: I were 17, I'd learn how to build LLMs from scratch

#423

Post by smcg » Mon, Aug 24, 2026, 2:03 PM UTC

_fizz_buzz_ wrote: post #214

Why are people so negative about this? It feels like a fun project and at 17 the stakes are not really high. Something one could easily do on summer break in a couple of weeks.

what hardware can a broke teenager get access to in a couple weeks? is this really better than learning how to program?

Re: I were 17, I'd learn how to build LLMs from scratch

#424

Post by aswegs8 » Mon, Aug 24, 2026, 2:05 PM UTC

serbuvlad wrote: post #402

zorked wrote: post #265

Earlier quoted context omitted.

There is an entire graveyward of browsers, almost all that didn't die are now forgotten.

Sure, but I'm sure most of their lead devs are well paid now. In general C++ work and similar, if not in Chrome development.

cause != effect

Re: I were 17, I'd learn how to build LLMs from scratch

#425

Post by AIorNot » Mon, Aug 24, 2026, 2:06 PM UTC

I think collectively we should all stop listening to Mr Graham..

He capitalizes on greed and hype but with a soft, sober and thoughtful voice so as to lull you with rationalism and now 20 years of his “disruption” has mostly ruined modern society and a whole generation of techies have been led astray into trying to “change the world” is the world of today (minus the magic technology really any better than 20 years ago?)

- Good for him and his Tech Bros, bad for the rest of society

Re: I were 17, I'd learn how to build LLMs from scratch

#426

Post by greenraven » Mon, Aug 24, 2026, 2:06 PM UTC

An LLM isn’t hard to make - the training data is hard to get and prepare.

The big companies stole the data. The average person can’t do that

Re: I were 17, I'd learn how to build LLMs from scratch

#427

Post by Kuyawa » Mon, Aug 24, 2026, 2:07 PM UTC

You can build the program, but to train it is another beast, billions of docs, images, videos, which a mere mortal doesn't have access to

Second in line, build your own agent, that's more in our ballpark, then customize it to your needs, both virtual and physical

Re: I were 17, I'd learn how to build LLMs from scratch

#428

Post by arscan » Mon, Aug 24, 2026, 2:10 PM UTC

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

I think his point is to do this to understand deeply what they can do, what they can’t do, and what they can almost do. And then find the highest value ‘almost’ use case and push there. Which doesn’t necessarily mean improve the llm, could be applying it in just the right way for the use case. Of course, the bitter lesson makes this hard and risky. But no more risky than investing your time in learning anything else these days.

Re: I were 17, I'd learn how to build LLMs from scratch

#429

Post by pwillia7 » Mon, Aug 24, 2026, 2:11 PM UTC

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

I bet this will get less true over time though as the rate of change slows down, allowing specialized models/training for specific use cases that aren't TAM heavy enough for the big labs to go after them. It's just now any general model is the best thing to use for everything and you're wasting money to build something on what will certainly be obsolete by the time you can get it to market

Re: I were 17, I'd learn how to build LLMs from scratch

#430

Post by CurbStomper » Mon, Aug 24, 2026, 2:12 PM UTC

[dead]