Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

401–410 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#401

Post by chvid »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

It is on my list to build a toy LLM from scratch.

Not that expect to make it big as a LLM researcher but building something from scratch gives a much deeper understanding than what you can get from simply using something.

Much in the same way as implementing and designing your own programming language makes you a much better programmer.

Re: I were 17, I'd learn how to build LLMs from scratch

#402

Post by serbuvlad »

zorked wrote: post #265
serbuvlad wrote: post #190

Earlier quoted context omitted.

If you learnt to build a browser in 2000 you're probably doing well for yourself lol. Not a lot of demand, but also probably not a lot of supply.

There is an entire graveyward of browsers, almost all that didn't die are now forgotten.

Sure, but I'm sure most of their lead devs are well paid now.

In general C++ work and similar, if not in Chrome development.

Re: I were 17, I'd learn how to build LLMs from scratch

#404

Post by serbuvlad »

enos_feedler wrote: post #293
serbuvlad wrote: post #190

Earlier quoted context omitted.

If you learnt to build a browser in 2000 you're probably doing well for yourself lol. Not a lot of demand, but also probably not a lot of supply.

For this example, if you built a browser in 2000 it would put you in a great position to launch android in 2008. I think this is pg’s point. You spend all this time learning the interesting bits and when an inflection comes along that makes a new product or service possible you could be the one to likely launch it

I am talking from the perspective of the individual developer.

You don't need your impressive product to succeed to land a good career.

If Alice is doing LLM-from-scratch work today, and Bob is doing agent harness work today, Bob's project is far more likely than Alice's to become useful/popular/profitable.

But if neither project survives, in 5 years, Alice will be more employable/at a higher market rate than Bob.

Re: I were 17, I'd learn how to build LLMs from scratch

#405

Post by khalic »

spolitry wrote: post #376
khalic wrote: post #201

Earlier quoted context omitted.

lol unnecessary and inefficient… I just finished fine tuning Gemma e2b for local code completion on my local machine. This comment just reinforces what the post actually means. We need people that are LLM natives, computing solves itself with time and with scale adjustments

That’s great but the industry has moved on to code generation and editing, not code completion.

1. I very much still write code myself without an LLM when I need top quality

2. That's why I have an agentic agent as well installed, Qwen 27B, outrageously good, better than sonnet 2 years ago. And it's mine, I can give it confidential info to work with since I own the whole chain. See where I'm going with this?

Re: I were 17, I'd learn how to build LLMs from scratch

#406

Post by hardwaregeek »

yobbo wrote: post #140
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

Yes. It's like looking at the (Apollo) moon rocket launch and then suggesting teenagers should learn to build rockets in their garages for the coming space age. It is viable as a toy project, but there are vanishingly few career opportunities.

That seems like a perfectly good use of time for a kid in the 70’s! You’d learn a lot of engineering skills and demonstrate a tenaciousness that most people don’t have. I don’t think the goal is to predict what will be the important technology in 10 years. The goal is to challenge oneself with hard tasks and learn interesting stuff. A lot of the “AI” people today were compilers people yesterday

Re: I were 17, I'd learn how to build LLMs from scratch

#407

Post by HPsquared »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

It is written in the first person, I suppose.

Re: I were 17, I'd learn how to build LLMs from scratch

#408

Post by fifilura »

spolitry wrote: post #361
paulryanrogers wrote: post #334

Earlier quoted context omitted.

For every wunderkind that has a long career there's also plenty who are overlooked or peak early.

Citation desperately needed.

https://en.wikipedia.org/wiki/Survivorship_bias

Re: I were 17, I'd learn how to build LLMs from scratch

#409

Post by q8zd3 »

giancarlostoro wrote: post #247
boredumb wrote: post #238

Earlier quoted context omitted.

I agree with mostly all of this, but personally I wrote a toy LLM almost 5 years ago and while it never saw much use outside of boring my wife with a shitty command line demo with glee it did help me understand how they worked and how to apply them, played a lot with JAX and pytorch, ended up building a ghetto version of MCP and an LLM-Pool to proxy requests to my baby local models and so I didn't struggle to see the…

Game cheating and reverse engineering MMO backends taught me a lot: databases, networking, securing a backend (and frontend), limitations of simpler languages when comparing them to more native options for building backends.

It was Quake World for me
Post reply on HN