Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

141–150 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#141

Post by embedding-shape »

latexr wrote: post #127
embedding-shape wrote: post #32

Earlier quoted context omitted.

> I am kind of amazed how negative the comments are here, especially on HN. I can't recall or point out exactly when, but there is a stark before/after moment where the opinions of anything pg went from "Interesting and maybe true in some ways" to what we see today, lots of knee-jerk reactions and hardly any comments about the actual content. Hazarding a guess, I think the moment Altman became the CEO and later durin…

> I can't recall or point out exactly when, but there is a stark before/after moment where the opinions of anything pg went from "Interesting and maybe true in some ways" to what we see today, lots of knee-jerk reactions and hardly any comments about the actual content. Hard disagree. This submission is still being highly upvoted, while another recent post[1] on the harms caused by Graham’s fellows[2], with a fairly…

> Mine is that HN in general (certainly more than any other website) used to worship[3] everything he wrote [...] It definitely was not “interesting and maybe true in some ways”, it was much more hardcore than that.

I guess it depends on what submission you look at, previous comment of mine solely based on memory. Now I went to https://news.ycombinator.com/from?site=twitter.com/paulg, clicked "More" a bunch of times, and seems my memory was more or less correct, none of the submissions I clicked on are "pg worship" (hyperbolic or not). Just one example: https://news.ycombinator.com/item?id=19418701

Maybe you need to enable "Show Dead" or something? Pgs articles on HN definitely never was free of any critique in the HN comments, just like any article. Although I do agree with you that it used to be different than it is today, and same with Musk too, and Altman, and probably more individuals, where they were lauded before but now pretty much just mentioning them poisons the conversation.

Re: I were 17, I'd learn how to build LLMs from scratch

#142

Post by livinglist »

mdp2021 wrote: post #137
livinglist wrote: post #55

When I was 17 I was building Windows Phone apps, bad decision on my part.

I also did, and had written code for/on other mobile devices before, and did write a large number of even very ambitious software for other mobile devices later. I do not see any past constructive experience as a waste of time.

Did you write anything for Symbian or PalmOS before WP?

Re: I were 17, I'd learn how to build LLMs from scratch

#143

Post by mdp2021 »

badgersnake wrote: post #60

If I were 17, but have the money I have now he means.

The times are different. When I was 17, we had to buy records; a 17yo today can listen to the whole of the available produced music, plus interviews and all other uncommon and related material, for free from the comfort of "here and now".

Possibilities exist now that did not exist before. Those who do not exploit this are fools.

Re: I were 17, I'd learn how to build LLMs from scratch

#144

Post by embedding-shape »

BoredomIsFun wrote: post #128
embedding-shape wrote: post #106

Earlier quoted context omitted.

> Besides, building a LLM _truly_ from the scratch, just using the famous 2017 paper and numpy manuals is not fun at all, esp. for a high schooler. To you it isn't, is my entire point here. But why extrapolate what you think is fun, to others? Sure, I don't find that fun either (although useful), but who am I to say it isn't fun for others ?

We can continue this pointless conversation, in the tone "who you are to tell what is fun to others and whst is not". You'd be impervious to any argument stating that dealing with far beyound someone understanding and requiring countless hours of digging into difficult math is not fun even for those who thinks it should be fun, as they presumably, loves everything STEM.

> We can continue this pointless conversation

Yeah, if you see it as pointless, then I prefer not to. Have a enjoyable day!

Re: I were 17, I'd learn how to build LLMs from scratch

#145

Post by jhatemyjob »

koe123 wrote: post #31

While knowledge is always great, I would encourage people not to seek advice from successful people like this (survivorship bias). Moreover I am not sure it is even good advice? Would you advise a 17 y.o. to learn how transistors work or how to code (i.e. is LLM training the right level in the stack)? LLM training, a discipline where relevant work is already out of reach for 99.999% of budgets really as essential as…

I just don't think he realizes how saturated it got over the years. Or maybe he knows at a conscious level, but not subconsciously.

In 2000 (his era), it would have been really smart to study the source of Linux or Apache. Would have paid dividends over decades. Cuz that knowledge was so rare. The number of people hacking on LLMs now dwarfs the number of people hacking on web servers 30 years ago, by several orders of magnitude.

And if you turn back the clock even more, I mean just even having access to a computer, let alone owning one, would have put you at a massive advantage.

I don't know what to call it. The pioneers should be respected obviously, but at the same time you need to understand that for them, the game wasn't nearly as played out as it is now.

I just don't think you can afford to be dicking around with LLMs like you could afford to dick around with random Linux distros 20 years ago. Too many people willing to do it for free these days.

You don't wanna end up being the 2030 equivalent of a certain SNES emulator developer, or maintainer of a package manager for jailbroken iPhones, I mean the list goes on and on. Being a hacker doesn't automatically give you a path to being rich, or even making a decent living. It hasn't been that way for a while.

Re: I were 17, I'd learn how to build LLMs from scratch

#146

Post by [deleted] »

Zylokloto wrote: post #99
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

We finetune LLMs. Small ones like Gemma 4 for semantic tasks. There are plenty of areas were we need people to do this for insurances, banks etc. AI/ML exists on many levels.

[deleted]

Re: I were 17, I'd learn how to build LLMs from scratch

#147

Post by latexr »

eptcyka wrote: post #18
utopiah wrote: post #9

... and it would be totally pointless. I mean first that is already what plenty of 17yo are actually doing, because that is what they do at school or in parascholar activities. There are already countless of such tutorials where you can do that in an afternoon. The pointless part though is precisely why Amazon and others are hunting for rare books, all the low hanging fruits have been picked already so just training…

Why would 17 year old do something that only brings them money? I do not think Mr Graham here is advocating for the path that makes most money as a result of learning how to train a model. I assume that tinkering and learning about LLMs is what enterprising 17 year olds will do to discover ways they can get a competitive edge or further the SotA with their insights further down the line.

No wonder the state of everything when 17 year olds are getting pressured to be “enterprising” and “get a competitive edge”. How about learning to be empathetic, respecting your fellow humans, caring for the place you live in, enhancing the lives of others? We shouldn’t be teaching 17 year olds to be greedy, selfish, self-aggrandising blowhards like Zuckerberg, Musk, and Graham. They are not good role models for the future of humanity.

Re: I were 17, I'd learn how to build LLMs from scratch

#148

Post by mdp2021 »

livinglist wrote: post #142
mdp2021 wrote: post #137

Earlier quoted context omitted.

I also did, and had written code for/on other mobile devices before, and did write a large number of even very ambitious software for other mobile devices later. I do not see any past constructive experience as a waste of time.

Did you write anything for Symbian or PalmOS before WP?

Yes, absolutely. (Actually, "this friend of mine" goes back to programmable calculators ;) - my very first versatile palm computing devices.)

Re: I were 17, I'd learn how to build LLMs from scratch

#149

Post by yobbo »

epolanski wrote: post #107
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

+1, I have a friend, math PhD that's been working on ML research 5+ years in London yet he has not been able to find any position. The only jobs that he found he was highly over qualified or paid very little. In any case, it doesn't look like there's this crazy rush to hire all ML talent, even the one that understand the math and technology deeply.

As always and everywhere, it's who you know (and who knows you) that matters.

Re: I were 17, I'd learn how to build LLMs from scratch

#150

Post by wy35 »

epolanski wrote: post #115
wiseowise wrote: post #110

[flagged]

Is going to university really that good of advice nowadays? Everybody goes to college nowadays and the average white collar has lots of debt and relatively minor financial benefits over a skilled trade worker. edit: woah, so many people insulted by that. In my bubble and friends, me and another friend are the only people that make very good money compared to non-graduates. Plenty of others opened their shops, went in…

It depends on the university and your goals, I suppose.

Many people are surprised to learn how affordable elite colleges are if you genuinely need financial aid. I had no idea -- was pleasantly surprised when my alma mater took over 80% off of my tuition.

Post reply on HN