Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

691–700 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#691

Post by tech_ken »

s1artibartfast wrote: post #624
tech_ken wrote: post #550

Earlier quoted context omitted.

Developing an intuition about high dimensional systems is pretty different from understanding the character of some specific point on a 1e9+ dimensional manifold of parameters, in my professional opinion (setting aside all the degrees of freedom that come from the structure of the thing). Sure one can understand generic principles like the curse of dimensionality, but truly groking how an LLM works is basically an op…

Seems like a silly goalpost. As you say, there's a lot of value to unlock with understanding the generic principles, and I would add specific application. Lots of people are building fantastic Tools or pulling down million dollar salaries without groking the precise representation of a single weight.

> Lots of people are building fantastic Tools or pulling down million dollar salaries without groking the precise representation of a single weight.

I'll borrow my response from a far greater applied mathematician than myself: https://www.youtube.com/watch?v=_oNgyUAEv0Q

Re: I were 17, I'd learn how to build LLMs from scratch

#692

Post by mda »

ThrowawayR2 wrote: post #650
mda wrote: post #288

Earlier quoted context omitted.

I think it is more about people are a bit sick of filthy rich people giving this kind of advice. I would also not read anything he preaches.

Hacker News is a Y Combinator (which PG co-founded) sponsored website whose original code was created by PG and whose guidelines were originally formed by PG. For those who are "sick"(sic) of rich people, VCs, or capitalism in general, you've come to the wrong place.

I stand by what I said, are you gatekeeping HN now? This is a tech news site, I don't care if he once was a co founder.

Re: I were 17, I'd learn how to build LLMs from scratch

#693

Post by nostrademons »

Yann LeCun's response is interesting. I've heard that the transformer architecture underlying LLMs is actually really effective for robotics, but progress is a lot slower here because instead of just scraping the web and Anna's Archive, you have to outfit workers with motion-capture suits that record their every move, and then have them physically do enough jobs that you can gather good training data on how the job is done. This is a little challenging when no blue-collar worker wants to be replaced by a robot.

Long-term there's probably a lot more potential, and much bigger markets, in robotics than in LLMs, but it will really take the long term to get there.

Re: I were 17, I'd learn how to build LLMs from scratch

#694

Post by [deleted] »

mda wrote: post #692
ThrowawayR2 wrote: post #650

Earlier quoted context omitted.

Hacker News is a Y Combinator (which PG co-founded) sponsored website whose original code was created by PG and whose guidelines were originally formed by PG. For those who are "sick"(sic) of rich people, VCs, or capitalism in general, you've come to the wrong place.

I stand by what I said, are you gatekeeping HN now? This is a tech news site, I don't care if he once was a co founder.

[deleted]

Re: I were 17, I'd learn how to build LLMs from scratch

#695

Post by [deleted] »

mda wrote: post #692
ThrowawayR2 wrote: post #650

Earlier quoted context omitted.

Hacker News is a Y Combinator (which PG co-founded) sponsored website whose original code was created by PG and whose guidelines were originally formed by PG. For those who are "sick"(sic) of rich people, VCs, or capitalism in general, you've come to the wrong place.

I stand by what I said, are you gatekeeping HN now? This is a tech news site, I don't care if he once was a co founder.

[deleted]

Re: I were 17, I'd learn how to build LLMs from scratch

#696

Post by mdp2021 »

mdp2021 wrote: post #139
yapyap wrote: post #89

This is always such a nonsense question-answer thing, asking a person who already succeeded what they would do if they were young. Even worse when they ask themselves.

Support your idea with arguments.

Edit: not "support", I meant: give it a justification. For example, to give the "succeed" part some sense, instead of suggesting assumptions.

Re: I were 17, I'd learn how to build LLMs from scratch

#698

Post by shelled »

chris_va wrote: post #23

I am kind of amazed how negative the comments are here, especially on HN. Learning to hack something together in high school using the latest technology (vacuum tubes, radios, microprocessors, web/javascript) has been a common theme in the tech world for generations. With LLMs and online tutorials, this isn't even a difficult suggestion. Do people think learning new tech is somehow wasted effort?

I have also noticed a general drop in fawning over and deification of 'made a billion, must be a visionary' figures, and a receding tolerance for every sundry kind of hot take and proclamation. Mustn't be related.

Re: I were 17, I'd learn how to build LLMs from scratch

#700

Post by Myrmornis »

loveparade wrote: post #77

Horrible advice. This may have been good advice 10 years ago, but not today. There are no positions for people who "kind of understand how toy LLMs work" because so many engineers do these days. Most of the real LLM optimization work is at the edge of research and highly proprietary and not something you could ever do without infra that costs millions. But of course, 10 years ago this wasn't obvious.

Very silly reply. If you learn how toy LLMs work at 17 don't you think that would help you at university and grad school? There's plenty of time to go into the private sector workforce later.
Post reply on HN