Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I were 17, I'd learn how to build LLMs from scratch

twitter.com

Post reply on HN
Print view View original

251–260 of 725 posts

Re: I were 17, I'd learn how to build LLMs from scratch

#251

Post by ejanus »

embedding-shape wrote: post #114
epolanski wrote: post #107

Earlier quoted context omitted.

+1, I have a friend, math PhD that's been working on ML research 5+ years in London yet he has not been able to find any position. The only jobs that he found he was highly over qualified or paid very little. In any case, it doesn't look like there's this crazy rush to hire all ML talent, even the one that understand the math and technology deeply.

> +1, I have a friend, math PhD that's been working on ML research 5+ years in London yet he has not been able to find any position. Maybe people simply don't want math PhDs but something else? Since 1-2 years ago I started doing consulting/freelancing in the ML space, but more on the infrastructure, deployments and similar stuff, as a general purpose developer, and I have a waiting list of clients interested in more…

Congrats!

Is it possible to see some of your old works? Personal research?

Re: I were 17, I'd learn how to build LLMs from scratch

#252

Post by jgraham »

maciejzj wrote: post #226
Chance-Device wrote: post #206

I think a problem a lot of people are grappling with here is that due to LLMs and AI generally, it’s basically impossible to predict what the future will look like or what jobs will still be around. I’d probably say something like: do something you enjoy and seems like it might be useful, but accept that the pace of change may mean that whatever you study ends up being irrelevant. Whatever solution there ends up bein…

I would advise any somewhat ambitious 17 year old to avoid tech and get into healthcare if they can stomach human interactions and bodily fluids. Sure, it is not all sunshine and rainbows, but there will still be plenty of work helping people who are ill or elderly. Even in the worst-case economic scenario, medicine will be a more socially rewarding and stable life path.

This seems like a much more interesting question to me.

Telling other people's children what to do is easy and basically doesn't have any downside to being wrong. With your own children things are a bit different.

So: what are people here with school age children telling their own kids about the future? If their kids ask, what kind of careers would they encourage them to pursue, assuming they have the skills and interest?

When I was last in the Bay Area, maybe about a decade ago the bookshops were full of titles like "Python for Preschoolers" (I exaggerate, but only slightly). Clearly at the time a lot of people working in tech thought that cultivating an interest in programming was going to be the path to being a successful (by some metric) adult. Is that still the case?

Re: I were 17, I'd learn how to build LLMs from scratch

#253

Post by yoz-y »

mike_hearn wrote: post #184
amelius wrote: post #174

Earlier quoted context omitted.

Well, there's a lot more to learn from the former than the latter ...

Really? The latter was immediately useful to lots of people which is motivating, and it had a nice smooth learning curve (html -> js -> php -> databases -> apps -> backend). Learning HTML is the first step to learning how to make full blown apps. Making a browser at 17 is like trying to climb Everest as your first hike. The expected outcome is burnout and demotivating failure. At best you'll learn some C++ or Rust. 1…

17 is a weird age but ymmv. I left home at 16 alone to study abroad. I had tons of free time due to dorm curfews and such. Unfortunately, I was too poor to have a computer and the computers we had access to were completely locked up. (Naturally we waltzed past the locks to play some games but it was also under surveillance)

Paradoxically I coded way more between ages 12-14, I regret my wasted late teens.

Re: I were 17, I'd learn how to build LLMs from scratch

#255

Post by giancarlostoro »

yobbo wrote: post #140
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

Yes. It's like looking at the (Apollo) moon rocket launch and then suggesting teenagers should learn to build rockets in their garages for the coming space age. It is viable as a toy project, but there are vanishingly few career opportunities.

Building an LLM covers a lot of CS fundamentals and forces you to do lots of research in order to implement one, especially on more modest hardware.

Re: I were 17, I'd learn how to build LLMs from scratch

#256

Post by WhompingWindows »

chris_va wrote: post #23

I am kind of amazed how negative the comments are here, especially on HN. Learning to hack something together in high school using the latest technology (vacuum tubes, radios, microprocessors, web/javascript) has been a common theme in the tech world for generations. With LLMs and online tutorials, this isn't even a difficult suggestion. Do people think learning new tech is somehow wasted effort?

His writing was overrated: It's that simple. People now see his blog posts for what they are: decent blog posts.

Re: I were 17, I'd learn how to build LLMs from scratch

#257

Post by g3e0 »

oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

"Necessity is the mother of invention" - limited hardware has always forced people to find cleverer ways of doing more with less. Current models are clearly nowhere near the efficiency limit (the brain does vastly more with far less power).

Re: I were 17, I'd learn how to build LLMs from scratch

#258

Post by make_it_sure »

gf000 wrote: post #53
InsideOutSanta wrote: post #40

Earlier quoted context omitted.

> seek advice from unsuccessful people Intuitively, I would guess that they have a better grasp of what made them fail than successful people have of what made them succeed.

Yep, many successful people greatly underappreciate the effect of simple dumb luck in their lives. And often they just make up complex reasoning chains, even wholly believing them, that more of it was in their control/talent, etc. Nonetheless, there are many successful people I would gladly listen to for advice, though they are often successful in a different meaning than what venture capitalists would use (e.g. pare…

same do unsuccesful ones. They usually overappreciate luck of succesful people from what I see, as a way of coping, many say "that guy was just lucky, i was unlucky". If you dig deeper, that person wasn't just "unlucky", they lacked the persistence, work ethics and other qualities compared to other succesful ones.

Of course, there are degrees and exceptions on every side, but the coping mechanism is very strong among people.

Re: I were 17, I'd learn how to build LLMs from scratch

#260

Post by dosisking »

joshuakcockrell wrote: post #123
oersted wrote: post #13

There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…

This is like saying, “teens shouldn’t learn how to make their own game engine because no one is hiring for that.” You’re missing the point. Understanding how Unity works fundamentally makes you a better Unity dev.

> You’re missing the point. Understanding how Unity works fundamentally makes you a better Unity dev.

Writing your own game engine makes you realize that the Unity engine is not really that well written....

Post reply on HN