I'd just build an agent, it's easier than you think and you'd learn a lot about the "magic" of LLMs.
I were 17, I'd learn how to build LLMs from scratch
271–280 of 725 posts
Re: I were 17, I'd learn how to build LLMs from scratch
#272A lot of people here are responding to the message but not to the meaning. It would be a good idea for young people to deeply know how these programs work. Not so that they can spend their career building them, but so that they can approach the next class of problems we'll all start trying to solve, with intuition all the way down to the weights and underlying mathematics. And also, to develop a healthy intuition of…
Depends, the assumption things are predictable always negatively affects both Market Bears and Bulls alike.
Indeed, if credulous folks look to the world expecting people to bestow success upon them... than the disillusionment with reality will hit their savings harder.
The Shrek movie market correction correlations are undeniably funny, and a new film is due July 2027. OpenAI may be going public in the next few months while still losing $2.25 for every $1 of customer revenue, and with 6 other firms sharing over $4Tn in debt disclosed to investors in a footnote.
There is only one direction things can go at the Peak of inflated expectations. Popcorn ready. =3
Re: I were 17, I'd learn how to build LLMs from scratch
#273Earlier quoted context omitted.
Yes. It's like looking at the (Apollo) moon rocket launch and then suggesting teenagers should learn to build rockets in their garages for the coming space age. It is viable as a toy project, but there are vanishingly few career opportunities.
It's like looking at the early internet and then suggesting teenagers should write browsers as their projects instead of webpages.
If someone writes a browser in their teens, they will probably learn more about the web than if they were just writing web pages
Re: I were 17, I'd learn how to build LLMs from scratch
#274A lot of people here are responding to the message but not to the meaning. It would be a good idea for young people to deeply know how these programs work. Not so that they can spend their career building them, but so that they can approach the next class of problems we'll all start trying to solve, with intuition all the way down to the weights and underlying mathematics. And also, to develop a healthy intuition of…
> "Build an OS" wasn't a common university project because we were all expected to go […] helps you deeply understand how to intuit building for a whole class of problems.
Sure, but it was for a specific degree with a syllabus that taught you the foundational knowledge. It was not expected from the law students to learn how to build one.
Re: I were 17, I'd learn how to build LLMs from scratch
#275[flagged]
Is going to university really that good of advice nowadays? Everybody goes to college nowadays and the average white collar has lots of debt and relatively minor financial benefits over a skilled trade worker. edit: woah, so many people insulted by that. In my bubble and friends, me and another friend are the only people that make very good money compared to non-graduates. Plenty of others opened their shops, went in…
> Is going to university really that good of advice nowadays?
Just some anecdata, but every single one of my university professors was quite bad, but they think that since they are the professor, that means they are smart and the expert.
Re: I were 17, I'd learn how to build LLMs from scratch
#276Earlier quoted context omitted.
Really? The latter was immediately useful to lots of people which is motivating, and it had a nice smooth learning curve (html -> js -> php -> databases -> apps -> backend). Learning HTML is the first step to learning how to make full blown apps. Making a browser at 17 is like trying to climb Everest as your first hike. The expected outcome is burnout and demotivating failure. At best you'll learn some C++ or Rust. 1…
17 is a weird age but ymmv. I left home at 16 alone to study abroad. I had tons of free time due to dorm curfews and such. Unfortunately, I was too poor to have a computer and the computers we had access to were completely locked up. (Naturally we waltzed past the locks to play some games but it was also under surveillance) Paradoxically I coded way more between ages 12-14, I regret my wasted late teens.
Me too:
ages 8-15: lots of great 8-bit computer fun.
ages 15-20: girls, booze, motorcycles.
20 onwards: get a PC, back to computers, realize how much I've been missing.
To be honest, judging by my own kids and their friends, late teens seem to generally be an era of hard to avoid stupidity.
Re: I were 17, I'd learn how to build LLMs from scratch
#277telling a 17 year old to get into tech right now is horrible advice, literally telling them to get at the back of a line with a better part of a million more experienced people in it.
Shaka, when the walls fell...
Re: I were 17, I'd learn how to build LLMs from scratch
#278There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…
Yes. It's like looking at the (Apollo) moon rocket launch and then suggesting teenagers should learn to build rockets in their garages for the coming space age. It is viable as a toy project, but there are vanishingly few career opportunities.
But he’s not saying that it’s a career opportunity.
It seems to me like he’s saying that doing this thing would be 1) fun and 2) a great way to become employable in the future. I don’t believe he’s saying that this project would be some kind of job training exercise.
Re: I were 17, I'd learn how to build LLMs from scratch
#279There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…
"Necessity is the mother of invention" - limited hardware has always forced people to find cleverer ways of doing more with less. Current models are clearly nowhere near the efficiency limit (the brain does vastly more with far less power).
> Current models are clearly nowhere near the efficiency limit (the brain does vastly more with far less power).
I think this is disingenuous. One could say that drones are nowhere the efficiency limit either: a bee can fly for hours on the energy contained in just a few milligrams of honey, while our best battery-powered drones can't stay airborne for more than 30 minutes. But comparing energy efficiency of electric/mechanical devices to their biological counterparts is not an apples-to-apples comparison. There's a world of difference between the energy storage and delivery mechanisms.
And as many have pointed out already in the siblings, it's not just about the compute but the access to petabytes of training data.
Re: I were 17, I'd learn how to build LLMs from scratch
#280Earlier quoted context omitted.
If you learnt to build a browser in 2000 you're probably doing well for yourself lol. Not a lot of demand, but also probably not a lot of supply.
There is an entire graveyward of browsers, almost all that didn't die are now forgotten.
If any were built by teenagers, I’d imagine those teenagers ended up with pretty good careers in technology?