411–420 of 725 posts
Building an LLM from scratch has a hard split between a tutorial project you can complete in a weekend (that's useless for actual usage) and then a solid 1km high brick wall if you want to create anything actually useful from scratch.
Modified open models have a very active community around them, without the need to look much further than Hugging Face.
I will never understand this mentality, to let yourself be so dependent on something you don't understand at all is to live like a child. But it is very common. I doubt too many 17 year olds will bother even trying to understand what an LLM is, let alone build one from scratch.
Earlier quoted context omitted.
> They are highly adaptable even without fine tuning, in-context learning is still superior to fine tuning in most cases also. Good luck relying on in-context learning for a 600M LLM. > The actual adapter training is automated and put behind simple APIs. That's like saying it's worthless to learn infra because you can use serverless instead… > All the frontier labs have nearly identical model personalities, capabilit…
There is no reason to believe that generalist models need to be transformer (or transformer hybrids) it's just that people are risk averse and not spending their resources on the many scaled risky bets needed to explore the possibilities. There is more there.
Earlier quoted context omitted.
AI is the subtrate the future runs on. And so I think the idea is more to understand tomorrow ... from first principles. In the late 80's, as a teenager, I learned x86 assembly and C because that was the only way to squeeze out enough juice from my shitty CGA (and later VGA) card to programm the games/graphics that interested me. I haven't written assembly in years. But whatever I did in my career: it helped me and g…
> AI is the subtrate the future runs on. Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures. So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI…
At 75% replacement of a worker we would already have huge job losses as each individual would be doing what several before did.
The only alleviation would be the creation of new equivalently paid jobs, which is no better than a hypothesis right now.
There's this dilemma where in theory there's a ton of demand for engineers that can do real LLM machine-learning, but in practice there are very few available positions and entrepreneurship opportunities. The reality is that an incredibly small minority of companies in the world do any real training or optimisation. It's unnecessary and inefficient for most purposes unless you are fully dedicated to being an LLM comp…
There are high school students competing in contests that cover parts of the (Math) theory behind AI. A lot of high school research programs are integrating AI with other things and complex mathematical models…
To me, this is bizarre as Calculus is barely taught in high schools (and likely poorly).
Don’t get me wrong, these kids certainly aren’t the usual lot.
Yet, I really wonder if they know the fundamentals. Do they even understand derivatives or just memorized the rule for polynomials? Can they even explain what a transistor is?
Normal curriculum takes 5 years to go from Algebra I to Calculus. Real Analysis, Linear Systems, etc. are fundamentals taught only in college…
Feels like too many are trying sprint before even learning to walk.
A lot of people here are responding to the message but not to the meaning. It would be a good idea for young people to deeply know how these programs work. Not so that they can spend their career building them, but so that they can approach the next class of problems we'll all start trying to solve, with intuition all the way down to the weights and underlying mathematics. And also, to develop a healthy intuition of…
Earlier quoted context omitted.
If someone writes a browser in their teens, they will probably learn more about the web than if they were just writing web pages
They'll learn so much more that won't transfer to as many job opportunities. For ex, say more about C++ and less about cutting edge CSS (because modern browser tech is an ocean). I suppose they might luck into other adjacent or unrelated roles with the same skills.
If anyone reading this has this un-marketable skill, Carnegie robotics in Pittsburgh is hiring
carnegie-robotics.breezy.hr/p/2d85f5321cc7-software-engineer