Earlier quoted context omitted.
I'm not an expert in ML theory, but from my perspective, I think more compute would cause better fitting for a model, but possibly overfitting if more parameters are not added. For some problem space, adding additional parameters introduces another degree of freedom, which should allow a larger domain for the inputs mapping to outputs (more answers to questions). And if we define AGI as a network that can answer ques…
We know more parameters improves generalization so that is why it's unclear to me what they mean by more training flops.
Rippling Co-founder tweets notes of convo with Sama then deletes
11–13 of 13 posts
Re: Rippling Co-founder tweets notes of convo with Sama then deletes
#12Carmack is doing an AGI startup; maybe he can offer a definition of what it is exactly his company will attempt to deliver and how we can recognise it when/if it does?
Re: Rippling Co-founder tweets notes of convo with Sama then deletes
#13All these tech "influencers" and CEOs have such laughable opinions on AGI. First, they cannot really seem to give a formal definition of AGI (are we talking about some network that can problem solving in n > 2 problems or defining and creating consciousness)? Second, it's clear many of these people are just bandwagoning onto the hype. They started with crypto, then maybe some VR, and now are moving to AI/AGI coat-tai…
While what you are saying is mostly true for most of the "influencers", this guy was a 2x icpc world finalist back in the day, so probably knows a thing or two. Having said that, I read through the excerpt and it is actually outright wrong on some stuff.