221–230 of 525 posts
I am curious though how something like Moore's Law relates to this. Yes, model architectures will deal with complexity better and the amount of data helps as well. There must be a relation between technology innovation and cost which alludes to effectiveness. Innovation in computation, model architecture, quality of data, etc.
Earlier quoted context omitted.
>How are they at human performance? Because it performs at least average human level (mostly well above average) on basically every task it's given. "Invest something new" is a nonsensical benchmark for human level intelligence. The vast majority of people have never and will never invent anything new. If your general intelligence test can't be passed by a good chunk of humanity then it's not a general intelligence t…
Yeah these intelligence tests are not very good. I would argue some programmers do in fact invent something new. Not all of them, but some. Perhaps 10%. Second the point is not whether everyone is by profession an inventor but whether most people can be inventors. And to a degree they can be. I think you underestimate that by a large margin. You can lock people in a room and give them a problem to solve and they will…
Lol Okay
>And to a degree they can be. I think you underestimate that by a large margin.
Do i? Because i'm not the one making unverifiable claims here.
>You can lock people in a room and give them a problem to solve and they will invent a lot if they have the time to do it.
If you say so
Earlier quoted context omitted.
On the Lex Fridman podcast, he pretty much admitted he's not an AI (per se) and isn't the most excited about the tech (as he could be).
> he pretty much admitted he's not an AI Yeah, I also had a hunch he wasn't an AI. (I assume you meant "AI researcher" there :)) All joking aside, I wonder how that's affecting company morale or their ability to attract top researchers. I know if I was a top AI researcher, I'd probably rather work at a company where the CEO was an expert in the field (all else being equal).
Eh. Altman has a financial incentive to lie and obfuscate about what it takes to train a model like GPT-4 and beyond, so his word is basically worthless.
Earlier quoted context omitted.
Does this mean we’ve reached the next AI winter? This is as good as it gets for quite a long time? Honest question :) perhaps this will postpone everyone’s fears about the singularity…
No. Winter means people have lost interest in the research. If anything successes in ChatGPT etc will be motivation for continued efforts.
Related reading: https://dynomight.net/scaling/ In short it seems like virtually all of the improvement in future AI models will come from better algorithms, with bigger and better data a distant second, and more parameters a distant third. Of course, this claim is itself internally inconsistent in that it assumes that new algorithms won't alter the returns to scale from more data or parameters. Maybe a more precise…
Better data is still critical, even if bigger data isn't. The linked article emphasizes this.
I strongly believe the next generation of models will be based upon spiking neural concepts wherein action potentials are lazily-evaluated throughout the network (i.e. event-driven). There are a few neuron models that can be modified (at some expense to fidelity) in order to tolerate arbitrary delays between simulation ticks. Using actual latency between neurons as a means of encoding information seems absolutely ess…
Earlier quoted context omitted.
> he pretty much admitted he's not an AI Yeah, I also had a hunch he wasn't an AI. (I assume you meant "AI researcher" there :)) All joking aside, I wonder how that's affecting company morale or their ability to attract top researchers. I know if I was a top AI researcher, I'd probably rather work at a company where the CEO was an expert in the field (all else being equal).
I feel most CEOs are not top of their field but rather people who can take a vision and run with it.