This was predicted. Ai isnt going to get any better.
OpenAI, Google and Anthropic are struggling to build more advanced AI
261–270 of 622 posts
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#262Not sure if related or not, Sam Altman, ~12hrs ago: there is no wall [1] [1] https://x.com/sama/status/1856941766915641580
Breaking: Man says enigmatic thing to sustain hype and flow of money into his business.
Ditto- I have a feeling the investors in his latest 2.3 quintillion dollar series Z round wouldn't be as happy if he'd have tweeted "there is a wall"
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#263Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…
I don't think we've even started to get the most value out of current gen LLMs. For starters very few people are even looking at sampling which is a major part of the model performance. The theory behind these models so aggressively lags the engineering that I suspect there are many major improvements to be found just by understanding a bit more about what these models are really doing and making re-designs based on…
My big question is what is being done about hallucination? Without a solution it's a giant footgun.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#264Earlier quoted context omitted.
Whether self awareness is a requirement for AGI definitely gets more into the Philosophy department than the Computer Science department. I'm not sure everyone even agrees on what AGI is, but a common test is "can it do what humans can". For example, in this article it says it can't do coding exercises outside the training set. That would definitely be on the "AGI checklist". Basically doing anything that is outside…
I feel the test for AGI should be more like: "go find a job and earn money" or "start a profitable business" or "pick a bachelor degree and complete it", etc.
Can most humans do that? Find a job and earn money, probably. The other two? Not so much.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#265Earlier quoted context omitted.
Yeah I keep thinking this - how is Nvidia worth $3.5Trillion for making code autocomplete for coders
Nvidia was not the best example. They get to moon in the case that any AI exponential hits. Most others have less of a wide probability distribution.
I'm not sure about that. NVIDIA seems to stay in a dominant position as long as the race to AI remains intact, but the path to it seems unsure. They are selling a general purpose AI-accelerator that supports the unknown path.
Once massively useful AI has been achieved, or it's been determined that LLMs are it, then it becomes a race to the bottom as GOOG/MSFT/AMZN/META/etc design/deploy more specialized accelerators to deliver this final form solution as cheaply as possible.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#266Earlier quoted context omitted.
I see takes like this all the time and its so confusing. Why does knowing how things work under the hood make you think its not on the path towards AGI? What was lacking in the Attention paper that tells you AGI won't be built on LLMs? If its the supposed statistical nature of LLMs (itself a questionable claim), why does statistics seem so deflating to you?
> Why does knowing how things work under the hood make you think its not on the path towards AGI? Because I had no idea how these were built until I read the paper, so couldn’t really tell what sort of tree they’re barking up. The failure-modes of LLMs and ways prompts affect output made a ton more sense after I updated my mental model with that information.
But we don't know how human thinking works. Suppose for a second that it could be represented as a series of matrix math. What series of operations are missing from the process that would make you think it was doing some fascimile of thinking?
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#267Earlier quoted context omitted.
I see takes like this all the time and its so confusing. Why does knowing how things work under the hood make you think its not on the path towards AGI? What was lacking in the Attention paper that tells you AGI won't be built on LLMs? If its the supposed statistical nature of LLMs (itself a questionable claim), why does statistics seem so deflating to you?
> Why does knowing how things work under the hood make you think its not on the path towards AGI? Because I had no idea how these were built until I read the paper, so couldn’t really tell what sort of tree they’re barking up. The failure-modes of LLMs and ways prompts affect output made a ton more sense after I updated my mental model with that information.
Right, but its behavior didn't change after you learned more about it. Why should that cause you to update in the negative? Why does learning how it work not update you in the direction of "so that's how thinking works!" rather than, "clearly its not doing any thinking"? Why do you have a preconception of how thinking works such that learning about the internals of LLMs updates you against it thinking?
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#268Earlier quoted context omitted.
Yeah they're the shovel sellers of this particular goldrush. Most other businesses trying to actually use LLMs are the riskier ones, including OpenAI, IMO (though OpenAI is perhaps the least risky due to brand recognition).
Or they become the Webvan/pets.com of the bubble.
Nvidia is more likely to become CSCO or INTC but as far as I can tell, that's still a few years off - unless ofcourse there is weakness in broader economy that accelerates the pressure on investors.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#269Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…
I don't think we've even started to get the most value out of current gen LLMs. For starters very few people are even looking at sampling which is a major part of the model performance. The theory behind these models so aggressively lags the engineering that I suspect there are many major improvements to be found just by understanding a bit more about what these models are really doing and making re-designs based on…
Great & motivational comment. Any pointers on where to start playing with the internals and sampling?
Doesn’t need to be comprehensive, I just don’t know where to jump off from.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#270Earlier quoted context omitted.
No. The scaling laws may be dead. Does this mean the end of LLM advances? Absolutely not. There are many different ways to improve LLM capabilities. Everyone was mostly focused on the scaling laws because that worked extremely well (actually surprising most of the researchers). But if you're keeping an eye on the scientific papers coming out about AI, you've seen the astounding amount of research going on with some v…
Scaling laws are not dead. The number of people predicting death of Moore's law doubles every two years. - Jim Keller https://www.youtube.com/live/oIG9ztQw2Gc?si=oaK2zjSBxq2N-zj1...
There are way too many personal definitions of what "Moore's Law" even is to have a discussion without deciding on a shared definition before hand.
But Goodhart's law; "When a measure becomes a target, it ceases to be a good measure"
Directly applies here, Moore's Law was used to set long term plans at semiconductor companies, and Moore didn't have empirical evidence it was even going to continue.
If you say, arbitrarily decide CPU, or worse, single core performance as your measurement, it hasn't held for well over a decade.
If you hold minimum feature size without regard to cost, it is still holding.
What you want to prove usually dictates what interpretation you make.
That said, the scaling law is still unknown, but you can game it as much as you want in similar ways.
GPT4 was already hinting at an asymptote on MMLU, but the question is if it is valid for real work etc...
Time will tell, but I am seeing far less optimism from my sources, but that is just anecdotal.