Hope this would be a constant reminder that brute force can only get one that far, though it may still be useful when it is. With lots of intuition gained, it’s time to ponder things a bit more deeply.
OpenAI, Google and Anthropic are struggling to build more advanced AI
331–340 of 622 posts
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#332Earlier quoted context omitted.
> Except our users experience the failed attempts (LLM replies that are wrong, even when backed by RAG) and it's incredibly hard to hide those from them. This has been my team's experience (and frustration) as well, and has led us to look at using LLMs for classifying / structuring, but not entrusting an LLM with making a decision based on things like a database schema or business logic. I think the technology and to…
> "let's only allow the LLM to do things we know it is rock-solid at." Even this is insanely hard in my opinion. The one thing that you would assume LLM to excel at is spelling and grammar checking for the English language, but even the top model (GPT-4o) can be insanely stupid/unpredictable at times. Take the following example from my tool: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... 5 models are…
Those Apple engineers stated in a very clear tone:
- every time a different result is produced.
- no reasoning capabilities were categorically determined.
So this is it. If you want LLM - brace for different results and if this is okay for your application (say it’s about speech or non-critical commands) then off you are.
Otherwise simply forget this approach, and particularly when you need reproducible discreet results.
I don’t think it gets any better than that and nothing so far implicated it will (with this particular approach to AGI or whatever the wet dream is)
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#333It's interesting the way things turned out so far with LLMs, especially from the perspective of a software engineer.
We are trained to keep a certain skepticism when we see software which appears to be working because, ultimately, the only question we care about is "Does it meet user requirements?" and this is usually framed in terms of users achieving certain goals.
So it's interesting that when AI came along, we threw caution to the wind and started treating it like a silver bullet... Without asking the question of whether it was applicable to this goal or that goal...
I don't think anyone could have anticipated that we could have an AI which could produce perfect sentences, faster than a human, better than a human but which could not reason. It appears to reason very well, better than most people, yet it doesn't actually reason. You only notice this once you ask it to accomplish a task. After a while, you can feel how it lacks willpower. It puts into perspective the importance of willpower when it comes to getting things done.
In any case, LLMs bring us closer to understanding some big philosophical questions surrounding intelligence and consciousness.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#334Earlier quoted context omitted.
Because it can't apply any reasoning that hasn't already been done and written into its training set. As soon as you ask it novel questions it falls apart. The big LLM vendors like OpenAI are playing whack-a-mole on these novel questions when they go viral on social media, all in a desperate bid to hide this fatal flaw. The Emperor has no clothes.
>As soon as you ask it novel questions it falls apart. What do you mean by novel? Almost all sentences it is prompted on are brand new and it mostly responds sensibly. Surely there's some generalization going on.
Novel as in requiring novel reasoning to sort out. One of the classic ways to expose the issue is to take a common puzzle and introduce irrelevant details and perhaps trivialize the solution. LLMs pattern match on the general form of the puzzle and then wander down the garden path to an incorrect solution that no human would fall for.
The sort of generalization these things can do seems to mostly be the trivial sort: substitution.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#335Hope this would be a constant reminder that brute force can only get one that far, though it may still be useful when it is. With lots of intuition gained, it’s time to ponder things a bit more deeply.
Maybe, if you want to relearn the bitter lesson.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#336Earlier quoted context omitted.
But a LLM can certainly make up a lot information that never existed before.
I strongly believe this gets into an information theoretical constraint akin to why perpetual motion machines don't work. In theory, yes you could generate an unlimited amount of data for the models, but how much of it is unique or valuable information? If you were to compress all this generated training data using a really good algorithm, how much actual information remains?
I make a lot of shitposts, how much of that is valuable information? Arguably not much. I doubt information value is a good way to estimate inteligence because most people's daily ramblings would grade them useless.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#337Earlier quoted context omitted.
> "let's only allow the LLM to do things we know it is rock-solid at." Even this is insanely hard in my opinion. The one thing that you would assume LLM to excel at is spelling and grammar checking for the English language, but even the top model (GPT-4o) can be insanely stupid/unpredictable at times. Take the following example from my tool: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... 5 models are…
Those Apple engineers stated in a very clear tone: - every time a different result is produced. - no reasoning capabilities were categorically determined. So this is it. If you want LLM - brace for different results and if this is okay for your application (say it’s about speech or non-critical commands) then off you are. Otherwise simply forget this approach, and particularly when you need reproducible discreet resu…
> Those Apple engineers
Which Apple engineers? Yours is the only reference to the company in this comment section or in the article.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#338Earlier quoted context omitted.
Whether self awareness is a requirement for AGI definitely gets more into the Philosophy department than the Computer Science department. I'm not sure everyone even agrees on what AGI is, but a common test is "can it do what humans can". For example, in this article it says it can't do coding exercises outside the training set. That would definitely be on the "AGI checklist". Basically doing anything that is outside…
> Whether self awareness is a requirement for AGI definitely gets more into the Philosophy department than the Computer Science department. Depends on how you define “self awareness” but knowing that it doesn't know something instead of hallucinating a plausible-but-wrong is already self awareness of some kind. And it's both highly valuable and beyond current tech's capability.
Good point!
I'm wondering wether it would count, if one would extend it with an external program, that gives it feedback during inference (by another prompt) about the correctness of it's output.
I guess it wouldn't, because these RAG tools kind of do that and i heard no one calling those self aware.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#339It seems obvious to me that Common Crawl plus Github public repositories have more than an enough data to train an AI that is as good as any programmer (at tasks not requiring knowledge of non-public codebases or non-public domain knowledge). So the problem is more in the algorithm.
I think just reading the code wouldn't make you a good programmer, you'd need to "read" the anti-code, ie what doesn't work, by trial and error. Models overconfidence that their code will work often leads them to fail in practice.
AlphaGo got better by playing against itself. I wonder if the pathway forward here is to essentially do the same with coding. Feed it some arbitrary SRS documents - have it attempt to develop them including full code coverage testing. Have it also take on roles of QA, stakeholders, red-team security researchers, and users who are all aggressively trying to find edge cases and point out everything wrong with the application. Have it keep iterating and learn from the findings. Keep feeding it new novel SRSs until the number off attempts/iterations necessary to get a quality product out the other side drops to some acceptable number.
Re: OpenAI, Google and Anthropic are struggling to build more advanced AI
#340Earlier quoted context omitted.
> "let's only allow the LLM to do things we know it is rock-solid at." Even this is insanely hard in my opinion. The one thing that you would assume LLM to excel at is spelling and grammar checking for the English language, but even the top model (GPT-4o) can be insanely stupid/unpredictable at times. Take the following example from my tool: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... 5 models are…
Those Apple engineers stated in a very clear tone: - every time a different result is produced. - no reasoning capabilities were categorically determined. So this is it. If you want LLM - brace for different results and if this is okay for your application (say it’s about speech or non-critical commands) then off you are. Otherwise simply forget this approach, and particularly when you need reproducible discreet resu…
(for reference: https://arxiv.org/pdf/2410.05229 )