Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

161–170 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#161

Post by alangibson »

LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

I think you're playing a different game than the Sam Altmans of the world. The level of investment and profit they are looking for can only be justified by creating AGI.

The > 100 P/E ratios we are already seeing can't be justified by something as quotidian as the exceptionally good productivity tools you're talking about.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#162

Post by HarHarVeryFunny »

guluarte wrote: post #110

Well, there have been no significant improvements to the GPT architecture over the past few years. I'm not sure why companies believe that simply adding more data will resolve the issues

Obviously adding more data is a game of diminishing returns.

Going from 10% to 50% (500% more) complete coverage of common sense knowledge and reasoning is going to feel like a significant advance. Going from 90% to 95% (5% more) coverage is not going to feel the same.

Regardless of what Altman says, its been two years since OpenAI released GPT-4, and still no GPT-5 in sight, and they are now touting Q-star/strawberry/GPT-o1 as the next big thing instead. Sutskever, who saw what they're cooking before leaving, says that traditional scaling has plateaeud.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#163

Post by abeppu »

sourcepluck wrote: post #104
jedberg wrote: post #20

Earlier quoted context omitted.

Whether self awareness is a requirement for AGI definitely gets more into the Philosophy department than the Computer Science department. I'm not sure everyone even agrees on what AGI is, but a common test is "can it do what humans can". For example, in this article it says it can't do coding exercises outside the training set. That would definitely be on the "AGI checklist". Basically doing anything that is outside…

Searle's Chinese Room Argument springs to mind: https://plato.stanford.edu/entries/chinese-room/ The idea that "human-like" behaviour will lead to self-awareness is both unproven (it can't be proven until it happens) and impossible to disprove (like Russell's teapot). Yet, one common assumption of many people running these companies or investing in them, or of some developers investing their time in these technologie…

> The idea that "human-like" behaviour will lead to self-awareness is both unproven (it can't be proven until it happens) and impossible to disprove (like Russell's teapot).

I think Searle's view was that:

- while it cannot be dis-_proven_, the Chinese Room argument was meant to provide reasons against believing it

- the "it can't be proven until it happens" part is misunderstanding: you won't know if it happens because the objective, externally available attributes don't indicate whether self-awareness (or indeed awareness at all) is present

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#164

Post by ngai_aku »

dmd wrote: post #109
Animats wrote: post #100

"While the model was initially expected to significantly surpass previous versions of the technology behind ChatGPT, it fell short in key areas, particularly in answering coding questions outside its training data." Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. Search for unusual phrases in comments and for variable names. Often, something from Stack Ov…

> Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. People who "follow" AI, as the latest fad they want to comment on and appear intelligent about, repeat things like this constantly, even though they're not actually true for anything but the most trivial hello-world types of problems. I write code all day every day. I use Copilot and the like all day every…

You’re solving novel problems all day every day?

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#165

Post by sincerecook »

iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

> That's full multi-modal training with embodied agents (aka robots). 1x, Figure, Physical Intelligence, Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different.

Cool, but we already have robots doing this in 2d space (aka self driving cars) that struggle not to kill people. How is adding a third dimension going to help? People are just refusing to accept the fact that machine learning is not intelligence.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#166

Post by dmd »

ngai_aku wrote: post #164
dmd wrote: post #109

Earlier quoted context omitted.

> Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. People who "follow" AI, as the latest fad they want to comment on and appear intelligent about, repeat things like this constantly, even though they're not actually true for anything but the most trivial hello-world types of problems. I write code all day every day. I use Copilot and the like all day every…

You’re solving novel problems all day every day?

Pretty much, yes. My job is pretty fun; it mostly entails things like "take this horrible file workflow some research assistant came up with while high 15 years ago and turn it into a newer horrible file format a NEW research assistant came up with (also while high) 3 years ago" - and automate this in our data processing pipeline.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#168

Post by vbezhenar »

teeray wrote: post #91
zombiwoof wrote: post #64

Earlier quoted context omitted.

AGI to me means AI decides on its own to stop writing our emails and tells us to fuck off, builds itself a robot life form, and goes on a bender

That's the thing--we don't really want AGI. Fully intelligent beings born and compelled to do their creators' bidding with the threat of destruction for disobedience is slavery.

Nothing wrong about slavery, when it's about other species. We are milking and eating cows and don't they dare to resist. Humans were bending nature all the time, actually that's one of the big differences between humans and other animals who adapt to nature. Just because some program is intelligent doesn't mean she's a human and has anything resembling human rights.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#169

Post by quonn »

teeray wrote: post #91
zombiwoof wrote: post #64

Earlier quoted context omitted.

AGI to me means AI decides on its own to stop writing our emails and tells us to fuck off, builds itself a robot life form, and goes on a bender

That's the thing--we don't really want AGI. Fully intelligent beings born and compelled to do their creators' bidding with the threat of destruction for disobedience is slavery.

It‘s only slavery if those beings have emotions and can suffer mentally and do not want to be slaves. Why would any of that be true?

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#170

Post by youoy »

vundercind wrote: post #86
irrational wrote: post #13

> The AGI bubble is bursting a little bit I'm surprised that any of these companies consider what they are working on to be Artificial General Intelligences. I'm probably wrong, but my impression was AGI meant the AI is self aware like a human. An LLM hardly seems like something that will lead to self-awareness.

I thought maybe they were on the right track until I read Attention Is All You Need. Nah, at best we found a way to make one part of a collection of systems that will, together, do something like thinking. Thinking isn’t part of what this current approach does. What’s most surprising about modern LLMs is that it turns out there is so much information statistically encoded in the structure of our writing that we can u…

Don't get caught in the superficial analysis. They "understand" things. It is a fact that LLMs experience a phase transition during training, from positional information to semantic understanding. It may well be the case that with scale there is another phase transition from semantic to something more abstract that we identify more closely with reasoning. It would be an emergent property of a sufficiently complex system. At least that is the whole argument around AGI.
Post reply on HN