Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

271–280 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#271

Post by sourcepluck »

abeppu wrote: post #163
sourcepluck wrote: post #104

Earlier quoted context omitted.

Searle's Chinese Room Argument springs to mind: https://plato.stanford.edu/entries/chinese-room/ The idea that "human-like" behaviour will lead to self-awareness is both unproven (it can't be proven until it happens) and impossible to disprove (like Russell's teapot). Yet, one common assumption of many people running these companies or investing in them, or of some developers investing their time in these technologie…

> The idea that "human-like" behaviour will lead to self-awareness is both unproven (it can't be proven until it happens) and impossible to disprove (like Russell's teapot). I think Searle's view was that: - while it cannot be dis-_proven_, the Chinese Room argument was meant to provide reasons against believing it - the "it can't be proven until it happens" part is misunderstanding: you won't know if it happens beca…

The short version of this is that I don't disagree with your interpretation of Searle, and my paragraphs immediately following the link weren't meant to be a direct description of his point with the Chinese Room thought experiment.

> while it cannot be dis-_proven_, the Chinese Room argument was meant to provide reasons against believing it

Yes, like Russell's teapot. I also think that's what Searle means.

> the "it can't be proven until it happens" part is misunderstanding: you won't know if it happens because the objective, externally available attributes don't indicate whether self-awareness (or indeed awareness at all) is present

Yes, agreed, I believe that's what Searle is saying too. I think I was maybe being ambiguous here - I wanted to say that even if you forgave the AI maximalists for ignoring all relevant philosophical work, the notion that "appearing human-like" inevitably tends to what would actually be "consciousness" or "intelligence" is more than a big claim.

Searle goes further, and I'm not sure if I follow him all the way, personally, but it's a side point.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#272

Post by Zopieux »

aniforprez wrote: post #160
namaria wrote: post #149

Earlier quoted context omitted.

This is a bad comparison. Intelligence didn't appear in some human brain. Intelligence appeared in a planetary ecosystem.

Also it took hundreds of millions of years to get here. We're basically living in an atomic sliver on the fabric of history. Expecting AGI with 5 of years of scraping at most 30 years of online data and the minuscule fraction of what has been written over the past couple of thousand years was always a pie-in-the-sky dream to raise obscene amounts of money.

I can't believe this still needs to be laid down years after the start of the GPT hype. Still, thanks!

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#273

Post by creativenolo »

crystal_revenge wrote: post #127
LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

I don't think we've even started to get the most value out of current gen LLMs. For starters very few people are even looking at sampling which is a major part of the model performance. The theory behind these models so aggressively lags the engineering that I suspect there are many major improvements to be found just by understanding a bit more about what these models are really doing and making re-designs based on…

> holding on of building things waiting for "that next big update", but there a so many small, annoying tasks that can be easily automated.

Also we only hear / see the examples that are meant to scale. Startups typically offer up something transformative, ready to soak up a segment of a market. And that’s hard with the current state of LLMs. When you try their offerings, it’s underwhelming. But there is richer, more nuanced hard to reach fruits that are extremely interesting - but it’s not clear where they’d scale in and of themselves.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#275

Post by bob1029 »

whazor wrote: post #167
pluc wrote: post #43

They've simply run out of data to use to fabricate legitimate-looking guesses. They can't create anything that doesn't already exist.

But a LLM can certainly make up a lot information that never existed before.

I strongly believe this gets into an information theoretical constraint akin to why perpetual motion machines don't work.

In theory, yes you could generate an unlimited amount of data for the models, but how much of it is unique or valuable information? If you were to compress all this generated training data using a really good algorithm, how much actual information remains?

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#277

Post by rafaelmn »

iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

>There is another emerging paradigm which is still small(er) scale but showing remarkable results. That's full multi-modal training with embodied agents (aka robots). 1x, Figure, Physical Intelligence, Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different.

Tesla is selling this view for almost a decade now in self-driving - how their car fleet feeding training data is going to make them leaders in the area. I don't find it convincing anymore

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#278

Post by warkdarrior »

sincerecook wrote: post #165
iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

> That's full multi-modal training with embodied agents (aka robots). 1x, Figure, Physical Intelligence, Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different. Cool, but we already have robots doing this in 2d space (aka self driving cars) that struggle not to kill people. How is adding a third dimension going to help? People are just refusing…

> Cool, but we already have robots doing this in 2d space (aka self driving cars) that struggle not to kill people. How is adding a third dimension going to help?

If we have robots that operate in 3D, they'll be able to kill you not only from behind or from the side, but also from above. So that's progress!

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#279

Post by portaouflop »

Der_Einzige wrote: post #225
dmd wrote: post #166

Earlier quoted context omitted.

Pretty much, yes. My job is pretty fun; it mostly entails things like "take this horrible file workflow some research assistant came up with while high 15 years ago and turn it into a newer horrible file format a NEW research assistant came up with (also while high) 3 years ago" - and automate this in our data processing pipeline.

Due to WFH, the weed laws where tech workers live, and the fast tolerance building of cannabis in the body - I estimate that 10% of all code written by west coast tech workers is done “while high” and that estimate is likely low.

Do tech workers write better or worse code while high ?

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#280

Post by fire_lake »

bbor wrote: post #228
LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

Great question. Im very confident in my answer, even though it’s in the minority here: we’re not even close to exhausting the potential. Imagine that our current capabilities are like the Model-T. There remains many improvements to be made upon this passenger transportation product, with RAG being a great common theme among them. People will use chatbots with much more permissive interfaces instead of clicking throug…

> Hinton, a Turing award winner, gave up $$$ to doom-say full time

This is a hint of something but a weak argument. Smart people are wrong all the time.

Post reply on HN