Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

511–520 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#511

Post by danielbln »

aniforprez wrote: post #160
namaria wrote: post #149

Earlier quoted context omitted.

This is a bad comparison. Intelligence didn't appear in some human brain. Intelligence appeared in a planetary ecosystem.

Also it took hundreds of millions of years to get here. We're basically living in an atomic sliver on the fabric of history. Expecting AGI with 5 of years of scraping at most 30 years of online data and the minuscule fraction of what has been written over the past couple of thousand years was always a pie-in-the-sky dream to raise obscene amounts of money.

We built planes, which works quite differently from birds, in the span of what, 100 years? I think we've long left evolution behind when building machines, thinking or otherwise, so I'm not sure why the powerful but inefficient evolutionary process is held to some gold standard here.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#512

Post by kreyenborgi »

dr_kiszonka wrote: post #407
cruffle_duffle wrote: post #301

Earlier quoted context omitted.

In my experience, the reality of long context windows doesn’t live up to the hype. When you’re iterating on something, whether it's code, text, or any document, you end up with multiple versions layered in the context. Every time you revise, those earlier versions stick around, even though only the latest one is the "most correct". What gets pushed out isn’t the last version of the document itself (since it’s FIFO),…

I like the idea of context editing and threaded conversations. I think I have seen some alternative UIs on HN that support branching.

gptel does this: https://github.com/karthink/gptel/?tab=readme-ov-file#extra-...

Here are the docs for an example of how it can look: https://news.ycombinator.com/item?id=42039895

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#513

Post by danielbln »

sssilver wrote: post #343

One thing that makes the established AIs less ideal for my (programming) use-case is that the technologies I use quickly evolve past whatever the published models "learn". On the other hand, a lot of these frameworks and languages have relatively decent and detailed documentation. Perhaps this is a naive question, but why can't I as a user just purchase "AI software" that comes with a large pre-trained model to which…

Pretraining or even post-training is cumbersome, complex and expensive. What is easy and cheap is in-context learning, which is why I just pull in the documentation I need the LLM to know about into the LLM's context.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#514

Post by weweersdfsd »

yk wrote: post #221
LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

To a certain extent I think we get a better understanding what llms can do, and my estimation for the next ten years is more like best UI ever rather than llms will replace humanity. Now best UI ever is something that can certainly deliver a lot of value, 80% of all buttons in a car should be replaced by actually good voice control, and I think that is were we are going to see a lot of very interesting applications:…

I think buttons should not be replaced, but rather augmented with voice control. I certainly want to be able to adjust air conditioning or use my washing machine while listening music or having otherwise noisy environemnt.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#515

Post by disgruntledphd2 »

fullstackchris wrote: post #476
ericmcer wrote: post #220

Earlier quoted context omitted.

I have tried a few AI coding tools and always found them impressive but I don't really need something to autocomplete obvious code cases. Is there an AI tool that can ingest a codebase and locate code based on abstract questions? Like: "I need to invalidate customers who haven't logged in for a month" and it can locate things like relevant DB tables, controllers, services, etc.

Cursor (Claude behind the scenes) can do that, however as always, your mileage may vary. I tried building a whole codebase inspector, essentially what you are referring to with Gemini's 2 million token context window but had troubles with their API when the payload got large. Just 500 error with no additional info so...

I've played around with Claude and larger docs and it's honestly been a bit of a crapshoot, it feels like only some of the information gets into the prompt as the doc gets larger. They're great for converting PDF tables to more usable formats though.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#516

Post by visarga »

eslaught wrote: post #504
surrTurr wrote: post #498

Earlier quoted context omitted.

i think you underestimate the amount of data a driver experiences in a single 5 minute drive

I never get this argument. I've seen a deer on a road maybe once. I've seen a rabbit on a road zero times. But I know what to do if I see one. Is that because the "video" of my perception has many "frames"? Even if that's true at some level, I think it's massively missing the point. Yeah, so I saw that one deer from a lot of angles. But current AI training is like the equivalent of taking every deer that has ever bee…

You might personally have seen a deer just once, but human evolution, and animal evolution prior to that have practiced this skill a lot. AI doesn't have the advantage of evolutionary priors baked in, so it needs explicit walking through many combinations to infer its structure from data, and is remarkably efficient. GPT-4 'only' trained on the amount of language that 30,000 humans use in their lifetime.

But we have seen from AlphaGo that when training data is extensive, it can rediscover strategy on its own and even surpass us. It's not inherently worse than human learning.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#518

Post by Barrin92 »

JumpCrisscross wrote: post #192
alangibson wrote: post #161

Earlier quoted context omitted.

I think you're playing a different game than the Sam Altmans of the world. The level of investment and profit they are looking for can only be justified by creating AGI. The > 100 P/E ratios we are already seeing can't be justified by something as quotidian as the exceptionally good productivity tools you're talking about.

> level of investment and profit they are looking for can only be justified by creating AGI What are you basing this on? IT outsourcing is a $500+ billion industry. If OpenAI et al can run even a 10% margin, that business alone justifies their valuation.

if the AI business is a bit more mundane than Altman thinks and there's diminishing returns the market is going to be even more commodified than it already is and you're not going to make any margins or somehow own the entire market. That's already the case, Anthropic works about as well, there's other companies a few months behind, open source is like a year behind.

That's literally Zucc's entire play, in 5 years this stuff is going to be so abundant you'll get access to good enough models for pennies and he'll win because he can slap ads on it, and openAI sits there on its gargantuan research costs.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#519

Post by fooker »

vbezhenar wrote: post #153
dr_dshiv wrote: post #145

Earlier quoted context omitted.

> I've started to realize that many people have been holding on of building things waiting for "that next big update" I’ve noticed this too — I’ve been calling it intellectual deflation. By analogy, why spend now when it may be cheaper in a month? Why do the work now, when it will be easier in a month?

Why optimise software today, when tomorrow Intel will release CPU with 2x performance?

If Intel could do that, they would be the one with a 3 trillion market cap. Not Nvidia.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#520

Post by zaptrem »

hatefulmoron wrote: post #255
fallat wrote: post #142

What a stupid piece. We are making leaps every 6 months still. Tell me this when there are no developments for 3 years.

I'm curious, what was the leap after GPT-4? What about the leaps after that, given a leap every 6 months?

O1, new Sonnet, all the music models and video models, the voice models like 4o, etc.
Post reply on HN