Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

301–310 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#301

Post by cruffle_duffle »

machiaweliczny wrote: post #224
LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

Long context is a scam. Claude is best but it’s still gets lost with longer context

In my experience, the reality of long context windows doesn’t live up to the hype. When you’re iterating on something, whether it's code, text, or any document, you end up with multiple versions layered in the context. Every time you revise, those earlier versions stick around, even though only the latest one is the "most correct".

What gets pushed out isn’t the last version of the document itself (since it’s FIFO), but the important parts of the conversation—things like the rationale, requirements, or any context the model needs to understand why it’s making changes. So, instead of being helpful, that extra capacity just gets filled with old, repetitive chunks that have to be processed every time, muddying up the output. This isn’t just an issue with code; it happens with any kind of document editing where you’re going back and forth, trying to refine the result.

Sometimes I feel the way to "resolve" this is to instead go back and edit some earlier portion of the chat to update it with the "new requirements" that I didn't even know I had until I walked down some rabbit hole. What I end up with is almost like a threaded conversation with the LLM. Like, I sometimes wish these LLM chatbots explicitly treated the conversion as if it were threaded. They do support basically my use case by letting you toggle between different edits to your prompts, but it is pretty limited and you cannot go back and edit things if you do some operations (eg: attach a file).

Speaking of context, it's also hard to know what things like ChatGPT add to it's context in the first place. Many of times I'll attach a file or something and discover it didn't "read" the file into it's context. Or I'll watch it fire up a python program it writes that does nothing but echo the file into it's context.

I think there is still a lot of untapped potential in strategically manipulating what gets placed into the context window at all. For example only present the LLM with the latest and greatest of a document and not all the previous revisions in the thread.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#302

Post by mvdtnz »

jmward01 wrote: post #175

Every negative headline I see about AI hitting a wall or being over-hyped makes me think of the early 2000's with that new thing the 'internet' (yes, I know the internet is a lot older than that). There is little doubt in my mind that ten years from now nearly every aspect of life will be deeply connected to AI just like the internet took over everything in the late 90's and early 2000's and is now deeply connected t…

Even if you're right (you're not) whatever "AI" looks like in 20+ years will have virtually nothing in common with these stupid statistical word generators.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#303

Post by JumpCrisscross »

HarHarVeryFunny wrote: post #295
JumpCrisscross wrote: post #282

Earlier quoted context omitted.

> you are missing a lot of "ifs" in that hypothetical The big one being I'm not assuming AGI. Low-level coding tasks, the kind frequently outsourced, are within the realm of being competitive with offshoring with known methods. My point is we don't need to assume AGI for these valuations to make sense.

Current AI coding assistants are best at writing functions or adding minor features to an existing code base. They are not agentic systems that can develop an entire solution from scratch given a specification, which in my experience is more typcical of the work that is being outsourced. AI is a tool, whose full-cycle productivity benefit seems questionable. It is not a replacement for a human.

> they are not agentic systems that can develop an entire solution from scratch given a specification, which in my experience is more typcical of the work that is being outsourced

If there is one domain where we're seeing tangible progress from AI, it's in working towards this goal. Difficult projects aren't in scope. But most tech, especially most tech branded IT, is not difficult. Everyone doesn't need an inventory or customer-complaint system designed from scratch. Current AI is good at cutting through that cruft.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#304

Post by czhu12 »

If it becomes obvious that LLM's have a more narrow set of use cases, rather than the all encompassing story we hear today, then I would bet that the LLM platforms (OpenAI, Anthropic, Google, etc) will start developing products to compete directly with applications that supposed to be building on top of them like Cursor, in an attempt to increase their revenue.

I wonder what this would mean for companies raising today on the premise of building on top of these platforms. Maybe the best ones get their ideas copied, reimplemented, and sold for cheaper?

We already kind of see this today with OpenAI's canvas and Claude artifacts. Perhaps they'll even start moving into Palantir's space and start having direct customer implementation teams.

It is becoming increasing obvious that LLM's are quickly becoming commoditized. Everyone is starting to approach the same limits in intelligence, and are finding it hard to carve out margin from competitors.

Most recently exhibited by the backlash at claude raising prices because their product is better. In any normal market, this would be totally expected, but people seemed shocked that anyone would charge more than the raw cost it would take to run the LLM itself.

https://x.com/ArtificialAnlys/status/1853598554570555614

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#305

Post by senko »

HarHarVeryFunny wrote: post #295
JumpCrisscross wrote: post #282

Earlier quoted context omitted.

> you are missing a lot of "ifs" in that hypothetical The big one being I'm not assuming AGI. Low-level coding tasks, the kind frequently outsourced, are within the realm of being competitive with offshoring with known methods. My point is we don't need to assume AGI for these valuations to make sense.

Current AI coding assistants are best at writing functions or adding minor features to an existing code base. They are not agentic systems that can develop an entire solution from scratch given a specification, which in my experience is more typcical of the work that is being outsourced. AI is a tool, whose full-cycle productivity benefit seems questionable. It is not a replacement for a human.

There are a number of agentic systems that can develop more complex solutions. Just a few off the top of my head: Pythagora, Devin, OpenHands, Fume, Tusk, Replit, Codebuff, Vly. I'm sure I've missed a bunch.

Are they good enough to replace a human yet? Questionable[0], but they are improving.

[0] You wouldn't believe how low the outsourcing contractors' quality can go. Easily surpassed by current AI systems :) That's a very low bar tho.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#306

Post by fragmede »

troupo wrote: post #87
famouswaffles wrote: post #27

Earlier quoted context omitted.

At this point, AGI means many different things to many different people but OpenAI defines it as "highly autonomous systems that outperform humans in most economically valuable tasks"

This definition suits OpenAI because it lets them claim AGI after reaching an arbitrary goal. LLMs already outperform humans in a huge variety of tasks. ML in general outperform humans in a large variety of tasks. Are all of them AGI? Doubtful.

It's not just marketing bullshit though. Microsoft is the counterparty to a contract with that claim. money changes hands when that's been achieved, so I expect if sama thinks he's hit it, but Microsoft does not, we'll see that get argued in a court of law.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#307

Post by kozikow »

crystal_revenge wrote: post #127
LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

I don't think we've even started to get the most value out of current gen LLMs. For starters very few people are even looking at sampling which is a major part of the model performance. The theory behind these models so aggressively lags the engineering that I suspect there are many major improvements to be found just by understanding a bit more about what these models are really doing and making re-designs based on…

> "The theory behind these models so aggressively lags the engineering"

The problem is that 99% of theories are hard to scale.

I am not an expert, as I work adjacent to this field, but I see the inverse - dumbing down theory to increase parallelism/scalability.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#308

Post by delusional »

dmd wrote: post #166
ngai_aku wrote: post #164

Earlier quoted context omitted.

You’re solving novel problems all day every day?

Pretty much, yes. My job is pretty fun; it mostly entails things like "take this horrible file workflow some research assistant came up with while high 15 years ago and turn it into a newer horrible file format a NEW research assistant came up with (also while high) 3 years ago" - and automate this in our data processing pipeline.

If I understand that correctly you're converting file formats? That's not exactly "novel"

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#309

Post by imiric »

superjose wrote: post #293

I'm more on the camp that these techs don't need to be perfect, but they need to be practical enough. And I think the latter is good enough for us to do exciting things.

How practical can they be when current flagship models generate incorrect responses more than 50% of the time[1]?

This might be acceptable for amusing us with fiction and art, and for filling the internet with even more spam and propaganda, but would you trust them to write reliable code, drive your car or control any critical machinery?

The truly exciting things are still out of reach, yet we just might be at the Peak of Inflated Expectations to see it now.

[1]: https://openai.com/index/introducing-simpleqa/

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#310

Post by jmward01 »

JohnMakin wrote: post #256
jmward01 wrote: post #175

Every negative headline I see about AI hitting a wall or being over-hyped makes me think of the early 2000's with that new thing the 'internet' (yes, I know the internet is a lot older than that). There is little doubt in my mind that ten years from now nearly every aspect of life will be deeply connected to AI just like the internet took over everything in the late 90's and early 2000's and is now deeply connected t…

It's strange to me that's your takeaway. The reason that the internet was overhyped in the 2000's is because it was and also heavily overvalued. It took a massive correction and seriously disruptive bubble burst to break the delusion and move on to something more sustainable.

I disagree that it was over hyped. It has transformed our society so much that I would argue it was vastly under-hyped. Sure, there were a lot of silly companies that sprang up and went away because they weren't sound, but so much of the modern economy is based on the internet that it is hard to say any business isn't somehow internet related today. You would be hard pressed to find any business anywhere that doesn't at least have a social media account. If 2000 was over-hyping things I just don't see it.
Post reply on HN