Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

431–440 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#431

Post by soheil »

sincerecook wrote: post #165
iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

> That's full multi-modal training with embodied agents (aka robots). 1x, Figure, Physical Intelligence, Tesla are all making rapid progress on functionality which is definitely beyond frontier LLMs because it is distinctly different. Cool, but we already have robots doing this in 2d space (aka self driving cars) that struggle not to kill people. How is adding a third dimension going to help? People are just refusing…

How is self-driving a 2D problem when you navigate a 3D world? (please do visit hilly San Francisco sometime) not to mention additional dimensions like depth, velocity vectors among others.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#432

Post by Roger-L »

LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

Yes, I personally think that training an "all-knowing" artificial intelligence is not as good as training n "experts" in a single field.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#433

Post by Bjorkbat »

Havoc wrote: post #429

The new Gemini just hit some good benchmarks. This smells like it’s mostly based on OAI having a bit of bad luck with next model rather than a fundamental slowdown / barrier. They literally just made a decent sized leap with o1

Not meeting expectations != not better than the previous models.

The Information reporting was a bit more clear on this. Orion is better than GPT-4, it's just that they were expecting a leap in capabilities comparable to what we saw going from GPT-3 to GPT-4. In other words, they were expecting essentially a GPT-5, and Orion wasn't that good.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#434

Post by MBCook »

kristianp wrote: post #387
sdesol wrote: post #248

Earlier quoted context omitted.

> "let's only allow the LLM to do things we know it is rock-solid at." Even this is insanely hard in my opinion. The one thing that you would assume LLM to excel at is spelling and grammar checking for the English language, but even the top model (GPT-4o) can be insanely stupid/unpredictable at times. Take the following example from my tool: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... 5 models are…

> It keeps complaining that GitHub is spelled like Github, when it isn't I feel like this is unfair. That's the only thing it got wrong? But we want it to pass all of our evals, even ones the perhaps a dictionary would be better at solving? Or even an LLM augmented with a dictionary.

Does it matter?

As a user I want it to be right, even if that contradicts the normal rules of the language.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#435

Post by echelon »

woopwoop wrote: post #423
jmward01 wrote: post #175

Every negative headline I see about AI hitting a wall or being over-hyped makes me think of the early 2000's with that new thing the 'internet' (yes, I know the internet is a lot older than that). There is little doubt in my mind that ten years from now nearly every aspect of life will be deeply connected to AI just like the internet took over everything in the late 90's and early 2000's and is now deeply connected t…

That's funny, because to me these headlines about how deep learning is over-hyped and hitting the wall remind me of headlines from ten years ago about how... deep learning is over-hyped and hitting the wall.

That was before people could generate animation and music.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#436

Post by sky2224 »

LASR wrote: post #115

Question for the group here: do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs? I lead a team exploring cutting edge LLM applications and end-user features. It's my intuition from experience that we have a LONG way to go. GPT-4o / Claude 3.5 are the go-to models for my team. Every combination of technical investment + LLMs yields a new list of potential…

> do we honestly feel like we've exhausted the options for delivering value on top of the current generation of LLMs?

I know we absolutely have not, but I think we have reached the limit in terms of the Chatbot experience that ChatGPT is. For some reason the industry keeps trying to force the chatbot interface to do literally everything to the point that we now have inflated roles like "Prompt Engineers". This is to say that people suck at knowing what they want off the rip, and LLMs can't help with that if they're not integrated in technology in such a way where a solid foundation is built to allow the models to generate good output.

LLMs and other big data models have incredible potential for things like security, medicine, and the power industry to name a few fields. I mean I was recently talking with a professor about his research in applying deep learning to address growing security concerns in cars on the road.

The application is far from reaching the ceiling.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#437

Post by fsndz »

Sam Altman might be wrong then?

Learning from data is not enough; there is a need for the kind of system-two thinking we humans develop as we grow. It is difficult to see how deep learning and backpropagation alone will help us model that. For tasks where providing enough data is sufficient to cover 95% of cases, deep learning will continue to be useful in the form of 'data-driven knowledge automation.' For other cases, the road will be much more challenging. https://www.lycee.ai/blog/why-sam-altman-is-wrong

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#438

Post by MBCook »

deegles wrote: post #263
crystal_revenge wrote: post #127

Earlier quoted context omitted.

I don't think we've even started to get the most value out of current gen LLMs. For starters very few people are even looking at sampling which is a major part of the model performance. The theory behind these models so aggressively lags the engineering that I suspect there are many major improvements to be found just by understanding a bit more about what these models are really doing and making re-designs based on…

My big question is what is being done about hallucination? Without a solution it's a giant footgun.

CAN anything be done? At a very low level they’re basically designed to hallucinate text until it looks like something you’re asking for.

It works disturbingly well. But because it doesn’t have any actual intrinsic knowledge it has no way of knowing when it made a “good“ hallucination versus a “bad“ one.

I’m sure people are working at piling things on top to try and influence what gets generated or catch and move away from errors errors other layers spot… but how much effort and resources will be needed to make it “good enough“ that people don’t worry about this anymore.

In my mind the core problem is people are trying to use these for things they’re unsuitable for. Asking fact-based questions is asking for trouble. There isn’t much of a wrong answer if you wanted to generate a bedtime story or a bunch of test data that looks sort of like an example you give it.

If you ask it to find law cases on a specific point you’re going to raise a judge‘s ire, as many have already found.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#439

Post by nutanc »

Let's keep aside the hype. Let's define more advanced AI. With current architectures, this basically means better copying machines(don't mean this in a bad way and don't want a debate on this. This is just my opinion based on my usage). Basically everything in the Internet has been crammed into the weights and the companies are finding it hard to do two things:

1. Find more data.

2. Make the weights capture the data and reproduce.

In that sense we have reached a limit. So in my opinion we can do a couple of things.

1. App developers can understand the limits and build within the limits.

2. Researchers can take insights from these large models and build better AI systems with new architectures. It's ok to say transformers have reached a limit.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#440

Post by throw310822 »

portaouflop wrote: post #279
Der_Einzige wrote: post #225

Earlier quoted context omitted.

Due to WFH, the weed laws where tech workers live, and the fast tolerance building of cannabis in the body - I estimate that 10% of all code written by west coast tech workers is done “while high” and that estimate is likely low.

Do tech workers write better or worse code while high ?

Should copilot be renamed to "designated driver"?
Post reply on HN