Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI, Google and Anthropic are struggling to build more advanced AI

bloomberg.com

Post reply on HN
Print view View original

231–240 of 622 posts

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#231

Post by xpe »

Animats wrote: post #100

"While the model was initially expected to significantly surpass previous versions of the technology behind ChatGPT, it fell short in key areas, particularly in answering coding questions outside its training data." Right. If you generate some code with ChatGPT, and then try to find similar code on the web, you usually will. Search for unusual phrases in comments and for variable names. Often, something from Stack Ov…

> LLMs do search and copy/paste with idiom translation and some transliteration.

In general, this is not a good description about what is happening inside an LLM. There is extensive literature on interpretability. It is complicated and still being worked out.

The commenter above might characterize the results they get in this way, but I would question the validity of that characterization, not to mention its generality.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#232

Post by luckydata »

Xenoamorphous wrote: post #159
norir wrote: post #80

Earlier quoted context omitted.

Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human: design a Turing complete programming language that is both human and machine readable and implement a self hosted compiler in this language that self compiles on existing hardware faster than any known language implementation that also self compiles. Additionally, for any syntactically or semantica…

> Here is an example of a task that I do not believe this generation of LLMs can ever do but that is possible for a human That’s possible for a highly intelligent, extensively trained, very small subset of humans.

does it mean people that can build languages and compilers are not humans? What is the point you're trying to make?

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#233

Post by vundercind »

nuancebydefault wrote: post #227
vundercind wrote: post #196

Earlier quoted context omitted.

There’s a distinction in behavior of a human and a Chinese room when things go wrong—when the rule book doesn’t cover the case at hand. I agree that a hypothetical perfectly-functioning Chinese room is, tautologically, impossible to distinguish from a real person who speaks Chinese, but that’s a thought experiment, not something that can actually exist. There’ll remain places where the “behavior” breaks down in ways…

I would argue maybe people also are not thinking but simply processing. It is known that most of what we do and feel goes automatically (subconsciously). But even more, maybe consciousness is an invention of our 'explaining self', maybe everything is automatic. I'm convinced this discussion is and will stay philosophical and will never get any conclusion.

Yeah, I’m not much interested in “what’s consciousness?” but I do think the automatic-versus-thinking distinction matters for understanding what LLMs do, and what we might expect them to be able to do, and when and to what degree we need to second-guess them.

A human doesn’t just confidently spew paragraphs legit-looking but entirely wrong crap, unless they’re trying to deceive or be funny—an LLM isn’t trying to do anything, though, there’s no motivation, it doesn’t like you (it doesn’t like—it doesn’t it, one might even say), sometimes it definitely will just give you a beautiful and elaborate lie simply because its rulebook told it to, in a context and in a way that would be extremely weird if a person did it.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#234

Post by amelius »

Miraste wrote: post #181
amelius wrote: post #118

Earlier quoted context omitted.

Yes, but literally anybody can do all those things. So while there will be many opportunities for new features (new ways of combining data), there will be few business opportunities.

HN always says this, and it's always wrong. A technical implementation that's easy, or readily available, does not mean that a successful company can't be built on it. Last year, people were saying "OpenAI doesn't have a moat." 15 years before that, they were saying "Dropbox is just a couple of chron jobs, it'll fail in a few months."

> HN always says this

The meaning here is different. What I'm saying is that big companies like OpenAI will always strive to make a generic AI, such that anyone can do basically anything using AI. The big companies therefore will indeed (like you say) have a profitable business, but few others will.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#235

Post by xpe »

pluc wrote: post #43

They've simply run out of data to use to fabricate legitimate-looking guesses. They can't create anything that doesn't already exist.

> They can't create anything that doesn't already exist.

I probably disagree, but I don't want to criticize my interpretation of this sentence. Can you make your claim more precise?

Here are some possible claims and refutations:

- Claim: An LLM cannot output a true claim that it has not already seen. Refutation: LLMs have been shown to do logical reasoning.

- Claim: An LLM cannot incorporate data that it hasn't been presented with. Refutation: This is an unfair standard. All forms of intelligence have to sense data from the world somehow.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#236

Post by fragmede »

knicholes wrote: post #108
iandanforth wrote: post #47

A few important things to remember here: The best engineering minds have been focused on scaling transformer pre and post training for the last three years because they had good reason to believe it would work, and it has up until now. Progress has been measured against benchmarks which are / were largely solvable with scale. There is another emerging paradigm which is still small(er) scale but showing remarkable res…

Once we've scraped the internet of its data, we need more data. Robots can take in video/audio data 24/7 and can be placed in your house to record this data by offering services like cooking/cleaning/folding laundry. Yeah, I'll pay $20k to have you record everything that happens in my house if I can stop doing dishes for five years!

People go and live in a house to get recorded 24/7, to be on tv, for far more asnine situations, for way less money.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#237

Post by xpe »

pluc wrote: post #43

They've simply run out of data to use to fabricate legitimate-looking guesses. They can't create anything that doesn't already exist.

> They've simply run out of data

Why do you think "they" have run out of data? First, to be clear, who do you mean by "they"? The world is filled with information sources (data aggregators for example), each available to some degree for some cost.

Don't forget to include data that humans provide while interacting with chatbots.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#238

Post by lokimedes »

BeefWellington wrote: post #204
drawnwren wrote: post #179

Earlier quoted context omitted.

Nvidia was not the best example. They get to moon in the case that any AI exponential hits. Most others have less of a wide probability distribution.

Yeah they're the shovel sellers of this particular goldrush. Most other businesses trying to actually use LLMs are the riskier ones, including OpenAI, IMO (though OpenAI is perhaps the least risky due to brand recognition).

Or they become the Webvan/pets.com of the bubble.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#239

Post by danjl »

cubefox wrote: post #10

It's very strange this got so few upvotes. The scoop by The Information a few days ago, which came to similar conclusions, was also ignored on HN. This is arguably rather big news.

There have been variations of this story going back several months now. It isn't really news. It is just building slowly.

Re: OpenAI, Google and Anthropic are struggling to build more advanced AI

#240

Post by danjl »

russellbeattie wrote: post #213

Go back a few decades and you'd see articles like this about CPU manufacturers struggling to improve processor speeds and questioning if Moore's Law was dead. Obviously those concerns were way overblown. That doesn't mean this article is irrelevant. It's good to know if LLM improvements are going to slow down a bit because the low hanging fruit has seemingly been picked. But in terms of the overall effect of AI and q…

Actually, Moore's Law has been dead for quite a few years now. Since we hit the power wall.
Post reply on HN