Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

111–120 of 376 posts

Re: Claude's Cycles [pdf]

#111

Post by vjerancrnjak »

ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

No. There is good signal in IMO gold medal performance.

These models actually learn distributed representations of nontrivial search algorithms.

A whole field of theorem provingaftwr decades of refinements couldn’t even win a medal yet 8B param models are doing it very well.

Attention mechanism, a bruteforce quadratic approach, combined with gradient descent is actually discovering very efficient distributed representations of algorithms. I don’t think they can even be extracted and made into an imperative program.

Re: Claude's Cycles [pdf]

#112

Post by DennisP »

wang_li wrote: post #78
zarzavat wrote: post #45

Earlier quoted context omitted.

I swear that AI could independently develop a cure for cancer and people would still say that it's not actually intelligent, just matrix multiplications giving a statistically probable answer! LLMs are at least designed to be intelligent. Our monkey brains have much less reason to be intelligent, since we only evolved to survive nature, not to understand it. We are at this moment extremely deep into what most people…

Last week I put "was val kilmer in heat" into the search box on my browser. The AI answer came back with "No, Val Kilmer was not in heat. Val Kilmer played Chris Shiherlis in the movie Heat but the film did not indicate that he was pregnant or in heat. His performance was nuanced and skilled and represents a high point of the film." I was not curious about whether he was pregnant. We are not only not close to human l…

Honestly, when I read your first sentence, given the lack of a capital H, my brain initially went the same direction the AI did. Then I realized what you meant but since I already went there, I might have made a similar response as a joke. For the sake of my ego I'm forced to reject your claim that this is evidence of stupidity.

Re: Claude's Cycles [pdf]

#113

Post by [deleted] »

bread-wood wrote: post #102
jdnier wrote: post #89

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon

Here I was assuming it was named after https://en.wikipedia.org/wiki/Claude_(alligator)

[deleted]

Re: Claude's Cycles [pdf]

#114

Post by beej71 »

xvector wrote: post #84
beej71 wrote: post #77

From my naive standpoint, LLMs like this seem to have some big strengths. One: possession of a superhuman expanse of knowledge. Two: making connections. Three: tireless trial and error. If you put those three things together, you end up with some cool stuff from time to time. Perhaps the proof of P!=NP is tied to an obscure connection that humans don't easily see due to individual lack of knowledge or predisposition…

This is why the whole "LLMs for mass surveillance" thing is scary imo.

Yeah, this is a dictator's dream scenario and hell for the citizens. Not only do you not want to get caught for saying something that The Great Leader disapproves of, but you're terrified that anything you say might get flagged by an AI.

Re: Claude's Cycles [pdf]

#115

Post by NitpickLawyer »

jdnier wrote: post #89

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon

Wait till you hear about nvidia and their GPU architecture naming scheme :)

Re: Claude's Cycles [pdf]

#116

Post by noslenwerdna »

ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

I find this kind of reduction silly.

All your brain is doing is bouncing atoms off each other, with some occasionally sticking together, how can it be really thinking?

See how silly it sounds?

Re: Claude's Cycles [pdf]

#117

Post by hmmmmmmmmmmmmmm »

cbovis wrote: post #93
throw310822 wrote: post #69

Earlier quoted context omitted.

> The training data If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?

The prompt is unique but the tokens aren't. Type "owejdpowejdojweodmwepiodnoiwendoinw welidn owindoiwendo nwoeidnweoind oiwnedoin" into ChatGPT and the response is "The text you sent appears to be random or corrupted and doesn’t form a clear question." because the prompt doesnt correlate to training data.

...? what is the response supposed to be here?

Re: Claude's Cycles [pdf]

#118

Post by hmmmmmmmmmmmmmm »

throw310822 wrote: post #69
qsera wrote: post #60

Earlier quoted context omitted.

>Probable given what? The training data.. >predicting what intelligence would do No, it just predict what the next word would be if an intelligent entity translated its thoughts to words. Because it is trained on the text that are written by intelligent entities. If it was trained on text written by someone who loves to rhyme, you would be getting all rhyming responses. It imitates the behavior -- in text -- of what…

> The training data If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?

Hamiltonian paths and previous work by Donald Knuth is more than likely in the training data.

Re: Claude's Cycles [pdf]

#120

Post by konne88 »

I didn't expect such a misleading intro from Knuth. It reads like Claude solved Knuth's math problem. In reality, Claude generated various example solution, and Knuth then manually generalized that to a formal proof. What Claude did is certainly useful, but it would have been nice to be clear about the scope of the contribution in the intro.
Post reply on HN