Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don't know how you get here from “predict the next word”

grumpy-economist.com

Post reply on HN
Print view View original

221–230 of 275 posts

Re: I don't know how you get here from “predict the next word”

#221

Post by BobaFloutist »

selridge wrote: post #91
compass_copium wrote: post #79

Earlier quoted context omitted.

But the programming language has explicitly laid out rules. It was not trained on those sets of rules, but it was trained on many trillions of lines of code. It has a map of how programs work, and an explanation of this new language. It's using training data and data it's fed to generate that result.

What doesn't that explain tho? What behavior would you need to see for that explanation to no longer hold? Because it seems like it explains too much.

I don't know how you'd prompt this, but if there was a clean example of an A.I. coming up with an idea that's completely novel in more than details, it would be compelling evidence that these next-token predictors have some weird emergent properties that don't necessarily follow from intricate, sophisticated webs of token-prediction.

E.g. "What might be a room-temperature superconductor" -> "some plausible iteration on existing high-temperature superconductors based on our current understanding of the underlying physics" would not be outside how we currently understand them.

"What might be a room-temperature superconductor?" -> "some completely outlandish material that nobody has studied before and, when examined, seems to have higher temperature superconducting than we would predict" would provoke some serious questions.

A fun experiment I've heard suggested is training a model on all scientific understanding just up to some counterintuitive quantum leap in scientific understanding, say, Einstein's theory of relativity, and then seeing if you can prompt it to "discover" or "invent" said leap, without explicitly telling it what to look for. This would of course be pretty hard to prove, but if you could get it to work on a local model, publish the training set and parameters so that anyone can replicate it on their own machine, that could be pretty darn compelling.

Re: I don't know how you get here from “predict the next word”

#222

Post by pianom4n »

deaux wrote: post #154
pianom4n wrote: post #67

Earlier quoted context omitted.

Do you think the submitter intended this as an ad? His post history doesn't seem suspicious. Or do you think article's author wrote this an an ad? He's a reputable academic who seems impressed with an AI tool he used and is honestly sharing his thoughts. For reference he published the 80 page inflation mini-book 2 weeks ago asking for feedback: https://www.grumpy-economist.com/p/inflation

> Or do you think article's author wrote this an an ad? He's a reputable academic who seems impressed with an AI tool he used and is honestly sharing his thoughts. Ghuntley used to be reputable on here, then the crypto money looked too juicy.

Are you seriously comparing a random hacker to a lifelong academic for their odds of becoming a crypto shill?

Re: I don't know how you get here from “predict the next word”

#223

Post by compass_copium »

selridge wrote: post #91
compass_copium wrote: post #79

Earlier quoted context omitted.

But the programming language has explicitly laid out rules. It was not trained on those sets of rules, but it was trained on many trillions of lines of code. It has a map of how programs work, and an explanation of this new language. It's using training data and data it's fed to generate that result.

What doesn't that explain tho? What behavior would you need to see for that explanation to no longer hold? Because it seems like it explains too much.

Programs are fundamentally lists of instructions. LLMs are very good at building these lists. That it performs well when you say "Build a list you've seen before, but do it in a slightly different way this time. Here's the exact way I want you to do it." is not surprising. I would honestly be surprised if it couldn't do it.

As the other commenter suggested, a genuinely novel scientific idea would be surprising. A new style of art (think Picasso or Pollack coming along), not just an iteration on Ghibli, would be surprising. That's actual creativity.

Re: I don't know how you get here from “predict the next word”

#224

Post by margalabargala »

TuringTest wrote: post #145
317070 wrote: post #119

Earlier quoted context omitted.

As an expert in the field: this is exactly right. LLMs are trained to do whole book prediction, at training time we throw in whole books at the time. It's only when sampling we do one or a few tokens at the time.

Isn't that the same as compressing the whole book , in a special differential format that compares how the text looks from any given point before and after?

Everything is the same as everything else. It's all just hydrogen and time mixed together.

Re: I don't know how you get here from “predict the next word”

#225

Post by jjmarr »

aktau wrote: post #213
jjmarr wrote: post #81

Earlier quoted context omitted.

I created a code review pipeline at work with a similar tradeoff and we found the cost is worth it. Time is a non-issue. We could run Claude on our code and call it a day, but we have hundreds of style, safety, etc rules on a very large C++ codebase with intricate behaviour (cooperative multitasking be fun). So we run dozens of parallel CLI agents that can review the code in excruciating detail. This has completely r…

Any LLM-based code review tooling I've tried has been lackluster (most comments not too helpful). Prose review is usually better. > So we run dozens of parallel CLI agents that can review the code in excruciating detail. This has completely replaced human code review for anything that isn't functional correctness but is near the same order of magnitude of price. Much better than humans and beats every commercial tool…

You need human alignment on what constitutes a "good" comment. That means consistent rules.

Otherwise, some people feel review is too harsh, other people feel it is not harsh enough. AI does not fix inconsistent expectations.

> But how does one separate the good comments from the bad comments?

If the AI took a valid interpretation of the coding guidelines, it is a legitimate comment. If the AI is being overly pedantic, it is a documentation bug and we change the rules.

Re: I don't know how you get here from “predict the next word”

#226

Post by selridge »

compass_copium wrote: post #223
selridge wrote: post #91

Earlier quoted context omitted.

What doesn't that explain tho? What behavior would you need to see for that explanation to no longer hold? Because it seems like it explains too much.

Programs are fundamentally lists of instructions. LLMs are very good at building these lists. That it performs well when you say "Build a list you've seen before, but do it in a slightly different way this time. Here's the exact way I want you to do it." is not surprising. I would honestly be surprised if it couldn't do it. As the other commenter suggested, a genuinely novel scientific idea would be surprising. A new…

>I would honestly be surprised if it couldn't do it.

You'd be surprised if an LLM couldn't write *any* program?

Re: I don't know how you get here from “predict the next word”

#227

Post by jjmarr »

1718627440 wrote: post #202
grey-area wrote: post #196

Earlier quoted context omitted.

Well no, style is important too for humans when they read a codebase, so the LLMs the parent is running clearly have some value for them. They're not claiming LLMs solved every problem, just that they made life easier by taking care of busywork that humans would otherwise be doing. I think personally this is quite a good use for them - offering suggestions on PRs say, as long as humans still review them as well.

But isn't style already achievable by running e.g. GNU indent?

Some examples of complex transformations linters can't catch:

* Function names must start with a verb.

* Use standard algorithms instead of for loops.

* Refactor your code to use IIFEs to make variables constexpr.

The verb one is the best example. Since we work adjacent to hardware, people like creating functions on structs representing register state called "REGISTER_XYZ_FIELD_BIT_1()" and you can't tell if this gets the value of the first field bit or sets something called field bit to 1.

If you rename it to `getRegisterXyzFieldBit1()` or `setRegisterXyzFieldBitTo1()` at least it becomes clear what they're doing.

Re: I don't know how you get here from “predict the next word”

#228

Post by selridge »

WithinReason wrote: post #173
selridge wrote: post #69

Earlier quoted context omitted.

Yes. I’m saying that “it’s just in the training data” is a cognitive containment of these models which is incomplete. You can insist that’s what’s happening, but you’ll be left unable to explain what’s going on beyond truisms.

It's called "generalization": https://en.wikipedia.org/wiki/Generalization_(learning)

>"If A and B are separately in the training data, the model can provide a result when A and B occur in the input because the model has made a connection between A and B in the latent space."

This statement (The one I was replying to) is fundamentally unbounded. There's nothing that can't be explained as a combination of "A" and "B" in "training data" because practically speaking we can express anything as such where the combination only needs to be convex along some high-dimensional semantic surface. Add on to that my scare quotes around "training data" because very few people have any practical idea of what is or isn't in there, so we can just make claims strategically. Do we need to explain a success? It was in the training data. A failure, probably not in the training data. Will anyone call us on this transparent farce? Not usually, no.

If a statement can--at will--explain everything and nothing, what's it worth?

Re: I don't know how you get here from “predict the next word”

#229

Post by fc417fc802 »

qsera wrote: post #200
fc417fc802 wrote: post #194

Earlier quoted context omitted.

And where is this objective metric for consciousness? Last I checked we didn't even have a sensible definition for it. It seems to me you're just kicking the can. Setting that issue aside. While I certainly don't believe LLMs to be conscious (an entirely subjective and arbitrary take on my part I admit) I don't see any reason that concepts such as "intelligence" and "understanding" should require it. When considering…

>humans it seems to me they are results based and highly contextual (ie largely arbitrary). Is that right? It seems that we generally say that "the computer is programmed to do", instead of "the computer understand" or "the computer knows", even if the programmed computer can produce the same result as a human who does it.

Of course we don't say that. You can't ask the (traditionally) programmed computer a freeform question and get a sensible answer back. We tried that for going on 50 years and it never really worked. (The highest achievement that comes to mind is answering jeopardy questions.)

You can very carefully construct a query in a dedicated language, debug that query, and get useful results back. But that's clearly just a human using a tool, not a machine exhibiting understanding or general knowledge.

Meanwhile you can ask a multi-billion parameter LLM a freeform question in ~any human language and it can produce a coherent and meaningful response. It can one shot pieces of code. Track down bugs based on compiler error messages. It might not (yet) be human level in many cases but to get hung up on that is to miss the point.

Re: I don't know how you get here from “predict the next word”

#230

Post by fc417fc802 »

qsera wrote: post #184
gkbrk wrote: post #182

Earlier quoted context omitted.

A microphone + ADC is hearing though, that's the whole reason we even produce microphones. So that our electronics can hear sound.

So according to you when can you qualify something as capable of hearing 1. Vibrate according input to the sound, is that hearing? 2. Generate electrical signals according to the sound, is that hearing? 3. Amplify electrical signal, does we cross the hearing mark? 4. Record the signal to a cassete tape (or use an ADC -> mp3), are we hearing yet? 5. Play it back through a speaker. Sure, we should be hearing now! At wh…

You can reduce the human auditory process to a similar mechanical list. At which specific point would you say a human is hearing?

You've fallen into the trap of human exceptionalism but you don't seem to be aware of that fact. Are you a substance dualist or not?

Post reply on HN