Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regression to the Mean: on LLMs and the quiet death of the new

rruxandra.github.io

Post reply on HN
Print view View original

61–70 of 103 posts

Re: Regression to the Mean: on LLMs and the quiet death of the new

#61

Post by memoriyato3 »

krona wrote: post #57
kolinko wrote: post #54

This is a recurring sentiment but flawed, I think. First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it. Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepte…

> They construct space between the points and extrapolate outside of the points. They don't. They interpolate between the points on a manifold.

so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM?

is the proof to the Riehmann hypothesis also somewhere on the manifold and we just need to prod the LLM with the right prompt so it locates the point?

Re: Regression to the Mean: on LLMs and the quiet death of the new

#62

Post by golly_ned »

Schlagbohrer wrote: post #43

"Offer it something it has never seen, and it doesn't light up. It corrects you. To a system built to predict the expected, the genuinely new is indistinguishable from a mistake. The pushback is soft, and constant: Did you mean: the familiar thing, offered in place of yours." Ouch, that's scary to think of. Also why does it read like its written by ChatGPT?

Even if something wasn't written by ChatGPT, I'm supposing ChatGPT's influence on writing has been so strong that (1) any typical reader of text on the internet will have ingested a lot of ChatGPT writing habits in their own writing, and (2) that any reader of ChatGPT text is so habituated to those writing habits that even non-ChatGPT generated text appears similar to ChatGPT generated text.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#63

Post by ux266478 »

AgentMatt wrote: post #18

I agree with the basic premise: that when using LLMs, they will tend towards some mean when resolving ambiguity. There's a very interesting opportunity here for a deeper investigation. - What does "regression to the mean" actually mean in practice when the LLM is conditioned on a possibly large amount of context? - How does this perceived regression to the mean affect the result in different applications? When implem…

LLMs might be the least interesting of the statistical models for creative purposes, they're kind of nasty to work with. Creating low rank adaptations is slow and expensive because the models are so fat, and meddling with the inference flow is a much more explosive game of cat and mouse. You can tell Flux where colors, shapes, textures, etc. should live on a canvas. Trying to wrangle an LLM into "spatially/temporally" arranging text in accordance with a writing style is a nightmare and a half. It's complicated, I can't really explain it well. Probably because I still haven't been able to put in the time to figure out the "grain" of pure text transformers. I can tell you that they're very hard to work with, though.

The simple fact of the matter is that these things are best viewed as a computer implementing a continuous form of computation. If you can understand the following words: "the function between function 1 and function 2 is function 1.5" and can imagine that this is a process with "infinite" descent, where you can continue to pull functions halfway between other functions trivially, that's a pretty great mental model to have. To that end, using them is like operating a big, complicated radio setup. Or a huge collection of synthesizers and filters. You're essentially tuning in to a structure. It can't help you figure out what structure you should reach for, or what you should do with that structure.

The creativity doesn't come from the tool itself. If you were only capable of having a mediocre body of work before, they aren't going to help you after. There is no removing human brilliance from the equation. AI is immensely exciting for what it promises, there is something genuinely new and interesting here, but it's not a crutch for the untalented. That's how it's getting sold, and it's having a massive disservice done to it. What we have is a genuinely new space to pioneer in. The interesting stuff and compelling art isn't going to be found at the end of a however-many-word positive prompt, but a symphony of total control over the model itself. Drawing 50 original works for the purpose of fine-tuning a diffusion model for a single project, and only as one particular component of the fully customized inference workflow, which will be scrapped when the project is completed. That's what artistic usage of statistical models looks like. The construction of a specific program for a specific purpose. Anything else is fantasy. The model can't give you that purpose.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#64

Post by queenkjuul »

AgentMatt wrote: post #18

I agree with the basic premise: that when using LLMs, they will tend towards some mean when resolving ambiguity. There's a very interesting opportunity here for a deeper investigation. - What does "regression to the mean" actually mean in practice when the LLM is conditioned on a possibly large amount of context? - How does this perceived regression to the mean affect the result in different applications? When implem…

It's got LLM fonts and styles too

Re: Regression to the Mean: on LLMs and the quiet death of the new

#65

Post by aerodexis »

krona wrote: post #28

The output of a GPT is an interpolation (an estimation of new data points inside the range of known data) rather than extrapolation (estimations outside that range). 99% of the time we don't need a true intellectual breakthrough to get the job done, and often 'new ideas' are simply riffs on or blends of old ones, like fashion or music genres. The worry to me, however, is that if society comes to rely on this form of…

The more I read these think-pieces on AI the clearer it is to see that they have very little to do with LLM technology, and are really just talking about the collapse of the set of values constructed during the enlightenment. The sociocognitive atrophy that you describe is inherent to the idea that newness and progress are the ultimate goal that we should organize society around. The problem is that newness and progress can only be defined relative to a status-quo, and hence are incoherent goals in themselves. When pursued far enough, they become their own strange, monstrous status-quo that betrays the original intentions of the people who pursued these values. Hence "we mistake the flattening for progress". This was the case before AI, it's just that AI makes it much harder to ignore, and in many ways encapsulates the problem.

The point w/ electronic calculators is the same point made by Plato regarding books. It used to be easy to laugh off these concerns, not so much today. Imo, this is the real progress: people are now asking meaty questions regarding the ultimate human purpose of books, calculators and technology.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#66

Post by marcosdumay »

throwaway13337 wrote: post #24

This is both true and not. It's true that in a project, a novel idea undeclared as such will be shaved off quietly by an llm. You really need to be explicit about wanting to keep it. You will get pushed into the mean. However, I'd say 90% of making something (that is useful) is repeating the old thing. We stand on the shoulders of giants. Or at least we should. Getting there can be difficult for most of us. I say thi…

> However, I'd say 90% of making something (that is useful) is repeating the old thing.

The entire point of software engineering is to make repetition unnecessary so people can focus on the new.

LLMs are pushing the people that use them into the worst possible set of practices.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#67

Post by krona »

memoriyato3 wrote: post #61
krona wrote: post #57

Earlier quoted context omitted.

> They construct space between the points and extrapolate outside of the points. They don't. They interpolate between the points on a manifold.

so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM? is the proof to the Riehmann hypothesis also somewhere on the manifold and we just need to prod the LLM with the right prompt so it locates the point?

> so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM?

By design this must be the case, even when you account for stochastic sampling i.e. 'temperature'. All it's outputs are a highly-dimensional combinatorial interpolation (I'm talking about GPTs here)

That's probably why Claude is very good at producing plausible nonsense rather than the often correct response of 'I don't know'.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#68

Post by mghackerlady »

xpct wrote: post #56
zwischenzug wrote: post #14

This touches on something I've (and many others) have felt throughout my life, not just since the advent of LLMs. To take a simple example: I grew up with computer games in the '80s where there were no 'physics engines' or frameworks for building games. As a result, each game was an expression of the author's personality somehow. Fast forward to the noughties, games bored me as they mostly looked and felt the same, o…

I'll stick to games and movies, as I believe both have been moving in a similar direction, becoming more of an object to be consumed, rather than to be experienced. I've thought about this in two ways: it's either that (a) when fields are fresh, creators explore orthogonal concepts and fit to what performs best relatively quickly, or (b) the available idea space just isn't that large by itself, and novelty wears off…

For this reason, I'll always love indie games. Anyone who finds themselves bored with moder AAA gaming should really go play some of the 2000s and 2010s indie darlings. Here, I'll even give you a list of games I've been playing

The Binding of Isaac: Rebirth, Bit. Trip, Cave Story, Crypt of the Necrodancer, Cuphead, Downwell, Fez, Hollow Knight, Limbo, Octodad: Dadliest Catch, Papers, Please, Proteus, Risk of Rain, Rogue Legacy, Shovel Knight, Spelunky, Stardew Valley, Super Meat Boy, Terraria, VA-11 Hall-A, VVVVVV

Can you tell I own a PS Vita?

Re: Regression to the Mean: on LLMs and the quiet death of the new

#69

Post by aerodexis »

orbital-decay wrote: post #60
krona wrote: post #28

The output of a GPT is an interpolation (an estimation of new data points inside the range of known data) rather than extrapolation (estimations outside that range). 99% of the time we don't need a true intellectual breakthrough to get the job done, and often 'new ideas' are simply riffs on or blends of old ones, like fashion or music genres. The worry to me, however, is that if society comes to rely on this form of…

>The output of a GPT is an interpolation (an estimation of new data points inside the range of known data) rather than extrapolation (estimations outside that range). That's a common meme but it's the opposite of true. Everything big models, not just transformers, mathematically do is extrapolation in the feature space, almost never interpolation. They're perfectly able of combining the ideas, although of course this…

so you're saying the difference b/w extrapolation and interpolation is subjective unless the difference is defined tautologically?

Re: Regression to the Mean: on LLMs and the quiet death of the new

#70

Post by kolinko »

liendolucas wrote: post #38
CJefferson wrote: post #6

This is my biggest concern. Speaking as someone who recently had to read 60 AI generated reports (the whole issue of how much students are using AI is one discussion), it was genuinely soul-destroying reading the same phrases, seem sentence structures, same arguments over and over. Depressed me the whole of the next day.

Man if I ever had to read 60 AI reports, I'd quit on the spot.

Why? Just get AI to grafe them and give you the outliers to judge
Post reply on HN