Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Regression to the Mean: on LLMs and the quiet death of the new

rruxandra.github.io

Post reply on HN
Print view View original

81–90 of 103 posts

Re: Regression to the Mean: on LLMs and the quiet death of the new

#81

Post by kolinko »

krona wrote: post #67
memoriyato3 wrote: post #61

Earlier quoted context omitted.

so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM? is the proof to the Riehmann hypothesis also somewhere on the manifold and we just need to prod the LLM with the right prompt so it locates the point?

> so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM? By design this must be the case, even when you account for stochastic sampling i.e. 'temperature'. All it's outputs are a highly-dimensional combinatorial interpolation (I'm talking about GPTs here) That's probably why Claude is very good at producing plausible nonsense rather than the often correct response of 'I don't k…

What you’re saying here makes no sense.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#82

Post by orbital-decay »

krona wrote: post #71
orbital-decay wrote: post #60

Earlier quoted context omitted.

>The output of a GPT is an interpolation (an estimation of new data points inside the range of known data) rather than extrapolation (estimations outside that range). That's a common meme but it's the opposite of true. Everything big models, not just transformers, mathematically do is extrapolation in the feature space, almost never interpolation. They're perfectly able of combining the ideas, although of course this…

> That's a common meme but it's the opposite of true. But you're restating what I just wrote - We're training a status quo machine and the probability of anything outside that distribution rapidly drops to zero.

I'm saying that the models have to be trained better for them to be considered creative. Looking at how many low hanging fruits there are in AI right now, I'd say it's possible given enough time and slow enough adoption. Looking at how little the AI labs and everyone else are interested in those low-hanging fruits and how much they're focused on politics, hype, and safety religion, I'm not optimistic.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#83

Post by btbuildem »

It's an absolute regression to the mean. We need to understand the correct place for LLM-driven coding agents et al: they're beige paste extruders. Which is actually a perfect match for 80% of corporate world.

For anything unique, they're a boon, in that they can free you from dealing with the boilerplate, but you must think on your own, must have your own insights, tap into your creativity. If you start to depend on them for ideation, it's slippery slope to the slop bowl.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#84

Post by rmah »

dmd wrote: post #49
lotsofpulp wrote: post #37

Earlier quoted context omitted.

The generic city one makes no sense. Does the author want each city to invent a new physics? Not only are there quite a few different looking buildings in each of the cities, but given the constraints of not have unlimited funds, surely one can understand that many columns of steel, concrete, and glass will look like columns of steel, concrete, and glass from afar.

Cities could look like https://www.arcosanti.org/ Cities could look like https://en.wikipedia.org/wiki/Habitat_67 Cities could look like https://www.atlasobscura.com/places/the-blue-city-of-jodhpur...

The first two look interesting, the last looks like a huge slum. They all look like bad places to live. IMO, of course.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#85

Post by kolinko »

terabytest wrote: post #27

We truly are reaching the end times when an article criticizing the use of LLM is in itself pure AI slop.

It’s also ironic that the slop article complaining about the lack of originality has very little intellectual originality itself.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#86

Post by bluefirebrand »

mghackerlady wrote: post #68
xpct wrote: post #56

Earlier quoted context omitted.

I'll stick to games and movies, as I believe both have been moving in a similar direction, becoming more of an object to be consumed, rather than to be experienced. I've thought about this in two ways: it's either that (a) when fields are fresh, creators explore orthogonal concepts and fit to what performs best relatively quickly, or (b) the available idea space just isn't that large by itself, and novelty wears off…

For this reason, I'll always love indie games. Anyone who finds themselves bored with moder AAA gaming should really go play some of the 2000s and 2010s indie darlings. Here, I'll even give you a list of games I've been playing The Binding of Isaac: Rebirth, Bit. Trip, Cave Story, Crypt of the Necrodancer, Cuphead, Downwell, Fez, Hollow Knight, Limbo, Octodad: Dadliest Catch, Papers, Please, Proteus, Risk of Rain, Ro…

Honestly the indie darlings are only getting better.

I've been playing a survival crafting game lately called Abiotic Factor and it is blowing me away with how creative and fresh it is, even in the fairly crowded "survival craft" genre

Re: Regression to the Mean: on LLMs and the quiet death of the new

#87

Post by lmeyerov »

kolinko wrote: post #76
lmeyerov wrote: post #33

I've been calling this Software Collapse It's the same problem that AI faces of Model Collapse: AIs that train on the internet ultimately just end up training on one another, stop moving forward, and end up as identical polished versions of one another I now think of it as a Dr. Jekyll/ Mr. Hyde situation for software projects: - Dr. Jekyll: For makers, the only limit is your imagination, architectural guidance, and…

That is not true - a model trained in the internet can both build verifiers to remove false/poor quality data from the next training, and build synthetic datasets that will supplement its training. Similar to a human that wants to learn something and invents exercises to practice. 1-2 years ago it was a theory, but new models are trained, successfully, on synthetic datasets.

This feels like a bit of a semantic debate, but maybe a few useful perspectives, as interpret current bespoke work as not so rosy wrt collapse:

- Synthetic datasets are typically human-steered today, which points to model collapse wrt learning from the internet. (Edit: or even simpler, model x data tapped out for cost/benefit even before collapse.) I don't think standard practice is (yet) AI looking at the internet and deciding to build its own gyms to go further. When it does, model collapse may happen again, and be even more expensive

- Distillation attacks are getting interesting here. There seem to be 2 kinds: intentionally querying other models, and maybe not so intentionally, learning from reasoning traces going through shared routers, esp. coding ones

- A lot of neolabs are trying to go where the big labs might not look as directly to avoid being squashed, which suggests they aren't ready to bet on being smarter, and that means the general AI is more about collapse / $

Re: Regression to the Mean: on LLMs and the quiet death of the new

#89

Post by aerodexis »

kolinko wrote: post #74
aerodexis wrote: post #69

Earlier quoted context omitted.

so you're saying the difference b/w extrapolation and interpolation is subjective unless the difference is defined tautologically?

There isn’t a big difference between interpolation and extrapolation when the space has an immense amount of dimensions, and when you are free to modify the space at will.

Nice point. I agree with this, and it implies that it's useless to apply the category of creativity to LLMs (at least wrt interpolation and extrapolation), and even more useless to suggest that we'll cross the creativity threshold once the models have been sufficiently embiggened.

Re: Regression to the Mean: on LLMs and the quiet death of the new

#90

Post by marssaxman »

rmah wrote: post #84
dmd wrote: post #49

Earlier quoted context omitted.

Cities could look like https://www.arcosanti.org/ Cities could look like https://en.wikipedia.org/wiki/Habitat_67 Cities could look like https://www.atlasobscura.com/places/the-blue-city-of-jodhpur...

The first two look interesting , the last looks like a huge slum. They all look like bad places to live . IMO, of course.

That's funny - to me, Jodhpur is most immediately appealing of the three. It reminds me of the medinas I visited in Fez and Marrakesh: overwhelming at first, as a visitor, but compelling, and full of life. I'd rather live in a thriving urban place like that, all human-scale and pedestrian, than some sprawling, soulless, car-dependent suburb like the ones a majority of North Americans inhabit.
Post reply on HN