Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cognitive Dark Forest

ryelang.org

Post reply on HN
Print view View original

81–90 of 301 posts

Re: The Cognitive Dark Forest

#81

Post by girvo »

chongli wrote: post #50
gobdovan wrote: post #37

Earlier quoted context omitted.

Instead of anti-fragility, I'd point you to the law of requisite variety instead. You'll notice that all AI improvements are insanely good for a week or two after launch. Then you'll see people stating that 'models got worse'. What happened in fact is that people adapted to the tool, but the tool didn't adapt anymore. We're using AI as variety resistant and adaptable tools, but we miss the fact that most deployments…

New models literally do get worse after launch, due to optimization. If you charted performance over time, it'd look like a sawtooth, with a regular performance drop during each optimization period. That's the dirty secret with all of this stuff: "state of the art" models are unprofitable due to high cost of inference before optimization. After optimization they still perform okay, but way below SOTA. It's like a kni…

> If you charted performance over time, it'd look like a sawtooth

People have, though, and it doesn't show that. I think it's more people getting hit by the placebo effect, the novelty effect, followed by the models by-definition non-determinism leading people to say things like "the model got worse".

Re: The Cognitive Dark Forest

#82

Post by georgemcbay »

djeastm wrote: post #59
nicbou wrote: post #27

Earlier quoted context omitted.

The problem for me is that I'm competing with the AI results that Google trained on my work. I'm losing the majority of my traffic to it, so at some point I'll have to give up because the work no longer supports me and no longer has an audience.

Same here. Knowledge is being commodified.

> Knowledge is being commodified.

Already was well before AI, the difference now is that a few big AI providers risk becoming the ultimate rent-seekers that will increasingly capture all of the value of that commodified knowledge whether the original knowledge generators want that or not. There is no opt out, everything will be vacuumed up into the machine mind.

This will almost certainly lead to vastly increased amounts of wealth inequality (on top of the already unsustainable levels we have today) and possibly a very messy societal disintegration (this is theoretically avoidable, but I am not convinced it is practically avoidable given our current socioeconomic/political realities).

Bright future ahead!

Re: The Cognitive Dark Forest

#83

Post by RajT88 »

Skyy93 wrote: post #71

This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…

> This was the same before, if you had a novel idea and make a product out of it others follow.

You've almost captured the full picture of it.

If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk.

Algorithms are cheap. Sure, they could use your LLM history to figure out what you did. Or the LLM could just reason it out. It could save them some work, sure.

But again - the hard part is not cloning the product, it's stealing your customers. People don't seem to be focused on the hard parts.

Re: The Cognitive Dark Forest

#85

Post by bdangubic »

rhubarbtree wrote: post #38

This is mislead by the nerd philosophy that the tech is the business. It absolutely isn’t, the tech is a small part of a startup. Witness that Spotify continues to exist despite being known and replicated by the major giants. Poetically expressed, but ultimately based on a false notion of what a business actually is.

you picked one fairly rare thing with an incredible non-tech moat that is also a cancer for the artist, bravo!!

Re: The Cognitive Dark Forest

#86

Post by middayc »

Skyy93 wrote: post #71

This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…

> This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly. Claude Opus 4.6 knowledge cut off was August 2025, so every idea you type in after this date is in the training data but not available, so you only have to be fast enough. Especially LLMs/AI-Agents like Claude enable this speed you need for bringing out something new.

You have a point about the update intervals and the higher speed they provide to developers. But you are talking about now, and I was making a thought experiment - about a potential future. LLM-s are not learning on the fly, but I suspect they do log the conversations, their responses and could also deduce from further interaction if a particular response was satisfactory to the user. So in a world where available training data is drying up, nobody is throwing all this away. Gemini even has direct upvote/downvote on responses. Algorithms will probably improve, and the intervals will probably shorten.

Given the detailed information that all the back and forwards generate - I think it's not hard to use similar technology to track underlying trends, get all the problems associated with them and all the solution space that is talked about - and generate the solution before even the ones who thought of it release it. Theoretically :)

I think the open development will become less open. I don't like it - but I think it's already happening. First - all the blogs and forums moved to specialized platforms (SO, discords, ..) and now event some of those are d(r)ying. If people (in extreme cases) don't even read the code they produce, why would they read about the code, discuss the code, that's not even in their care. That is without the theoretical fear of the global Borg slurping all they write.

Re: The Cognitive Dark Forest

#87

Post by stego-tech »

I’m still optimistic that this is cyclical in nature, and not an inevitable - or indefinite - outcome.

Humanity has endured regular cycles of shared enlightenment (usually accompanying profound technological or societal revolutions) and dark forests of protectionism, and we always find a way to the other side. Sometimes these cycles last a century; sometimes, but a few years. Still, we always make it to the other side.

In the case of LLMs, we have to make a few assumptions: that they will not lead to AGI, nor will we solve the problem of real-time learning or context windows. These are, admittedly, huge assumptions, but the current state of AI and compute suggests a nugget of truth to them for the time being. If that’s the case, then perhaps this “dark age” of the dark forest is bounded by the limitations of silicon-based computing (hence the push towards Quantum) and the human frustration with diminishing returns from technological investment. As artisans and brilliant minds withdraw, the forest risks starvation and withering from a lack of sustenance; if humans withdraw from technology because they must hand over IDs and personal data, because to engage with technology is to surrender to surveillance and persecution, then the natural trend will be to withdraw over time - and the markets will adapt accordingly, with or without external/government intervention.

That is to say that the dark forest only lasts as long as its inhabitants decide to persecute each other for daring to light a path forward. Right now, the incentives very much favor those willing to harm others for personal enrichment; that is not always the case, and humans decide when that reasoning becomes vilifiable.

Re: The Cognitive Dark Forest

#88

Post by [deleted] »

Phemist wrote: post #9
king_phil wrote: post #3

Dark forest makes no sense to me. Why would a civilization eradicate another, spending huge amounts of resources (time, energy, material) when the universe has such an enormous scale that you cannot even get to each other in a timescale that makes much sense...

The dark forest is conditional on that it does not require huge amounts of resources to eradicate another civilization and that (over time) the universe turns out not to be of a scale enormous enough (and in the book there are agents working to actively make it smaller). Bringing it back to the dark forest of idea space, it is an interesting question whether the the space of feasibly executable ideas being small (as…

[deleted]

Re: The Cognitive Dark Forest

#89

Post by oh_my_goodness »

RajT88 wrote: post #83
Skyy93 wrote: post #71

This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…

> This was the same before, if you had a novel idea and make a product out of it others follow. You've almost captured the full picture of it. If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk. Algorithms are cheap. Sure, they could use your LLM history to figur…

Yeah, and the big guys can't steal your customers. What a crazy idea.

Re: The Cognitive Dark Forest

#90

Post by rglover »

This feels dramatic. The parts are there to rebuild a better web. If you want to build it, build it. But most still want the money, they just want to get it while also retaining the moral high ground. A VPS can still be had for cheap. Code is now "free" (not necessarily good code but like you suggest "good enough"). The only thing stopping you at this point is your own ego (and its expectations of success).

"You wanna escape Armageddon, read a different book." - KRS-One

Post reply on HN