Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cognitive Dark Forest

ryelang.org

Post reply on HN
Print view View original

91–100 of 301 posts

Re: The Cognitive Dark Forest

#91

Post by middayc »

stego-tech wrote: post #87

I’m still optimistic that this is cyclical in nature, and not an inevitable - or indefinite - outcome. Humanity has endured regular cycles of shared enlightenment (usually accompanying profound technological or societal revolutions) and dark forests of protectionism, and we always find a way to the other side. Sometimes these cycles last a century; sometimes, but a few years. Still, we always make it to the other sid…

I seem to get into sort of existential crisis every few moths with the progress that llm-s are doing. I probably fool myself for a while that "it's not real", then at some point I can't fool myself any more - then I accept it somewhat ... then the new progress happens and it cycles again.

But as it's written at the top, this was a thought experiment, not a prediction. And while I tried to put all the bad scenarios on the table (with the theme of the dark forest that is), I think I again found a sense of optimism, because I also think this thought experiment has flaws.

So I hope, that after a while I will be able to write the contrary, I've already written down some points about it - I already have a title. But we will see. I am more optimistic after I wrote this than before. :P

Re: The Cognitive Dark Forest

#92

Post by skybrian »

How about putting an idea or a vibe-coded demo out there in hopes that others will copy it, because you want it to exist and become more common? But it's less work if someone else maintains it.

This is free as in free puppy.

Re: The Cognitive Dark Forest

#93

Post by hrimfaxi »

zenogais wrote: post #29
middayc wrote: post #26

Earlier quoted context omitted.

Well, I didn't know for this book, so I suspect or hope the exact points that I make won't map to the ones from the book. It is true that the original "The dark forest" book made an impression on me, so I was thinking about its theories often and trying to apply them to various situations.

Yeah, I fully believe independent invention by mapping "the dark forest" onto the internet is very possible.

The irony is that it undermines the premise. Multiple people independently arriving at the same conclusions means that you can hide your ideas from the dark forest but that won't stop them from being uncovered.

Re: The Cognitive Dark Forest

#94

Post by boutell »

OK, so maybe we're headed for a dark forest scenario as far as profit driven startups go.

But if your goal is simply for the thing to exist, there is a strong incentive to share.

Re: The Cognitive Dark Forest

#95

Post by hrimfaxi »

RajT88 wrote: post #83
Skyy93 wrote: post #71

This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…

> This was the same before, if you had a novel idea and make a product out of it others follow. You've almost captured the full picture of it. If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk. Algorithms are cheap. Sure, they could use your LLM history to figur…

> But again - the hard part is not cloning the product, it's stealing your customers. People don't seem to be focused on the hard parts.

Big companies seem to be bad at innovating but really, really good at enterprise sales.

Re: The Cognitive Dark Forest

#96

Post by RajT88 »

oh_my_goodness wrote: post #89
RajT88 wrote: post #83

Earlier quoted context omitted.

> This was the same before, if you had a novel idea and make a product out of it others follow. You've almost captured the full picture of it. If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk. Algorithms are cheap. Sure, they could use your LLM history to figur…

Yeah, and the big guys can't steal your customers. What a crazy idea.

The point is - they're going to do that anyways if they want to. Owning the LLM platforms makes it marginally cheaper to do so.

It's not the risk it's being made out to be.

Re: The Cognitive Dark Forest

#97

Post by middayc »

hrimfaxi wrote: post #93
zenogais wrote: post #29

Earlier quoted context omitted.

Yeah, I fully believe independent invention by mapping "the dark forest" onto the internet is very possible.

The irony is that it undermines the premise. Multiple people independently arriving at the same conclusions means that you can hide your ideas from the dark forest but that won't stop them from being uncovered.

Interesting irony :). If you don't produce the idea in fear of just feeding the forest, someone else will, so it might as well be you. It's true that this is very similar as current dilemmas some people have about their ideas.

The difference is that now people see just the outer shell of your ideas - but if you use LLM-s to search, explore, code your ideas, the system "knows" it all, or even more than you, given that it can "cross-pollinate".

Re: The Cognitive Dark Forest

#98

Post by jmull »

iugtmkbdfil834 wrote: post #80
jmull wrote: post #47

Earlier quoted context omitted.

I really liked those books, for all the creative ideas... it's fine that they don't all work, but the Dark Forest has to be among the worst of them. It was unfortunate it was highlighted. Some rebuttals, going point by point... 1. you can know the intentions of other entities by observing and communicating with them. 2. technology explosions, like pretty much exponential phenomena, are self limiting. They necessarily…

To an extent, rebuttals land. However, 1. You are assuming a lot in the sense that you assume presence of intention -- not something guaranteed to be a feature of an alien civilization, which is, well, alien. People think that anthropocenrism only applies to body shape and having legs, because the way it tends to express itself in popular culture is robots on legs and human body shape in aliens. And same point goes t…

You can observe patterns of behavior, develop theories understanding, attempt/experiment with interactions, and refine based on the results. That's communication (and doesn't assume anything about the other alien civilization).

Now, civilizations may be more or less willing to do this and more or less successful, but that's not the same thing as no one will dare try, as the dark forest theory wants.

(Personally, I think civilizations that are better at this will outcompete ones that are worse or refuse, though that's just my own opinion.)

> Bold assumption that they are self limiting.

Name the exponential phenomena that aren't self limiting -- that don't consume the medium which allows them to exist in the first place.

> I think the answer tends to be resources, which is the foundation of dark forest argument theory to begin with.

Well, yes. One of the reasons the dark forest theory isn't coherent.

> Any real alien reasons would be alien to us.

Yes, but this doesn't back up the dark forest theory. It also doesn't mean aliens cannot be understood at any level or interacted with in any way.

(The dark forest theory makes very strong claims on the logic, intentions, strategies, resource use/governance of alien civilizations, BTW, and wants this to be uniform amongst them... even though the one civilization we actually know of doesn't adhere to them.)

Re: The Cognitive Dark Forest

#99

Post by stego-tech »

middayc wrote: post #91
stego-tech wrote: post #87

I’m still optimistic that this is cyclical in nature, and not an inevitable - or indefinite - outcome. Humanity has endured regular cycles of shared enlightenment (usually accompanying profound technological or societal revolutions) and dark forests of protectionism, and we always find a way to the other side. Sometimes these cycles last a century; sometimes, but a few years. Still, we always make it to the other sid…

I seem to get into sort of existential crisis every few moths with the progress that llm-s are doing. I probably fool myself for a while that "it's not real", then at some point I can't fool myself any more - then I accept it somewhat ... then the new progress happens and it cycles again. But as it's written at the top, this was a thought experiment, not a prediction. And while I tried to put all the bad scenarios on…

[dead]

Re: The Cognitive Dark Forest

#100

Post by sigbottle »

jandrewrogers wrote: post #68
scottlawson wrote: post #4

The thesis that in the past it was safe to share ideas and projects because the execution was hard, and that now things have changed because of AI is an interesting AI, but I wonder if it is really true. It certainly seems true that for small projects and relatively narrow scoped things that AI can replicate them easily. I'm thinking specifically about blog posts where people share their first steps and simple progra…

It isn't just about AI. Some R&D domains started disappearing from literature and the public internet a decade before the first LLMs. The incentives to go dark emerged even when the adversary was other humans. AI is just accelerating a trend that was already there. Some areas of frontier computer science research have largely been dark for decades. The strategy is to quietly do several years of iterated hardcore R&D.…

Interesting, any examples of companies that followed this model?
Post reply on HN