Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Cognitive Dark Forest

ryelang.org

Post reply on HN
Print view View original

201–210 of 301 posts

Re: The Cognitive Dark Forest

#201

Post by asdff »

Skyy93 wrote: post #71

This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…

Another thing the author makes a note of is the idea of the prompts getting logged. I can imagine with some clever statisticians, before you even formulate the idea yourself of some product or company, the model can construct this before you even do, just based on what you already prompted. Then it can evaluate market fit, estimate return, start its own version of that company and make money and beat you to market should it turn out to be a good idea.

Now before you say this is unrealistic or isn't done today, just know this is all perfectly possible with existing technology. In fact this is a lot how adtech works already, using metadata to predict products you might want to buy before you even realize you want to buy them.

Re: The Cognitive Dark Forest

#202

Post by asdff »

jrowen wrote: post #155

It is also asymmetric. If you announce your presence, even if 4 out of 5 civs that notice you don’t annihilate you immediately (but they probably should), the fifth might. It’s just a probability game, with permadeath. So hiding is the most rational - the only - strategy of survival. This is a paranoid and cynical strategy that doesn't win out in the known history of life. What works is grow, expand, mingle, maintain…

>This is a paranoid and cynical strategy that doesn't win out in the known history of life. What works is grow, expand, mingle, maintain - assimilate but don't annihilate.

Uhh, yes it does. You are thinking of humans. Humans can mate with other humans. They can assimilate. Now, think of invasive species. What do they do? They don't mate with the natives, learn their culture, respect and give them space. Quite the opposite on all counts. They do what they do in their resource game. They might not out compete the natives and they might peter out. Or, they do out compete the natives, and before long, there aren't any natives or that careful equilibrium that was established beforehand.

Re: The Cognitive Dark Forest

#203

Post by asdff »

chairmansteve wrote: post #195
stego-tech wrote: post #87

I’m still optimistic that this is cyclical in nature, and not an inevitable - or indefinite - outcome. Humanity has endured regular cycles of shared enlightenment (usually accompanying profound technological or societal revolutions) and dark forests of protectionism, and we always find a way to the other side. Sometimes these cycles last a century; sometimes, but a few years. Still, we always make it to the other sid…

You are right. In the dark forest, the predators must eventually die out, because they can't find prey.

Some predators can go weeks to months between feeding. Some snakes, some spiders. Now consider some lovecraftian horror.

Re: The Cognitive Dark Forest

#204

Post by asdff »

itsgrimetime wrote: post #193

“These AI tools are garbage and can’t create anything worth creating” “These AI tools are so powerful they can steal your ideas with nothing but a sentence” I know that’s not exactly what OP is saying but the pretentiousness of the “we knew better” got to me a little bit. I think it’s a cool and unique analogy but I’m not as pessimistic. Ideas have become so cheap to try/experiment with, more people are able to try 1…

False equivalence here. The slop generators are not the tooling used to fingerprint you. But you should fear the slop generator anyhow, because even if it's shitty, your boss might not be aware and might fire you anyhow thinking your coworker can handle two people's jobs now with the tool. And maybe the wheels don't even come off with that decision because it isn't like engineering quality is 1:1 correlated with sales.

Re: The Cognitive Dark Forest

#205

Post by bsza »

Barrin92 wrote: post #130
bsza wrote: post #111

The best analogy I can think of (quite similar to this one) is that the internet is low Earth orbit and AI is the Kessler syndrome. We abandon the place not to hide ourselves, but because it is saturated with garbage, and anything you try to put up there will only result in even more garbage being generated, without any positive effect. The ideal solution would be to remove the garbage, but right now we can't even de…

I don't buy the analogy. The problem with Kessler syndrome is that low earth orbit is physically crowded, you run into collisions. I don't care about the garbage. I don't care about the AI era. I've been writing code in Emacs for 20 years, I'll be writing code in Emacs in 20 years, every open source project I contribute to still looks the same because all these AI people, like the blockchain people do is just make ne…

Well, if you open-source anything these days and it does make it big, you can be prepared for a flood of low-effort slop PRs that you must either review for free or stop accepting external contributions altogether, making it effectively closed-source. You can't choose to ignore the garbage, it will collide with your stuff, unless your stuff is small enough to avoid collisions (in which case no one will see it).

Re: The Cognitive Dark Forest

#206

Post by dreamglider »

For the better part of the last 20+ years big corpos had the $ to throw and replicate virtually anything they wanted. They got the cash and man power, yet they didn't do it. Why? Because they don't care, they have business to run, they need to somewhat keep focus which can't happen when scattering attention all over the place. The difference now is that for relatively >simpleOne seems to omit the fact that LLMs are fundamentally designd for workload quite different for what they are being used right now. Sure you can improve them but can't escape / workaround the current NLP design endlessly. Then there's the irony - Internet did deliver on free (as close as it gets) and easy access to information (any). Did this make people smarter, more knowledgable, more tech savvy & etc? Nope, it didn't. Just like the libraries didn't (queues at libraries were and are a rare event). Big deal that the information is readily available when people do not know what do with it or care to do anything.

Ideas are cheap, the chances of having some truly unique idea that is also business feasable are not that big. It's not so much about the ideas but rather the ability to execute, follow through and well - make sales while constantly improving what you got. Staying silent, going dark - have their merit but only when the wheels are already turning and one is into acting, not into fearful hiding.

In either case - awesome blog post!

Re: The Cognitive Dark Forest

#207

Post by mekoka »

chairmansteve wrote: post #194
mekoka wrote: post #182

Earlier quoted context omitted.

A typical outlook from 21st century human thinking. We love to draw from our still rather actual history of fear and addictions to zero-sum games, to extrapolate the far advancement of other civilizations. As millennia go by, species can obviously only evolve technologically, while remaining psychologically, philosophically, and spiritually stuck.

"As millennia go by, species can obviously only evolve technologically, while remaining psychologically, philosophically, and spiritually stuck". Interrsting take, and possibly (probably?) true of humans. But is it true of other (alien) sentient species?

Apologies, I should've been more obvious in my attempt at sarcasm.

Re: The Cognitive Dark Forest

#208

Post by Terr_ »

spartanatreyu wrote: post #108

This post puts forward two paths: 1) Everyone and everything is subsumed into the forest. Innovation becomes unprofitable for the innovator as the one who controls the forest uses their capital to clone every new innovation. 2) Everyone withdraws from the forest. Innovation goes private. The forest stops growing, but doesn't die. --- But there's two things the post doesn't consider: 1) Viral licensing. What happens t…

> Viral licensing

IANAL, but for some months now I've been pondering what could happen if a site--like a personal blog--had a legally-strong "click-wrap" Terms of Service, which unlike GPL means it rests mostly on contract law, rather than copyright.

For example, imagine a ToS that says something like:

1. You acknowledge I am providing you something of value (my content) and you agree to provide compensation/consideration if you use that in an AI model.

2. By doing that, you grant me an irrevocable worldwide license to use and sub-license all content that emerges from the model, notwithstanding any future agreement you may make in reselling access or outputs of that LLM to anyone else. If a conflict should occur, you agree to indemnify me against claims by that other entity.

3. If you believe my content was not a material factor in some output, the burden of proof is on you to identify the specific output and prove that my content did not influence the it.

In short, this doesn't stop someone from stealing my art/writing, but it does put a potential hole in their attempts to monetize it.

For example, if they scrape my music and then license a copy of the new Music Generator 3000 to Disney, and Disney makes a movie with that music...

Re: The Cognitive Dark Forest

#209

Post by xnorswap »

bsza wrote: post #205
Barrin92 wrote: post #130

Earlier quoted context omitted.

I don't buy the analogy. The problem with Kessler syndrome is that low earth orbit is physically crowded, you run into collisions. I don't care about the garbage. I don't care about the AI era. I've been writing code in Emacs for 20 years, I'll be writing code in Emacs in 20 years, every open source project I contribute to still looks the same because all these AI people, like the blockchain people do is just make ne…

Well, if you open-source anything these days and it does make it big, you can be prepared for a flood of low-effort slop PRs that you must either review for free or stop accepting external contributions altogether, making it effectively closed-source. You can't choose to ignore the garbage, it will collide with your stuff, unless your stuff is small enough to avoid collisions (in which case no one will see it).

Zero-contribution open source doesn't at all make it closed source.

It delivers on the value of open source, that anyone using your software is permitted to make and distribute their own changes.

SQLite is an example of a project that is open source but closed contribution.

Re: The Cognitive Dark Forest

#210

Post by yetihehe »

ohelm wrote: post #184
bsza wrote: post #111

The best analogy I can think of (quite similar to this one) is that the internet is low Earth orbit and AI is the Kessler syndrome. We abandon the place not to hide ourselves, but because it is saturated with garbage, and anything you try to put up there will only result in even more garbage being generated, without any positive effect. The ideal solution would be to remove the garbage, but right now we can't even de…

I would suffocate it. Know the greedy snake idiom? A snake is so hungry and greedy that it suffocates on its prey? Best you can do is to spread all of the goods it provides, as it is too greedy to not devour them itself. It will consume them and suffocate slowly.

> It will consume them and suffocate slowly.

Can we accelerate it perhaps? You know, spending ALL our resources on making the snake more fat is not a good idea. Its only good idea when you have so much resources that you can easily suffocate the snake with negligible (for us) amount. If you try to suffocate several million snakes, that might backfire a little.

Post reply on HN