The best analogy I can think of (quite similar to this one) is that the internet is low Earth orbit and AI is the Kessler syndrome. We abandon the place not to hide ourselves, but because it is saturated with garbage, and anything you try to put up there will only result in even more garbage being generated, without any positive effect. The ideal solution would be to remove the garbage, but right now we can't even de…
The Cognitive Dark Forest
161–170 of 301 posts
Re: The Cognitive Dark Forest
#162This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…
First of all: it's not as though no new LLMs are being trained. Of course they are.
Second: learning LLMs are not far off, and since they can typically search the web via agents, they effectively can "learn" now, and they can learn (not so well) by writing stuff into a document hidden from you. Indeed, some LLMs can inspect your other sessions with them and refer to them in future sessions -- I've noticed this with Claude.
Third: already we see some AI companies wanting to train their models on your prompts. It's going to happen.
> The next thing is that we also have open source and open weight models that everyone of use with a decent consumer GPU can fine-tune and adapt, so its not only in the hands of a few companies.
There's a pretty good chance that LLMs buff open source, yes.
> > We will again build and innovate in private, hide, not share knowledge, mistakes, ideas.
> Why should this happen? The moment you make your idea public, anyone can build it. [...]
This was always the case, but now the cycle is faster. Therefore if you must use an LLM you might use an LLM that you run on your own hardware -- now your prompts are truly yours. But as TFA notes the AIs will learn just from your (and your private LLMs') searches, and that will be enough in some cases for them to figure out what you're up to. Oh sure, maybe the Microsofts and Googles of the world will not be able to capitalize on millions of interesting idea floating about, but still! the moment you uncloak the machine will eat your future alive, so you'll try to stay off its radar and build a moat it can't see (good luck!). Well that's what TFA says; it seems very plausible to me.
Re: The Cognitive Dark Forest
#163This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…
> This was the same before, if you had a novel idea and make a product out of it others follow. You've almost captured the full picture of it. If you have a great idea, it's not going to be self-evidently enough of a great idea until you've proved it can make money. That's the hard part which comes at great personal, professional and financial risk. Algorithms are cheap. Sure, they could use your LLM history to figur…
Yes. A Red Hat, a Microsoft -- these companies have processes, organizational structure, politics, friction, etc. They might like your products, but replicating them might not be easy for reasons that have nothing to do with how easy it is given the freedom to do it. Small shops with vision might well have a bright future, for a while, maybe.
Re: The Cognitive Dark Forest
#164It is also asymmetric. If you announce your presence, even if 4 out of 5 civs that notice you don’t annihilate you immediately (but they probably should), the fifth might. It’s just a probability game, with permadeath. So hiding is the most rational - the only - strategy of survival. This is a paranoid and cynical strategy that doesn't win out in the known history of life. What works is grow, expand, mingle, maintain…
Most leaders in the Western/developed world have similar paranoid thought processes.
Re: The Cognitive Dark Forest
#165Earlier quoted context omitted.
Most leaders in the Western/developed world have similar paranoid thought processes.
Do China's or Russia's leaders not?
Re: The Cognitive Dark Forest
#166Re: The Cognitive Dark Forest
#167The existing megacorps have huge swaths of infrastructure, expenses, and requirements that require massive amounts of capex to maintain. Even if performative, Meta, Google, OpenAI, Anthropic, et. Al cannot simply layoff their entire engineering, accounting, HR, sales, and support infrastructure. Those orgs are large for “good” (historically necessary) reasons.
Now fast forward to today, and this is where I differ in opinion, it is our megacorps are the civilizations who should be scared of being discovered. Minus infrastructure providers, they are the large advanced entities which can be annihilated by someone with a decent budget and a good local model.
For ~$30k-$50k (primarily buying RTX 6000 pro GPUs and a CPU with enough PCIE lanes) “anyone” can build a system using open weight models that, and let me truly emphasize this: autonomously create functionality to compete. Previously it would take me months, or years, of immense dedication to show up after work and produce something of value. Now I can do it using excess compute on my existing workstation. No existing corporation can afford to undercut every possible idea. If I only gave 1000, 10,000, or a 100,000 users they cannot compete. That may, and I believe it will, provide more than enough capital to attack that megacorps X or Y. If I’m making $100k a month, I can afford multiple autonomous systems per month. After that initial capex, I can then hire other people to help manage them. At no point will a company with billions upon billions of dollars in quarterly capex be able to compete.
Maybe they can compete with one, two, ten, or a hundred but they cannot compete with the absolute onslaught on thousands of possible frontlines. They can cut costs, by reducing their workforce, but they’ll only be increasing their competition to save their earnings report.
And yes, I realize that the open weight models are created via obscene amounts of capital, but we’re lucky that competing nation states, and cultures, like China have immense incentive to do so. Good enough, is still good enough.
The forest may be dark, but it won’t be for much longer.
tldr; call the an ambulance, but not for me. It’s going to be for the existing power structure.
Re: The Cognitive Dark Forest
#168Re: The Cognitive Dark Forest
#169This article makes no real sense to me. >You think of something new and express it - through a prompt, through code, through a product - it enters the system. Your novel idea becomes training data. The sheer act of thinking outside the box makes the box bigger. This was the same before, if you had a novel idea and make a product out of it others follow. Especially for LLMs, they are not (till now) learning on the fly…
> This was the same before, if you had a novel idea and make a product out of it others follow. The article says: "Ideas are cheap - execution is hard" "Announcing, signaling your ideas offered much greater benefit than risk, because your value multiplied by connections, and execution was the moat you could stand behind." That's the key difference. It used to be much harder for a competitor to catch up to the state o…
Re: The Cognitive Dark Forest
#170Relatively grandiose post sweeps in all kinds of claims about the universe to merely warn a big corporation can copy your idea easily.
In the beginning, you reached out with reckless abandon. It was fun to banter with dogs online. Nobody would ever see unless they were looking through your wall. There was no search. No comment history. Bumping into someone in the vast night was enough of a miracle. Why hold back? There are some forum warriors on some PHP BB somewhere, but the domains they rule are insignificant. If you're talking to someone, your motivations are rooted somewhere in the grass.
First came the like button. Rather than blindly hoping what you say resonates with the sensibilities of people you probably knew IRL, rather than present your genuine self because there were no scores, the incentive signal would begin to distort us. Then the newsfeed meant that if you got enough likes, you might get a moment of fame. We all knew it was a terrible idea, a force that would only corrupt us. The personal nature of disjoint little walls living in isolation, was becoming replaced by global stack-ranking.
Then the algorithms came. With them came content marketing to jump the line. At first the ten blue links were filling in the sparsity. Along with that came only a little bias, connecting semantically distant topics, but with a little bit of a feedback loop, an resonator with an unknown response curve. Engagement could be measured, and before long, we were chasing the same likes we used to train the system, and trained by our likes, attracted we became to mysterious stable manifolds, chasing the chase we ourselves define, like Nascar, but insidiously more stupid.
Little by little, the incentive trails no longer lead back to the grass. Reality became suspended without support, a self-sustaining virtual reality determined to fight you to prove that it exists, to prove that its conclusions were right. Every out group is understood to be an echo chamber, an ant mill spiraling helplessly, yet cynically, those who understand these mills best also wind them up like beyblades to crash them into other communities, seeking advantage with the asymmetry of outrage. After the battles, say what was made common to say, and you will be rewarded.
The spinning wheels cannot steer themselves and instead are dictated by whichever chaotic divergence generates the most powerful local gravity well, but because the goal of most is to harvest karma at the bottom, and because the mass controls where the bottom is, over and over we find ourselves pushing all others into the nearest pit to more quickly generate the illusion-giving singularity.
Like Darth Nihilus, the internet seeks only to feed, to feed on the validation that only the internet can give, the permission-giving blessings it needs to tell itself why the grass is wrong. All those who speak of grass are wrong. All those who smell of grass wreak and are wrong. We must destroy the grass, all those appeal to grass. After all grass is dust, at last we will project our utopia into reality. At last we will be not only right but so right that our beliefs will project back into reality.
The spaces within this over-connected, globally addressed world grows into a new kind of sparseness, one where all knowledge of grass must be concealed. Those who can ground the conversations in primary sources flee. Those who can color reasoning with nuance instead withdraw. Reality has retreated as the most dominant reverberations roam like the predator cities of Mortal Engines, looking for any invalidating observations to roll over and consume. Any real life must pretend to be a bot to blend in with the background radiation.
Less like Skynet and more like a zombie apocalypse, the threat comes from within, from among us, from our corruptions, from our karma seeking performances, from our lack of any commitment to any underlying reality, from our flawed belief that the information spaces is some kind of reality stone that enables active control instead of a mere reaction, the shadows on the wall, the murky results of the true forms.
Yet in this new darkness, a certain light has always held. What one wishes, one knows another has wished. What one respects, one knows another respects. No matter the limits of self-knowledge, no matter the information desert one has to cross at night to live in instinct, it is an infinitely brighter signal than the cynical self-corruptions of living for the machine, living to win the games whose rules it was our job to write. What one believes, one knows another has believed. Look into your own center and the true center of others you have known.