611–620 of 1001 posts
"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…
Earlier quoted context omitted.
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…
> write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. Hey GPT-5 now write the code for the antidote.
It’s a very asymmetrical game. A virus is a special arrangement of a few thousand atoms, an antidote is a global effort and strained economy
The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…
"The Dark Forest" is a science fiction novel by Liu Cixin, which introduces a concept called "suspicion."(“猜疑链”) In this concept, the interaction between civilizations is viewed as an ongoing game of suspicion, where each civilization is worried that the other civilization may pose a threat to it. This kind of suspicion makes it difficult for civilizations to establish trust, thereby making the universe a dark forest…
But ok. Let's dig into this
"We can apply this concept of suspicion to the phenomenon in the field of modern artificial intelligence research. In AI research, some researchers believe that humans should pause the development of AI technology because of the potential unforeseeable risks it may bring."
An analogy would be that some of the scientists involved with the Manhattan project (see Szilárd petition) attempted to file an official protest against the use of nuclear weapons against civilian population and arguing that the weapons constructed were too dangerous to be a part of an arms race and should be placed under an international control body.
"However, another group of researchers believe that we should not stop because we cannot be certain whether those who are trying to catch up with us will use the advancement of AI technology as a hidden weapon."
Researchers are not decision makers in this or any other scenario. AI technology as a weapon? Sure but unless you are Bombs Away LeMay circa 1961 this really isn't a winnable or even useful use case. We're still in a blue sky phase. The fun thing about nuclear weapons is that if any nation feels that another is pulling too far ahead they can be used as a way to push for bi/multilateral discussions and eventually treaties limiting the production and/or deployment of weapons that risk of increasing tensions/lethality and so on . (see SALT I/II and START etc)
"This suspicion leads to a chain reaction, where people continue to worry about each other's motives and abilities, making it difficult for all parties to establish trust. Therefore, although there are good intentions, such proposals are unlikely to be implemented."
Again... read Schelling... this is just nonsense.
Earlier quoted context omitted.
Ask yourself two questions: 1. What is the ultimate AI are we trying to eventually build? 2. What are the consequences if we eventually succeed in building such an AI? For me, the answers would be 1. A general AI that would equal and surpass human intelligence. 2. God only knows. Will we get there? Who knows. We do know, however, that the consequences are great enough that only a fool would not at least attempt to pr…
Von Neumann Probe, and the consequences are potentially discovering alien life, colonizing the universe, and exploring the frontiers of science.
This would be totalitarian. This is arguing for a group of people to have the power to decide some field is "unsafe" as per some vague, unverifiable criteria, then set up a police structure to verify compliance, all outside the safeguards of democratic or judicial norms. Precautionary principle run amok.
Run amusk.
> Should we automate away all the jobs, including the fulfilling ones? This is the part about generative AI that gives me the most profound anxiety about our future. Every thing I know about human psychology tells me that in order for people to thrive, we need to be needed by others. A world where everyone is more or less alone consuming media automatically generated machines might be a world where people nominally h…
There seems to be a contradiction in the picture of the world you're drawing. Humans deeply need to be needed by others, yet when all our problems are solved we will sit around alone consuming media? If we were truly in that situation, why wouldn't our deep needs drive us to spend our days with friends and family, pursuing creative projects, and inventing new problems to solve?
* Eat a bag of chips instead of cooking a meal.
* Watch a travel show instead of leaving the house.
* Binge watch a vlogger and become a superfan instead of nurturing a real symmetric friendship.
* Read books about how to draw but never fill a sketchbook.
* Buy a gym membership but never go.
Because delaying gratification is hard and we're just poor apes doing the best we can.
Earlier quoted context omitted.
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…
I dunno, this sort of scenario really doesn’t worry me too much. There are thousands (maybe tens of thousands) of subject matter experts who could probably develop dangerous weapons like you describe, but none of them seem to just wake up in the morning and decide “today’s the day I’m going to bring the apocalypse”. I don’t think that this really changes that.
Now if this button was something you had to order from Amazon I think we’ve got a few days
There’s a scenario where people with the intent will have the capability in the foreseeable future
It is possible for humans to demonstrate this level of maturity. We can do it. Some prime example tasks are: good stewardship of the environment and planet, working on battery and energy tech, creating more sustainable human habitats, cessation attention-based addictive products like those produced by Meta and Tik Tok. Embracing human health through exercise and sleep. Encouraging good-faith listening and civil debate- and ignoring partisan click bait meme garbage. When humans can rise to those challenges- then we are ready to investigate powerful AI.
Edit to add:
Allocation of efficiency gains via technology to directly address poverty.
To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…
The problem is that anyone with sufficient money could rent out the infrastructure required to train GPT5 or what-have-you. They need not share it publicly, they can use it for their nefarious purposes privately. The analogous scenario would be preventing your military from experimenting with TNT, when the recipe for TNT is public knowledge. You can't copyright away public knowledge.
The solution is to allow as many different entities as possible to develop their own AIs so that they end up competing with each other and providing alternatives. What we don't want is a single AI which dominates our entire belief system.