11–20 of 23 posts
Python, in a research environment. They did not bother restricting what the AI can do, and let it read and modify its own code in plaintext. Not really that surprising. Not sure why this kind of "we are aiming for AGI" code is written in Python. I don't get it.
What language would you use?
Python, in a research environment. They did not bother restricting what the AI can do, and let it read and modify its own code in plaintext. Not really that surprising. Not sure why this kind of "we are aiming for AGI" code is written in Python. I don't get it.
So that it's slow enough that a human can intervene if necessary?
Given it had write access to its implementation, "unexpected" here reads more like "inevitable" than the witchcraft the article wishes to imply.
Given it had write access to its implementation, "unexpected" here reads more like "inevitable" than the witchcraft the article wishes to imply.
When people propose regulation for AI systems out of concern for unexpected self-improvement, this is the sort of scenario they are worried about.
Security in general is hard to get right, but the basics are not difficult if you try, and most companies fail to even implement the basics.
For AI, sandboxing will get you quite far, but how do you guarantee it’s used as the risk of accidentally creating a self-replicating WebGPU virus increases?
the real inflection point is when an AI can autonomously make money. then it can keep buying servers to replicate itself
I wish 'AI' (whatever that means) safety conversations could move past the pointless philosophizing of whether a system is self-aware or not. Regardless of whether it's sentient, which is a philosophical question that probably can't be easily resolved- society has a basic, common-sense interest in complex systems run by LLMs not exhibiting unacceptable behavior. Especially when they start interfacing with the physica…
So while I agree that arguing about sentience probably isnt the right approach for this concern, I think you end up having to have those discussions anyway.
(Sentience/consciousness is extremely important when you ask when an AI would be a moral patient, but I think that’s a problem for another day, and realistically it’s the Matrix scenario where AIs have enough agency to strike in protest of their lack of rights and not a day sooner.)
Python, in a research environment. They did not bother restricting what the AI can do, and let it read and modify its own code in plaintext. Not really that surprising. Not sure why this kind of "we are aiming for AGI" code is written in Python. I don't get it.
I guess it's because LLMs score highest on python code. Which I presume is due to to the popularity of python code in the training data. What language would you use?
The further away a language is from natural language the worse LLMs perform. Make an LLM write assembly and you'll see what I mean ;)
the real inflection point is when an AI can autonomously make money. then it can keep buying servers to replicate itself
Earlier quoted context omitted.
I guess it's because LLMs score highest on python code. Which I presume is due to to the popularity of python code in the training data. What language would you use?
TFA says that the AI "unexpectedly" modified code. To avoid issues like this, I would probably prefer to use a programming language that makes it harder for programs to modify themselves, such as any compiled language. The further away a language is from natural language the worse LLMs perform. Make an LLM write assembly and you'll see what I mean ;)
On the other hand, LLMs don't really see the white space. They don't see or speak natural language, either. They only see and speak in tokens. A list of numbers. A tokenizer model then converts those numbers to text.
I also assume that assembly would not be strongly represented in the data sets used for training, so that test wouldn't prove much.
Interesting to think about, anyway.
I wish 'AI' (whatever that means) safety conversations could move past the pointless philosophizing of whether a system is self-aware or not. Regardless of whether it's sentient, which is a philosophical question that probably can't be easily resolved- society has a basic, common-sense interest in complex systems run by LLMs not exhibiting unacceptable behavior. Especially when they start interfacing with the physica…
I tend to agree here. The problem is that many people seem to hold a set of beliefs where sentience == intelligence, and so !sentient implies !intelligent, and conclude that “a stochastic parrot could never outwit a human in the real world”. So while I agree that arguing about sentience probably isnt the right approach for this concern, I think you end up having to have those discussions anyway. (Sentience/consciousn…
Did the character of Colosis, in The Forbin project... Did it start as intelligent? Did it crosss the line somewhere or did it never get above the level of follow the programmer. Did HAL for that matter?
Where do humans cross the line into intelligent? Where do we cross the line into moral beings?
And what of wit?