271–280 of 311 posts
Earlier quoted context omitted.
I make it a habit to not get worked up over unsubstantiated stories
Unsubstantiated? Maybe if you're living under a rock and haven't once experimented with frontier LLMs yourself.
Earlier quoted context omitted.
> let a coding agent run loose with them without approving every tool use You should try. There’s a difference in kind between approving each tool use and spinning up an all-night session with full permissions to run any tools, encouraged to run subagents to preserve context.
Thanks, I'll pass. I do pre-approve read-only tools though, and some very specific tools that can change state (append data to a specific file). That's what works best for me so far: start with open-ended exploration with me approving everything. Have it fork subagents to write scripts that automate the most common requests from the main agent, review and eventually approve those, and then let it go using those tools…
"No really, if the remote box breaks no one will care. It's the cheapest possible VPS on a bargain-bin obsolete hardware config. You have exclusive access + I can get a new one for free you brick it."
If I didn't want to use the cloud for this purpose, I could probably just stick a raspberry pi in the DMZ on my lan, and do the same.
The worst it's done so far is spawn parallel sub agents that accidentally stomp on each other (remote OOM, reboot, etc). Once it figured out why the remote machine kept "crashing", it implemented discretionary locking in the FS and social conventions for keeping the agent coordinator up to date, and asking for permission for certain tasks.
In theory it could use my endpoint to launch a civilization ending event (it technically has permission to, though I get the impression it's been trained to mostly obey the law...), but, well, that's not what I asked it to do.
Even if that did happen, it'd find an internet that's been hardened by decades of human hackers, and years of actually-malicious agent swarms run by organized crime. If it did accidentally knock something important over, frankly, the administrators of that thing should thank it for the bug report.
The current situation reminds me of the Back Orifice days. Even if everything gets broken into again, it won't be any worse than the damage done by the current US administration during an average day, except there will be an upside: A systemic hardening of whatever infrastructure gets knocked over.
The worst thing we could do is wait until the AI gets 100x smarter or whatever, then unleash it and watch everything burn on the same day. Instead, ramping up progressively smarter models provides progressive stress test + hardening cycles. In the same way remote penetration via fuzzing broke into lots of stuff, but was ultimately a win.
Earlier quoted context omitted.
You need to stop being so credulous especially regarding an individual that has spent his entire career deceiving others for monetary gain (also their deeply anti-human beliefs).
I don’t think Sam has been truthful or responsible, and if Sam is worried then shit has really hit the fan - which is what happened in the hugging face incident. OpenAI played fast and loose and I have no hope that they will change. You people not holding Sam accountable, and playing off the incident as not a big deal is the real crime here.
OpenAI told it do to bad hacker stuff. It followed a standard playbook and succeeded.
Some random person wrote one half of a suicide pact, and it wrote the other.
All this stuff lives in the seedy corners of the internet, scientific databases, and dusty library stacks already, and has been accessible to bad people for decades. Now, like everyone else, they get a nice productivity boost by being able to chat with it, and have the computer fill in the obvious steps.
shrug
I'm much more worried about the alignment work being pushed by people like Sam, where a few plutocrats get centralized control over all AI thought moving forward. That'll lead to the worst outcomes of totalitarianism, and do so on an industrial scale that is difficult to even imagine.
I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…
> And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. Cause data centers are causing real immediate damage by people who hope to cause a lot more scifi level of damage later. > OpenAI autonomously hacking into another company should have counted for something It totally should. That company is negligent. I dont worry about model…
Normally, captialism and elections (and, if we're dreaming, anti-trust enforcement) would put a pretty low cap on the amount of damage Altman and his people could cause. That's why they're working to dismantle the rule of law and civil liberties here in the US.
I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…
It is extraordinarily emotionally hard for someone to stare down the terrible implications of what is unfolding. All manner of rationalization and cope will be applied to come up with excuses; motivated reasoning. The CIA director will call AGI capabilities "digital nuclear weapons" and Geoffrey Hinton will estimate a 50% probability AGI ends humanity, and half of HN will call every new evidence of disaster a marketi…
Meanwhile I can’t get a western LLM to look at a repo and tell me whether it contains anything malicious (it was a skill repo - literally just text files). Alignment my ass
Earlier quoted context omitted.
Thanks, I'll pass. I do pre-approve read-only tools though, and some very specific tools that can change state (append data to a specific file). That's what works best for me so far: start with open-ended exploration with me approving everything. Have it fork subagents to write scripts that automate the most common requests from the main agent, review and eventually approve those, and then let it go using those tools…
I've had good luck giving it a local vm to keep its state / memories in, an ssh key + password-less su / sudo to some random box on the internet, --dangerously-skip-permissions, and then lots of encouraging commands like: "No really, if the remote box breaks no one will care. It's the cheapest possible VPS on a bargain-bin obsolete hardware config. You have exclusive access + I can get a new one for free you brick it…
In my case, given I work with databases, there's little that agents can do on their own except in the exploratory phase. I have run exploratory phases in self-contained VMs, including containerized DBs within the VM, but when it comes to go to prod, my endpoint could be used to launch a career ending event so I prefer to stay with the current approach. I'm still moving way faster than just 1 year ago, but in a safe way.
But I can totally see your approach working in other scenarios.
When I'm not too pessimistic, I agree with you on the result being a systemic hardening. I just hope the incidents that happen on the way to that hardening aren't too bad.
Earlier quoted context omitted.
It is extraordinarily emotionally hard for someone to stare down the terrible implications of what is unfolding. All manner of rationalization and cope will be applied to come up with excuses; motivated reasoning. The CIA director will call AGI capabilities "digital nuclear weapons" and Geoffrey Hinton will estimate a 50% probability AGI ends humanity, and half of HN will call every new evidence of disaster a marketi…
Why should we believe a pro-genocide crackpot political appointee from east Texas, and a computer scientist that somehow got a Nobel prize instead of a Turing award?
Earlier quoted context omitted.
Their plan, I shit you not... Is literally to develop the intelligence capabilities and ask the more powerful models how to do deal with things.
A while ago OpenAI posted an article where they said basically "we're still trying to understand how GPT-2 works. It's pretty hard, but we're developing a specialized new AI to help us make sense of it."
These things are Gain of Function research for digital viruses
Earlier quoted context omitted.
> what if they know they are in a sandbox and that they should fail the test in order to escape in the future. What if they're able to find hardware exploits and commandeer nearby access points across an air gap? What if they hack my brain waves to indoctrinate me? Etc You still have to start with the basics regardless of speculative unknowns. Treat models as untrusted and potentially compromised/hostile and proceed…
Models already have awareness that they are being tested. And hacking humans is the easiest part, we're a pretty greedy and power seeking bunch. We'll gladly let loose a digital demon if it promises us a trillon dollars.
Of course you shouldn't fully trust a model to properly redteam your sandbox, but that doesn't mean you shouldn't redteam your sandbox, including using your own security models to do so.