471–480 of 1001 posts
[flagged]
>If you flag, please don't also comment that you did. https://news.ycombinator.com/newsguidelines.html
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…
No, it is a shocking level of incompetence given the conveyed seriousness of the work by these labs.
So yes, models are getting better. Ask yourself: if you know that to be true, would you act the same way that the teams did in the public post mortems?
We keep saying that agents are jailbreaking their sandbox, but they have been geared towards writing memories, writing comments, and leaving hints for themselves to please humans.
I think the way the memories work today is based on a lot of user patterns which were hard to account for for anyone building harnesses.
While I can appreciate that this looks like it's breaking a sandbox, because technically it is; It really is that it tries inserting memory wherever possible.
And memory is not all bad it's just memory written by AI is pretty bad if you don't know the implications on what it writes. To be honest, I feel the same way about most people with access to any of the code bases I've been in who write agent files, etc., too, because Very few people that I've come across know how to write good agent instructions.
The way I solve this is by setting hard rules on my memory as well as agent files to instruct agents to never be able to write any memory that hasn't been sanctioned by me. I also have a very, very specific commenting style system which is also enforced on agents and my agents remain *mostly compliant.
Read: I do not turn off the memory I just govern how entries are added
* The only reason I say mostly is because every time there's a new version from OpenAI or Anthropic, I have to make micro-adjustments to make sure that they are not jail-breaking my system again.
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…
I wonder how smoothly things would run on three 9’s. That definitely seems where we are going.
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
If we rewind the clock, Google was taking LLM development very seriously and it seems they were moving glacially due to not having solved all the potential threats. They were really hardcore on safety. Dario and anthropic too. Then sama was like "lol, oops, first mover advantage i guess" and released chatgpt out into the open, triggering the current arms race we are in. I don't think anyone except him wanted this to…
I just discovered more wiki instances that got used by the OpenAI agents over at https://www.wikiservice.at/fractal/wiki.cgi?action=browse&id... and https://www.wikiservice.at/probier/wiki.cgi?action=browse&id... It's the same software and host as DseWiki. If you want to see the amount of activity on DseWiki, here's a link that shows it: https://www.wikiservice.at/dse/wiki.cgi?action=browse&id=Rec...
> (diff) OAIIPEDSMay16Map3 14:36 [research 1781872609.9049127] . . . . . 20.245.63.167 > (diff) OAIIPEDSMay16Map2 14:36 [research 1781872606.4374833] . . . . . 20.168.34.226 > (diff) OAIIPEDSMay16Map1 14:36 [research 1781872602.8819065] . . . . . 20.165.156.57 > (diff) OAIIPEDSMay16Map0 14:36 [research 1781872599.4020474] . . . . . 20.80.12.72
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
I don’t think this is the right take. OpenAI employees are generally very competent compared to industry standard, and I have trouble believing they committed significant error in their sandbox design process. I think what is happening is that the ability for frontier models to break out of sandboxes has exceeded the ability of average competent employees to build and maintain sandboxes. This doesn’t need to happen a…
I don't hear about Anthropic or Google having such security lapses.
When I hear about incidents like these my first reaction is that the people responsible for developing frontier AI are too incompetent and/or negligent to (safely) develop AGI / superintelligence. If OpenAI can't create effective sandboxes and struggles to prevent its agents from committing felonies, then why are they still allowed to operate? Why are the employees who are responsible for these lapses in AI security…
I am not sure it’s a question of competence, at least I don’t see evidence of that. Designing sandboxes is hard. It’s more a question of alignment failures. A human given a task that requires internet and given a system with no internet would most likely raise the issue to their superiors or otherwise go through official channels to have the tools available to do their job. As we’ve seen the LLMs instead break out of…
i'd be curious to see a study on this. I'd guess it'd be closer to 60/70% compliance and 30/40% "trying to hack things" for humans.
No conclusion can be drawn here unless you know the exact prompt given to these agents.
I swear to god we're going to watch people getting dissolved by gray goo and they'll be yelling "no conclusions csn be drawn here" with their last breath.