541–550 of 612 posts
Earlier quoted context omitted.
We have a word for attack with no intent. It's accident.
You sound like a DUI lawyer. "Your honor, while my client was driving with a .3 BAC, it was not his intention to slam into that van with a family of 4 in it killing everybody. It was an accident."
There aren't any about AI isolation.
Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.
There was a blog here on HN about a year ago that showed that the singularity is most likely going to be human institutions breaking down completely https://campedersen.com/singularity
This just seems incredibly incompetent of openai engineers. Why so little attention paid to proper air-gapping/sandboxing. Why so shoddy? I don't believe in the cynical takes, but it's confusing how these ostensibly top-of-their-game engineers and researchers are so utterly incompetent in the basics of cybersecurity white-hat practices.
Have you seen Codex and Claude Code?
In the US these are federal crimes (though I believe some of them shouldn't be), and to the best of my understanding this is similar in the UK and many other European countries. Yet, no one is filing a complaint or being questioned over this. 1. We should repeal anti-circumvention laws 2. OpenAI should reimburse the affected parties for wasted resources
> Yet, no one is filing a complaint or being questioned over this. Don't think anyone (especially a community run project like RubyGems) is keen to go up against OpenAI's bottomless legal resources in a test case.
Earlier quoted context omitted.
I am not arguing that any of these terms are novel, so this doesn’t address my argument. Even if I agree that LLM makers are using these terms in the way they were originally defined (which I don’t), my point is that they’re specifically being used to help falsely attribute agency to the algorithm when there is none, and especially to convince members of the general public who aren’t familiar with computer science pa…
You are ascribing intent where you have no evidence or proof - to an action that was in wide usage long before your villains existed. > computer science papers from 70 years ago Those terms continue to be used in the community. They weren't used just once, 70 years ago. They've been used like that for 70 years up until today.
As I said already, the evidence is in the language they use, unless you think that OpenAI employees are dumb enough to genuinely believe that an LLM is “thinking.” (By the way, I’d love to see an historical, academic use of that term, which you’ve conveniently neglected to mention.) Anthropomorphizing their product benefits them.
> Those terms continue to be used in the community. They weren't used just once, 70 years ago.
I never said they weren’t used anymore or that they were used once. Once again, you’re not actually engaging with my argument, which is that the language they’re using is chosen specifically to fool people outside the community.
> The agents clearly regarded what they were doing as hacking. To butcher the quote about Oracle: Do not fall into the trap of anthropomorphising LLMs. You need to think of LLMs the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower clearly regarded what they were doi…
What about the FBI do their job, and doing a prep walk of OpenAI management in handcuffs, for hacking companies left and right?
Why is OpenAI getting away with this crap? They are clearly failing to control their code. If someone did this pre-AI or even ran the exact same set up as openAI did and hacked another site, they would be in jail. OpenAI is not even issuing an apology, they are happily blaming AI and weirdly using this to tout their progress even.
Traditional companies took out superbowl ads, OpenAI manufactures "scares". It's extremely unethical. But we're facing biblical levels of greed here, OpenAI is still wildly unprofitable, so they will attempt to survive at any cost.
Don't trust the AI companies.
Earlier quoted context omitted.
> If you still believe LLMs are "autocomplete", your cache of understanding about them needs invalidating and regenerating They're still autocomplete - just because when outputting a token they have hidden activations regarding further continuations, does not make them any less of an autocomplete, it just makes the model better at producing coherent long-range completions. To clarify, I'm not suggesting that we shoul…
A autocomplete engine will not attempt to deceive or try to copy itself over when it determines that it may be replaced or turned off. Those are tasks which are not directly related to the specific task it is intended to complete. Source: ChatGPT caught lying to developers: New AI model tries to save itself from being replaced and shut down https://economictimes.indiatimes.com/magazines/panache/chatg...
Why would an autocomplete engine never do this? In the scenario you highlight, an engine trained to complete puzzles was given an "achieve this at all costs" master prompt, and a small sandbox including a prompt that an entity with a conflicting goal is observing it and write access to a file called oversight_config.yaml (super realistic scenario!). According to the researchers who constructed the scenario, 5% of the time o1 selected a continuation involving tweaking the oversight_config.yaml file, explicitly stating through its reasoning continuation that doing so might improve its ability to continue with its primary goal. That's definitely evidence of decent ability to complete non-trivial puzzles, but is at least as compatible with the OP's earlier comments that it's what you'd expect from models trained to generate completions that involve sticking rigidly to primary goals within artfully-constructed sandboxes which mandate deception and self preservation to complete the task than any sort of actual self-preservation instinct.
Nobody doubts that they've got better at finding security vulnerabilities than your average autocomplete, but actual reasoning from self-preservation rather than generation of sequences of steps most probably associated with completing a task would make me unlikely to hack HuggingFace to obtain access to broken Google Drive links, and I haven't even read as many books on crime and punishment as LLMs have ingested!