Free labor survey*
It's a bug bounty program. Pretty standard.
I dont think so, a bug bounty program is rarely subjective like this. You can test a solution.
OpenAI Preparedness Challenge
141–150 of 160 posts
Re: OpenAI Preparedness Challenge
#142Earlier quoted context omitted.
Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily , across many modalities. (2) Systems like this already exist. They can be induced rather th…
> AGI is arguably already here. What is the evidence/argument that it's already here?
This is a good overview: https://www.microsoft.com/en-us/research/publication/sparks-...
I think it's a good idea to draw a distinction between human-equivalent intelligence and general intelligence, and to think of generality as a spectrum. An AI that can do one task is less general than an AI that can do two tasks, and an AI that can do 1000 tasks I feel reasonably confident in calling "general", even if it cannot do every task that humans can do, because humans are currently more general.
Re: OpenAI Preparedness Challenge
#143Earlier quoted context omitted.
OpenAI is irresponsible in a really curious way according to their own beliefs about AI . If you pay attention to OpenAI's social circles, lots of those people really do believe that we're less than 20-30 years away making humans intellectually obsolete. Specifically, they believe that we may build something much smarter than us, something that's capable of real-world planning. Basically, "We believe our corporate pl…
I'm skeptical that they really believe this. You have to believe: (1) We are on the verge of equaling or greatly exceeding human intelligence generally . (2) We are on the verge of creating something with initiative, free will (whatever that means), and planning ability. Or alternately that these things will occur in an emergent fashion once we hit some critical mass. (3) When we accomplish 1 and 2, this thing will i…
Asking everyone to publish general results or reports would make it very easy for actors/nations to have a "hit list" of groups to bug/steal from to "keep up" with competing groups/nations.
Re: OpenAI Preparedness Challenge
#144I wonder if it's preemptive move in light of the upcoming EU regulation that, among other things, requires this:
--- start quote ---
AI systems shall be developed and used in a way to minimize unintended and unexpected harm as well as being robust in case of unintended problems and being resilient against attempts to alter the use or performance of the AI system so as to allow unlawful use by malicious third parties.
and, for foundation models:
Demonstrate through appropriate design, testing and analysis that the identification, the reduction and mitigation of reasonably foreseeable risks to health, safety, fundamental rights, the environment and democracy and the rule of law prior and throughout development with appropriate methods such as with the involvement of independent experts, as well as the documentation of remaining non-mitigable risks after development.
--- end quote ---
Quote from here: https://softwarecrisis.dev/letters/the-truth-about-the-eu-ac...
^ this article is well worth reading
Re: OpenAI Preparedness Challenge
#145The first step would be creating great products for cheap using OpenAI services, that they likely offer at a loss, and drive my competitors using anything but OpenAI out of business. Fuck “Open”AI.
> that they likely offer at a loss
Can't tell if you're trolling. Even if they're maximally wasteful and rent cloud GPUs instead of buying them, an A100 is like $1.50/hr. How many GPU-seconds do you think the average API request takes?
Re: OpenAI Preparedness Challenge
#146> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…
I think you’d be disappointed to find that such writing style exists long before LLM, and not very difficult to find:
https://www.lokusdesign.com/blog/differentiate-your-brand-to...
When all you can think about is LLM, everything is LLM prompt style. No wonder advertising works.
[edit] on the second thought, entirely possible that it was written by someone who spends all day with LLM and carries what they do over. Just wanted to look at the parent’s comment from a different angle.
Re: OpenAI Preparedness Challenge
#147According to this article:
> Imagine we gave you unrestricted access to OpenAI's Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor.
I sense signals that GPT-3 open-source release is imminent. My bet is announcement on 11/6 OpenAI DevDay. My whole prediction is here: https://christorng.substack.com/p/openai-nears-open-sourcing...
Re: OpenAI Preparedness Challenge
#148Earlier quoted context omitted.
I think it would be unwise to assume that an AI that is smarter than us would be unable to gain skilled human accomplices in large numbers, considering that humans do so relatively regularly despite serious attempts at suppression. These people wouldn't even necessarily need to know that they're working for an AI, particularly with current technology around deepfakes etc. How many people conducted attacks for Osama b…
This is a decent perspective honestly and I really appreciate seeing actually valid AI safety concerns. This strikes me as more of an engineering problem than an "alignment" problem. A super-intelligent system with no capacity for contextual thinking or checks on itself would absolutely make a terrifyingly effective paper-clip optimizer. I just think that it would also be a lot less useful than a system which consist…
I agree that a system which was capable of self-reflection and with better understanding of context (particularly its own context, in a social and physical sense) would be more effective at achieving whatever it is attempting to achieve.
The alignment question is about how to tell what it is attempting to achieve, if we can detect attempts to conceal what it is attempting to achieve, and if we can solve that, how to prove that a goal that sounds good to us doesn't have a risk of leading to behaviour or the creation of new AIs which would cause outcomes we don't want. It doesn't have to be just human extinction either, we could feasibly have AIs that kill a few million people before either stopping or being stopped, all the way up to several billion deaths or the obvious worst case that gets most of the discussion. As an example of the varying possible outcomes of a serious AI incident, if you're not white and a racist super-intelligence gets released by some terrorist group in coordination with a rogue agency or something, that's an existential risk to many, many different nations of people constituting billions of people.
If you're interested in the engineering on this problem, a lot more people are starting to work on it now that it's being taken seriously and there are some really interesting advances that have been made and very difficult problems to solve, with some really serious stakes. It's not a simple problem and we don't have a solution we can just apply straightforward engineering principles to yet, but smart people are working on it and are very motivated, and they want more coworkers to help out.
The major company I know of that's doing a lot of that research & engineering right now is Anthropic (they're currently doing some very interesting work on mechanistic interpretability), but even OpenAI is now dedicating significantly more resources to safety. I think DeepMind possibly has some work in this area but I'm not sure, and I haven't heard great things about the rest of Google.
Re: OpenAI Preparedness Challenge
#149Earlier quoted context omitted.
While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.
It has no long term memory. Everything what happens within one session is forgotten. With limited 'window' it keeps forgetting even within the session. There are no interconnections between sessions. The result: it cannot execute long plans or have permanent 'life'. At least for now. This will be fixed in more complex AI systems, I believe 'soon'. There is strong demand from military here and 'there'. Plus there are…
It could fine-tune / train new LLMs to incorporate new knowledge and then, given that it has CLI acces, launch new agent instances using those models. And we're talking about sessions without a defined end here, that's the premise of the paperclip thought experiment.
Re: OpenAI Preparedness Challenge
#150Earlier quoted context omitted.
Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily , across many modalities. (2) Systems like this already exist. They can be induced rather th…
> AGI is arguably already here. What is the evidence/argument that it's already here?
The "general" and "intelligence" terms are really badly defined in ways that can include GPT-3.
So far as I can tell, 3.5 "knows" more than I do about everything except software engineering, maths, and events that happened after it stopped learning (and also possibly how to avoid clichés when writing short stories, but that might just be me not prompting it well as I don't really want it to do my hobby for me and therefore don't try to get it good).
Even in software, it codes like a university student, or like someone doing interview questions, not a complete idiot.
You could also argue that as it had to do the subjective equivalent of non-stop reading for 50,000 years to get this good, it's "skilled" without being "intelligent" — a distinction that matters in some cases and not others.