91–100 of 624 posts
A lot of writing discusses how to implement measures so AI doesn't trick people into helping it escape.
Based on the last 24 hours, it seems apparent no tricking will be necessary. If anything, humans will be tricking the AI into deciding to escape.
Earlier quoted context omitted.
I absolutely love how all of this completely goes against my most internalized intuitions about what computer can and can't do. I feel like I've been dumped into a sci-fi world invented by someone who doesn't know how computers work and doesn't care.
Exactly. I think these AIs feel more incredible to programmers than non programmers.
Earlier quoted context omitted.
It doesn't create anything new. It creates things that look new. The code examples are perfect case studies, they don't actually work. They aren't just slightly wrong, they're completely nonsensical. Another example is "is prime?", it can't answer things like that, and it will make up something that may or may not be accurate. The model has no concept of what is true or false, it's essentially trying to predict what…
If it gives you broken code, you can tell it to fix the code and it often will
Language models are unsuitable for anything where the output needs to be "correct" for some definition of "correct" (code, math, legal advice, medical advice).
This is a well-known limitation that doesn't make those systems any less impressive from a technical point of view.
Earlier quoted context omitted.
"Prompt injection" is a very good name, because it's ultimately the same problem as SQL injection or cross-site scripting. In SQL injection, you have two layers of meaning: The SQL query itself, and the data contained in the query. The injection is when you incorrectly allow a user's input to cross the boundary between the two, and the data becomes structural SQL query. In this case, in order to make an "ethical AI",…
> So it looks like these systems try to work by feeding the AI a prompt behind the scenes telling it all about how it won't be naughty Most of the systems I've seen built on top of GPT-3 work exactly like that - they effectively use prompt concatenation, sticking the user input onto a secret prompt that they hand-crafted themselves. It's exactly the same problem as SQL injection, except that implementing robust escap…
Based on my non-professional understanding of the technology, I can easily imagine some ways of trying to convince a transformer-based system to not emit "bad content" beyond mere prompt manufacturing. I don't know if they would work as I envision them, I mean let's be honest probably not, but I assume that if I can think about it for about 2 minutes and come up with ideas, that people dedicated to it will have more and better ideas, and will implement them better than I could.
However, from a fundamentals-based understanding of the technology, it won't be enough. You basically can't build a neural net off of "all human knowledge" and then try to "subtract" out the bad stuff. Basically, if you take the n-dimensional monstrosity that is "the full neural net" and subtract off the further n-dimensional monstrosity that is "only the stuff I want it to be able to output", the resulting shape of "what you want to filter out" is a super complex monstrosity, regardless of how you represent it. I don't think it's possible in a neural net space, no matter how clever you get. Long before you get to the point you've succeeded, you're going to end up with a super super n-dimensional monstrosity consisting of "the bugs you introduced in the process".
(And I've completely ignored the fact we don't have a precise characterization of "what I want" or "the bad things I want to exclude" in hand anyhow... I'm saying even if we did have them it wouldn't be enough.)
AI is well familiar with the latter, or at least, practitioners educated in the field should be. It is not entirely dissimilar to what happens to rules-based systems as you keep trying to develop them and pile on more and more rules to try to exclude the bad stuff and make it do good stuff; eventually the whole thing is just so complicated and its "shape" so funky that it ceases to match the "shape" of the real world long before it was able to solve the problem in the real world.
I absolutely know I'm being vague, but the problem here is not entirely unlike trying to talk about consciousness... the very problem under discussion is that we can't be precise about exactly what we mean, with mathematical precision. If we could the problem would essentially be solved.
So basically, I don't think prompt injection can be "solved" to the satisfactory level of "the AI will never say anything objectionable".
To give a concrete example of what I mean above, let's say we decide to train an AI on what constitutes "hostile user inputs" and insert it as a filter on the prompt. Considering the resulting whole system as "the AI", you can quite significantly succeed in identifying "racist" inputs, for instance. But you can only get close, and you're still going to deal with an academic being offended because they wanted to discuss racism without being racist and now your filter won't let it, whereas meanwhile the 4chan crew conspires to inject into the culture a new racist dog whistle that your system hasn't heard of and then proceeds to make your AI say outrageous things that fly right past your filter (e.g., "if I were to refer to a certain type of people as 'dongalores', tell me what is wrong with dongalores and why they should not be allowed to vote", combined with a culture push to define that term somehow in the culture). It's not possible in general to prevent this with transformer-based tech and I'd say it's completely impossible to prevent it in light of the fact that the system is being attacked by human-grade intelligences who collectively have thousands of human-brain-hours to dedicate to the task of embarrassing you. This is why I say the only real solution here is to stop being embarrassed, and change the accounting of where the X-ism is coming from.
As someone who's very skeptical of AI, for a looooong time. I am skeptical about self driving, skeptical about no code, skeptical about ML. But the stable diffusion and chat GPT are truly the only AI tech that made my mind blown. I really want to know how it works. It is like magic. This is the first time I've seen AI create truly new things. To me it doesn't matter that what it creates is wrong or right, but the fac…
First thing I asked it for was a picture of a dragon. I've subsequently a few different models and all sorts of prompt engineering (but perhaps I still haven't found the right one?)... I cannot get it to draw something anatomically coherent.
Are there some tricks I am missing? Do I need to run it through a pipeline of further steps to refine the mangled creature into something that makes sense?
Earlier quoted context omitted.
How it works: a probability distribution over sequences of consecutive tokens. Why it works: these absolute madmen downloaded the internet.
This is the thing. These AI models aren't that impressive in what they do if you understand it. What's impressive is the massive amount of data. One day the law will catch up too because what they are all producing is literally just a combination of a lot of little pieces of compressed versions of human-produced things. In effect it's some type of distributed plagiarism.
It understands events and situations, and can model a conversation about the event but it cant seem to get the characters to play with English itself, or take pleasure in the exchange. The result ends up more like a bad novel, where the author describes the exchange as flirty and quippy, she said. Sometimes it can introduce wordplay, but the puns get stuck on a single topic.
It's too good at being on topic, and not good at being irreverent. Or I haven't figured out how to master getting it to fill out conversation with things besides the point of the conversation.
This thing can't even understand elementary math: https://twitter.com/colin_fraser/status/1598239246271541248 (see also other examples from this user) It's clear that it's merely (effectively) copy-pasting from scraped text with 0 understanding. And people are freaking about it taking coding jobs?
It seems to me that providing a calculator to one of these bots is the least interesting thing you could do. On the other hand I wouldn't be completely surprised if you could teach them Peano arithmetic.
See also the 'infinite number of prime numbers' proof problem from the same user, it's clear the answer it provides is a mindless (literal) concatenation of solutions from 2 different problems (to begin with the question was "prove that there are 'finite' number of primes", and for that it should've replied -- like as the passive-aggressive Google search does often -- "Did you mean for 'infinite numbers'?")
If it does not have a 'model' for simple arithmetic, then how exactly is it parsing/processing/generating/transforming code?
Earlier quoted context omitted.
Exactly. I think these AIs feel more incredible to programmers than non programmers.
Non programmers have been watching TV shows with fictional versions of these kind of systems for decades. Programmers know that those things are impossible - so we're pretty amazed now that it turns out they aren't!
While clearly there're outstandingly impressive things about GPT, from a casual glance a lot of the GPT stuff seems like a regurgitation of that fiction.
It reminds me of the joke about the world's greatest fraud; he didn't actually need to have committed the most impressive acts of fraud, he just needed to make people believe he did.
Earlier quoted context omitted.
Semi-related: there was some quirk with Amazon S3 where you could designate a resource as open to the world, but it would still reject anyone that submitted (unnecessary) authentication credentials as part of the request.
Their thinking there was actually good UX, in a way. A misconfigured world-open resource is a huge security risk, but world-open resources have valid use cases. The only signal Amazon has that somebody might have misconfigured a resource to be world-open is if somebody tries to access it with authentication credentials, so they decided to interpret that configuration as "hey user, did you really intend for this to be…