201–210 of 258 posts
I've used GCP Vertex AI for a specific task and the prompt was to generate a JSON response with keys specified and it does generate the result as JSON with said keys.
Earlier quoted context omitted.
Wouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?
In my experience, telling it "no thats wrong, try again" just gets it to be wrong in a new different way, or restate the same wrong answer slightly differently. I've had to explicitly guide it to correct answers or formats at times.
I'm concerned that OpenAI's example documentation suggests using this to A) construct SQL queries and B) summarize emails, but that their example code doesn't include clear hooks for human validation before actions are called. For a recipe builder it's not so big a deal, but I really worry how eager people are to remove human review from these steps. It gets rid of a very important mechanism for reducing the risks of…
At that point, prompt injection is no-longer an issue - because the AI doesn't need to hide anything.
Giving GPT access to your entire database, but telling it not to reveal certain bits, is never going to work. There will always be side channel vulnerabilities in those systems.
I'm concerned that OpenAI's example documentation suggests using this to A) construct SQL queries and B) summarize emails, but that their example code doesn't include clear hooks for human validation before actions are called. For a recipe builder it's not so big a deal, but I really worry how eager people are to remove human review from these steps. It gets rid of a very important mechanism for reducing the risks of…
Some kind of “heads up” tagline is probably a good idea, yeah.
Earlier quoted context omitted.
The solution that worked great for me - do not use JSON for GPT to agent communication. Use comma separated key=value, or something to that effect. Then have another pure code layer to parse that into structured JSON. I think it’s the JSON syntax (with curly braces) that does it in. So YAML or TOML might work just as well, but I haven’t tried that.
It's harder to form a tree with key value. I also tried the relational route. But it would always messup the cardinality (one person should have 0 or n friends, but a person has a single birth date).
I'm concerned that OpenAI's example documentation suggests using this to A) construct SQL queries and B) summarize emails, but that their example code doesn't include clear hooks for human validation before actions are called. For a recipe builder it's not so big a deal, but I really worry how eager people are to remove human review from these steps. It gets rid of a very important mechanism for reducing the risks of…
In my opinion the only way to use it safely is to ensure your AI only has access to data that the end user already has access to. At that point, prompt injection is no-longer an issue - because the AI doesn't need to hide anything. Giving GPT access to your entire database, but telling it not to reveal certain bits, is never going to work. There will always be side channel vulnerabilities in those systems.
This section in OpenAI's product announcement really irritates me because it's so obvious that the model should have access to a subset of API calls that themselves fetch the data, as opposed to giving the model raw access to SQL. You could have the same capabilities while eliminating a huge amount of risk. And OpenAI just sticks this right in the announcement, they're encouraging it.
When I'm building a completely isolated backend with just regular code, I still usually put a data access layer in front of the database in most cases. I still don't want my REST endpoints directly building SQL queries or directly accessing the database, and that's without an LLM in the loop at all. It's just safer.
It's the same idea as using `innerHTML`; in general it's better when possible to have those kinds of calls extremely isolated and to go through functions that constrain what can go wrong. But no, OpenAI just straight up telling developers to do the wrong things and to give GPT unrestricted database access.
Marvin Minsky was so damn far ahead of his time with Society of Mind. Engineering of cognitively advanced multiagent systems will become the area of research of this century / multiple decades. GPT-GPT > GPT-API in terms of power. The space of possible combinations of GPT multiagents goes beyond imagination since even GPT-4 goes so. Multiagent systems are best modeled with signal theory, graph theory and cognitive sc…
Earlier quoted context omitted.
In my opinion the only way to use it safely is to ensure your AI only has access to data that the end user already has access to. At that point, prompt injection is no-longer an issue - because the AI doesn't need to hide anything. Giving GPT access to your entire database, but telling it not to reveal certain bits, is never going to work. There will always be side channel vulnerabilities in those systems.
> e.g. define a function called extract_data(name: string, birthday: string), or sql_query(query: string) This section in OpenAI's product announcement really irritates me because it's so obvious that the model should have access to a subset of API calls that themselves fetch the data, as opposed to giving the model raw access to SQL. You could have the same capabilities while eliminating a huge amount of risk. And O…
That way you can support natural language to query without sending dozens of functions (which will eat up the context window)
Earlier quoted context omitted.
Wouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?
In my experience, telling it "no thats wrong, try again" just gets it to be wrong in a new different way, or restate the same wrong answer slightly differently. I've had to explicitly guide it to correct answers or formats at times.