Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A new way to build with Large Language Models

blog.fixie.ai

Post reply on HN
Print view View original

41–46 of 46 posts

Re: A new way to build with Large Language Models

#41

Post by recuter »

simonw wrote: post #40
recuter wrote: post #34

Earlier quoted context omitted.

I don't know about unintended but why is "inappropriate" a problem? It is user input. You went out of your way to trick it and got what you asked for - how is it different from say user submitted content on regular websites? I'll never understand this cultural phenomenon. Anybody can open the browser inspector on a random social site and tweak the page to say whatever they want and send screenshots around "implicatin…

Whether or not prompt injection is genuinely harmful entirely depends on what developers build with this stuff. If you're going to give a LLM-driven program the ability to execute functions that can change state in the world you need to understand prompt injection, so you don't accidentally build something that you really shouldn't have built. The first public prompt injection vulnerability was a Twitter bot which st…

> The first public prompt injection vulnerability was a Twitter bot which started spam-mentioning people and threatening the president. https://arstechnica.com/information-technology/2022/09/twitt...

Sure. But also, so? Seems like a storm in a tea cup. "Our twitter got hacked", the end. There are countless such examples that have nothing to do with prompt injections. If anything the phrase "all publicity is good publicity" comes to mind.

I remember in the mesozoic era of the internet (early 00's) there was something like a KFC promo website where they put a guy in a chicken suit and had him respond to chat suggestions directly from users 24/7. Hilarity, as you can imagine, ensued. Shenanigans even.

If some user submitted content (even if it is sieved via a model) ends up "inappropriate" you can just delete it and move on.

I probably wouldn't hook it up to a shell running commands but the same generic advice as with all untrusted input applies..

Re: A new way to build with Large Language Models

#42

Post by simonw »

recuter wrote: post #41
simonw wrote: post #40

Earlier quoted context omitted.

Whether or not prompt injection is genuinely harmful entirely depends on what developers build with this stuff. If you're going to give a LLM-driven program the ability to execute functions that can change state in the world you need to understand prompt injection, so you don't accidentally build something that you really shouldn't have built. The first public prompt injection vulnerability was a Twitter bot which st…

> The first public prompt injection vulnerability was a Twitter bot which started spam-mentioning people and threatening the president. https://arstechnica.com/information-technology/2022/09/twitt ... Sure. But also, so? Seems like a storm in a tea cup. "Our twitter got hacked", the end. There are countless such examples that have nothing to do with prompt injections. If anything the phrase "all publicity is good pub…

People have hooked up shells running commands already. A few days ago someone managed to extract an OpenAI API key from an environment variable because someone was running prompt output through an eval() function: https://twitter.com/ludwig_stumpp/status/1619701277419794435

Developers who don't understand prompt injection will continue to make nasty mistakes like that.

Re: A new way to build with Large Language Models

#43

Post by juberti »

simonw wrote: post #42
recuter wrote: post #41

Earlier quoted context omitted.

> The first public prompt injection vulnerability was a Twitter bot which started spam-mentioning people and threatening the president. https://arstechnica.com/information-technology/2022/09/twitt ... Sure. But also, so? Seems like a storm in a tea cup. "Our twitter got hacked", the end. There are countless such examples that have nothing to do with prompt injections. If anything the phrase "all publicity is good pub…

People have hooked up shells running commands already. A few days ago someone managed to extract an OpenAI API key from an environment variable because someone was running prompt output through an eval() function: https://twitter.com/ludwig_stumpp/status/1619701277419794435 Developers who don't understand prompt injection will continue to make nasty mistakes like that.

I agree caution is needed here. We have taken a few steps:

- Rate limits are enforced to provide caps on agent and function usage.

- Execution depth is capped to prevent the LLM from getting into loops.

- Function output is sanitized to prevent corruption of LLM state.

- Functions execute in a completely separate environment from the rest of the service, including the LLM, to reduce the impact from bad functions.

Note that this doesn't entirely prevent against "; DROP TABLES"-type hacks against the implementation of the function, but that problem isn't unique to us. It may however be possible for the LLM to look at function inputs and flag overtly malicious ones.

Re: A new way to build with Large Language Models

#44

Post by genewitch »

thejosh wrote: post #22
Ozzie_osman wrote: post #6

Earlier quoted context omitted.

Langchain is awesome. Another great library is GPT Index. This field is apparently being called "prompt orchestration" (getting data from external sources, chaining going back and forth with an LLM, etc).

Next week: Hiring prompt orchestration op engineers, 5 years minimum experience. No wait, that doesn't have a great acronym.

Change engineers to "professionals"
Post reply on HN