I wonder how long it will take before people start to use LLMs to generate prompts...
People are already "chaining" AIs, eg, using GPT to generate prompts for Midjourney, etc
Prompt Engineering: Steer a large pretrained language model to do what you want
11–20 of 53 posts
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#12Lilian Weng's blog is my go-to example for an extremely high quality tech blog, it's truly remarkable how consistently excellent each post is. The only downside is the sadness I feel for being incapable of producing content even remotely near that level of quality myself.
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#13I find it fascinating that emergent behaviour of these models is being used to wrangle them into doing what we want, but I find it worrying that this is considered the right approach for issues of safety. It seems obvious to me that an out-of-band approach to safety and security is necessary and that this method is never going to work in a generalised way.
I spent a few years of my career in formal verification. As a result of that, I suspect that the desiderata of “technical alignment” research is most likely undecidable, and the existing RLHF and prompt engineering approaches are probably the best we can do.
A neural network is a large, nonlinear function. In many applications, such as those that release their source code and allow fine tuning, it is adaptive. Internally the weights evolve according to solutions of differential equations. All of these things smell of undecidability when trying to prove safety/alignment properties of them:
- The theory of differential equations (which governs the evolution of the weights during training and retraining) is undecidable - Nonlinear functions have decidable cases as long as we stick to polynomials, but even then the decision algorithms are doubly exponential—-much higher complexity than anything in NP-complete, so practically undecidable for very large functions - The neural network is a very large function
In summary, they are large, nonlinear, and adaptive, these three characteristics make the problem smell intractable and likely undecidable to me.
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#14LLMs will become another building block and we will once again stand on the shoulders of giants.
With even less understanding of how that foundation was built.
It's definitely convenient to be able to throw together a Python script that imports the world and "just works", or to point my secretary at a no-code tool so she can automate her billing processes and email handling without needing to deal with the engineers. But in that handful of really performance-critical, highly demanding problems, it's important to be able to have someone or a few someones who have some comprehension of the full stack, all the from sand to scripting languages. Most of my day is just translating business logic from natural language to precise machine languages, but the best parts are the performance-critical, complex, demanding problems where you need to understand gate delays and bandwidth limits and RTOS priorities and concurrent programming and API minutiae.
That's a tall pyramid of knowledge to have to build on, and the area of our T-shaped expertise grows large, but I worry about the future that involves LLMs within APIs. It's one thing to be aware that multithreading locks or and database availability requirements have to contend with non-determinism, so you have to assume the worst and design around that, but when a tall part of the stack is a non-deterministic LLM taking in natural language and doing unpredictable things based on a neural network with billions of nodes, how can you make any guarantees about the outcome?
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#15I wonder if "prompt engineering" will be replaced by improvements to LLMs within the next couple years and this is just a bad DX step gap along the journey
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#16I wonder if "prompt engineering" will be replaced by improvements to LLMs within the next couple years and this is just a bad DX step gap along the journey
It pretty much seems like it, as this is the infancy of LLM UX.
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#17I wonder if "prompt engineering" will be replaced by improvements to LLMs within the next couple years and this is just a bad DX step gap along the journey
Communication is basically prompt engineering.
You can say whatever you like and live with the results, or try to phrase your communication in a way that elicits the kind of response you want from the other party.
We’ve engineered it away with computers where we learn a programming language or the rules of an operating system and the acceptable input is translated into a deterministic result.
But natural language is much more creative and open ended.
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#18I wonder how long it will take before people start to use LLMs to generate prompts...
I’ve done this before.
“How would you prompt an LLM to do…”
“Take the following prompt and ask clarifying questions…”
“Rephrase this prompt to be clear, concise and achievable…”
I’ve had mixed results. But sometimes it is illuminating to see it clarify things I may have omitted.
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#19I find it fascinating that emergent behaviour of these models is being used to wrangle them into doing what we want, but I find it worrying that this is considered the right approach for issues of safety. It seems obvious to me that an out-of-band approach to safety and security is necessary and that this method is never going to work in a generalised way.
I spent a few years of my career in formal verification. As a result of that, I suspect that the desiderata of “technical alignment” research is most likely undecidable, and the existing RLHF and prompt engineering approaches are probably the best we can do. A neural network is a large, nonlinear function. In many applications, such as those that release their source code and allow fine tuning, it is adaptive. Intern…
Isn’t prompt engineering a human way to explore the latent space of the model to create the mathematical interactions we want/deem useful? A human way of traversing the hyperdimensionalty. Given your background, would you say that prompt engineering is about finding the spaces in latent space the bound the model’s “knowledge” for our uses?
Re: Prompt Engineering: Steer a large pretrained language model to do what you want
#20I find it fascinating that emergent behaviour of these models is being used to wrangle them into doing what we want, but I find it worrying that this is considered the right approach for issues of safety. It seems obvious to me that an out-of-band approach to safety and security is necessary and that this method is never going to work in a generalised way.
Is it even possible to know the surface attack area for potential jailbreaks when models have unknown emergent behavior, the inner workings are blackbox and the input is anything that can be described by human language?
I don't perceive this as even being solvable and on the looming question of alignment going forward towards AGI I see the same. An unsolvable problem as the very premise to resolution is a paradox itself. Elaborated in further detail - https://dakara.substack.com/p/ai-singularity-the-hubris-trap