Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building reliable systems out of unreliable agents

rainforestqa.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Building reliable systems out of unreliable agents

#41

Post by maciejgryka »

CuriouslyC wrote: post #39

Prompt engineering is honestly not long for this world. It's not hard to build an agent that can iteratively optimize a prompt given an objective function, and it's not hard to make that agent general purpose. DSPy already does some prompt optimization via multi-shot learning/chain of thought, I'm quite certain we'll see an optimizer that can actually rewrite the base prompt as well.

I hear you and am planning to try DSPy because it seems attractive, but I'm also hearing people with a lot of experience being cautions about this https://x.com/HamelHusain/status/1777131374803402769 so I wouldn't make this a high-conviction bet.

Re: Building reliable systems out of unreliable agents

#42

Post by msp26 »

tedtimbrell wrote: post #22

On the topic of wrappers, as someone that's forced to use GPT-3.5 (or the like) for cost reasons, anything that starts modifying the prompt without explicitly showing me how is an instant no-go. It makes things really hard to debug. Maybe I'm the equivalent of that idiot fighting against JS frameworks back when they first came out it but it feels pretty simple to just use individual clients and have pydantic load/val…

No, you're along the right lines. Every prompting wrapper I've tried and looked through has been awful.

It's not really the authors' faults, it's just a weird new problem with lots of unknowns. It's hard to get the design and abstractions correct. I've had the benefit of a lot of time at work to build my own wrapper (solely for NLP problems) and that's still an ongoing process.

Re: Building reliable systems out of unreliable agents

#43

Post by infecto »

CuriouslyC wrote: post #38
maciejgryka wrote: post #15

Earlier quoted context omitted.

Oh this is fun! So you basically define personalities by picking well-known people that are probably represented in the training data and ask them (their LLM-imagined doppelganger) to vote?

In the research literature, this process is done not by "agent" voting but by taking a similarity score between answers, and choosing the answer that is most representative. Another approach is to use multiple agents to generate a distribution over predictions, in sort of like bayesian estimation.

For clarification on the first part. The research suggests you can utilize the same prompt over multiple runs as the input to picking the answer.

Re: Building reliable systems out of unreliable agents

#44

Post by CuriouslyC »

maciejgryka wrote: post #41
CuriouslyC wrote: post #39

Prompt engineering is honestly not long for this world. It's not hard to build an agent that can iteratively optimize a prompt given an objective function, and it's not hard to make that agent general purpose. DSPy already does some prompt optimization via multi-shot learning/chain of thought, I'm quite certain we'll see an optimizer that can actually rewrite the base prompt as well.

I hear you and am planning to try DSPy because it seems attractive, but I'm also hearing people with a lot of experience being cautions about this https://x.com/HamelHusain/status/1777131374803402769 so I wouldn't make this a high-conviction bet.

I don't have the context to fully address that tweet, but in my experience there is a repeatable process to prompt design and optimization that could be outlined and followed by a LLM with iterative capabilities using an objective function.

The real proof though is that most "prompt engineers" already use chatgpt/claude to take their outline prompt and reword it for succinctness and relevance to LLMs, have it suggest revisions and so forth. Not only is the process amenable to automation, but people are already doing hybrid processes leveraging the AI anyhow.

Re: Building reliable systems out of unreliable agents

#45

Post by cpursley »

cpursley wrote: post #36

If you’re using Elixir, I thought I’d point out how great this library is: https://github.com/thmsmlr/instructor_ex It piggybacks on Ecto schemas and works really well (if instructed correctly).

While I'm at at, this Elixir library is great as well: https://github.com/brainlid/langchain

Re: Building reliable systems out of unreliable agents

#46

Post by mistermann »

CuriouslyC wrote: post #38
maciejgryka wrote: post #15

Earlier quoted context omitted.

Oh this is fun! So you basically define personalities by picking well-known people that are probably represented in the training data and ask them (their LLM-imagined doppelganger) to vote?

In the research literature, this process is done not by "agent" voting but by taking a similarity score between answers, and choosing the answer that is most representative. Another approach is to use multiple agents to generate a distribution over predictions, in sort of like bayesian estimation.

Any chance you could expand on both of these, even enough to assist in digging deeper into them? TIA.

Re: Building reliable systems out of unreliable agents

#47

Post by CuriouslyC »

mistermann wrote: post #46
CuriouslyC wrote: post #38

Earlier quoted context omitted.

In the research literature, this process is done not by "agent" voting but by taking a similarity score between answers, and choosing the answer that is most representative. Another approach is to use multiple agents to generate a distribution over predictions, in sort of like bayesian estimation.

Any chance you could expand on both of these, even enough to assist in digging deeper into them? TIA.

The TLDR is you can prompt the LLM to take different perspectives than its default, then combine those. If the LLM is estimating a number, the different perspectives give you a distribution over the truth, which shows you the range of biases and the most likely true answer (given wisdom of the crowd). If the LLM is generating non-quantifiable output, you can find the "average" of the answers (using embeddings or other methods) and select that one.

Re: Building reliable systems out of unreliable agents

#48

Post by mritchie712 »

CuriouslyC wrote: post #38
maciejgryka wrote: post #15

Earlier quoted context omitted.

Oh this is fun! So you basically define personalities by picking well-known people that are probably represented in the training data and ask them (their LLM-imagined doppelganger) to vote?

In the research literature, this process is done not by "agent" voting but by taking a similarity score between answers, and choosing the answer that is most representative. Another approach is to use multiple agents to generate a distribution over predictions, in sort of like bayesian estimation.

for my use case (generating an interesting H1), using a similarity score would defeat the purpose.

In this approach, I'm looking for the diamond in the rough. It's often dissimilar from the others. With this approach, the diamond can still get a high number of votes.

Re: Building reliable systems out of unreliable agents

#49

Post by mritchie712 »

all2 wrote: post #17
mritchie712 wrote: post #12

This is a great write up! I nodded my head thru the whole post. Very much aligns with our experience over the past year. I wrote a simple example (overkiLLM) on getting reliable output from many unreliable outputs here[0]. This doesn't employ agents, just an approach I was interested in trying. I choose writing an H1 as the task, but a similar approach would work for writing any short blob of text. The script generat…

I'd be curious to see some examples and maybe intermediate results?

here's some examples[0]:

this one scored high:

Pinned Down - Powerful Analytics Without the Need for Engineering or SQL

this one scored low:

Analytics Made Accessible for Everyone.

Each time I've compared the top scoring results to those at the bottom, I've always preferred the top scoring variations.

0 - https://docs.google.com/spreadsheets/d/1hdu2BlhLcLZ9sruVW8a_...

Re: Building reliable systems out of unreliable agents

#50

Post by CuriouslyC »

mritchie712 wrote: post #48
CuriouslyC wrote: post #38

Earlier quoted context omitted.

In the research literature, this process is done not by "agent" voting but by taking a similarity score between answers, and choosing the answer that is most representative. Another approach is to use multiple agents to generate a distribution over predictions, in sort of like bayesian estimation.

for my use case (generating an interesting H1), using a similarity score would defeat the purpose. In this approach, I'm looking for the diamond in the rough. It's often dissimilar from the others. With this approach, the diamond can still get a high number of votes.

That approach definitely has promise. I would have agents rate answers and take the highest rated rather than vote for them though, since you're losing information about ranking and preference gradients with n choose 1. Also, you can do that whole process in one prompt, in case you're re-prompting currently, it's cheaper to batch it up.
Post reply on HN