Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building reliable systems out of unreliable agents

rainforestqa.com

Post reply on HN
Print view View original

51–56 of 56 posts

Re: Building reliable systems out of unreliable agents

#51

Post by mistermann »

CuriouslyC wrote: post #47
mistermann wrote: post #46

Earlier quoted context omitted.

Any chance you could expand on both of these, even enough to assist in digging deeper into them? TIA.

The TLDR is you can prompt the LLM to take different perspectives than its default, then combine those. If the LLM is estimating a number, the different perspectives give you a distribution over the truth, which shows you the range of biases and the most likely true answer (given wisdom of the crowd). If the LLM is generating non-quantifiable output, you can find the "average" of the answers (using embeddings or othe…

Ah ok, so both are implemented via a call(s) to the LLM, as opposed to a standard algorithmic approach?

Re: Building reliable systems out of unreliable agents

#52

Post by CuriouslyC »

mistermann wrote: post #51
CuriouslyC wrote: post #47

Earlier quoted context omitted.

The TLDR is you can prompt the LLM to take different perspectives than its default, then combine those. If the LLM is estimating a number, the different perspectives give you a distribution over the truth, which shows you the range of biases and the most likely true answer (given wisdom of the crowd). If the LLM is generating non-quantifiable output, you can find the "average" of the answers (using embeddings or othe…

Ah ok, so both are implemented via a call(s) to the LLM, as opposed to a standard algorithmic approach?

Once you have bayesian prior distributions (which it makes total sense for llms to estimate) you can do tons of nifty statistical techniques. It's only the bottom layer of the analysis stack that's LLM generated.

Re: Building reliable systems out of unreliable agents

#53

Post by all2 »

mritchie712 wrote: post #49
all2 wrote: post #17

Earlier quoted context omitted.

I'd be curious to see some examples and maybe intermediate results?

here's some examples[0]: this one scored high: Pinned Down - Powerful Analytics Without the Need for Engineering or SQL this one scored low: Analytics Made Accessible for Everyone. Each time I've compared the top scoring results to those at the bottom, I've always preferred the top scoring variations. 0 - https://docs.google.com/spreadsheets/d/1hdu2BlhLcLZ9sruVW8a_...

I love the spreadsheet. That's exactly what I was looking for. Thank you!

Re: Building reliable systems out of unreliable agents

#54

Post by namaria »

CuriouslyC wrote: post #39

Prompt engineering is honestly not long for this world. It's not hard to build an agent that can iteratively optimize a prompt given an objective function, and it's not hard to make that agent general purpose. DSPy already does some prompt optimization via multi-shot learning/chain of thought, I'm quite certain we'll see an optimizer that can actually rewrite the base prompt as well.

It strikes me as bad reasoning to look at a system that is designed to be very complex and stochastic as a way to get some creativity out of it ("generative AI" so to speak) and try to bolt down added apparatus to get deterministic behavior out of it.

We have deterministic programming systems. They're called compilers.

Re: Building reliable systems out of unreliable agents

#55

Post by CuriouslyC »

namaria wrote: post #54
CuriouslyC wrote: post #39

Prompt engineering is honestly not long for this world. It's not hard to build an agent that can iteratively optimize a prompt given an objective function, and it's not hard to make that agent general purpose. DSPy already does some prompt optimization via multi-shot learning/chain of thought, I'm quite certain we'll see an optimizer that can actually rewrite the base prompt as well.

It strikes me as bad reasoning to look at a system that is designed to be very complex and stochastic as a way to get some creativity out of it ("generative AI" so to speak) and try to bolt down added apparatus to get deterministic behavior out of it. We have deterministic programming systems. They're called compilers.

I think you're missing the point. If an application had simple logic, the program would have been written in a simple language in the first place. This is about taking fuzzy processes that would be incredibly difficult to program, and making them consistent and precise.

Re: Building reliable systems out of unreliable agents

#56

Post by caseyy »

maciejgryka wrote: post #33
caseyy wrote: post #30

Earlier quoted context omitted.

Oh yeah. I read few-shot like it means trying a few times to get an appropriate output. That’s how the author uses the word “shot” in the beginning of the article. Priming is a specific term that means giving examples in the context window. But yeah, the author seems to describe this. Still, you can go a long way with priming. I wouldn’t even think of fine-tuning before trying priming for a good while. It might still…

Ha good point, I did say "let's have another shot" when I just meant another try at generating! FWIW "few shot prompting" is how most people refer to this technique, I think (e.g. see https://www.promptingguide.ai/techniques/fewshot ), I haven't heard "priming" before, though it does convey the right thing. And the reason we don't really do it is context length. Our contexts are long and complex and there are so many…

Interesting, I didn’t hear about few shot prompting. There’s a ton of stuff written on specifically “priming” as well. People use different terms I suppose.

It makes sense about the context window length, it can be limiting. For small inputs and outputs, it’s great. And it’s remarkably effective with diminishing returns. This is why I have 5 shots as a concrete example. You probably need more than 1 or 2, but for a lot of applications, probably less than 20. For most basic tasks like extracting words from a document or producing various summaries, for example.

It depends on the complexity of the task and how much you’re worried about over-fitting to your data set. But if you’re not so worried, the task is not complex, and the inputs and outputs are small, then it works very well with only shots.

And it’s basically free in the context of fine-tuning.

It might be worth expanding on it a bit in this or a separate article. It’s a good way to increase reliability to a workable extent in unreliable LLMs. Although a lot has been written on few short prompting/priming already.

Post reply on HN