Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing Agents in Haystack: Make LLMs resolve complex tasks

haystack.deepset.ai

Post reply on HN
Print view View original

31–40 of 106 posts

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#31

Post by alangou »

arthurcolle wrote: post #20

Doesn't this remind anyone else of Bitcoin autonomous agents? https://en.bitcoin.it/wiki/Agent#:~:text=An%20agent%20is%20a... . It would be interesting if you could somehow combine a wallet plugin to give the GPT something to work to increase. That + that AutoGPT repo from a day ago + maybe Mitogen for self-replication might be a cool combo to really kick off a fun "dumb" AI on the loose

This is cool. Some of the weaknesses they point out in software agents can be obviated with the latest AI advancements—give a chatbot the tools of the courts, and maybe the autonomous agent won’t be at a disadvantage in scams anymore.

Interesting to see if an AI agent given the objective to generate profit would resort to running scams. Encoding moral boundaries becomes important. What if you asked it to raise money for a good cause and it uses “whatever means necessary?”

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#32

Post by newhouseb »

lsy wrote: post #8

Is there some way of holding the LLM response to a given prompt constant? It sounds like a lot of this relies on the LLM getting the right answer in sequence, so I'm guessing they do something like keep the temperature at 0? Otherwise you are going to wind up with possibly different behavior run-to-run. And even if they do have something like the above, don't we end up with potentially breaking changes once models ar…

There are a couple different approaches:

- Rerun the prompt until you get a format that is consistent

- Steer the output token selection towards a predefined prompt

For the latter, I've built a proof of concept that takes in a JSON schema with a huggingface transformer and constrains token selection by modifying the output probabilities such that only schematically valid tokens can be emitted, see "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish. Unfortunately, given OpenAI's current API this is only possible on locally run models. That is... at any level of cost effectiveness. It's technically possible but (worst case) quadratically expensive against their current APIs.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#33

Post by teruakohatu »

Spivak wrote: post #29
teruakohatu wrote: post #27

Earlier quoted context omitted.

It is different with LLMs. Most people can give a level of uncertainty along with an answer, and often do. LLMs can't, and worse, are trained to put an emphasis on the prompts. Humans are often trained to be skeptical of prompts. If I said, "the moon is made of cheese. What type of cheese do you think it is?" most humans would automatically object, but with LLMs you can usually craft a prompt that would get it to ans…

I mean it kinda can. Here's the full prompt. I have no idea about aspartame, I just picked something that it's definitely not sure about. Answer with a JSON object of the form {"confidence": $ >, "en": $ >}. User: What is 2 + 2? Bot: {"confidence": "very", "en": "2 + 2 is 4"} User: Is aspartame healthy? Bot: {"confidence": "somewhat", "en": "Aspartame has not yet been found to have any adverse effects on humans."} Us…

Interesting.

> The response: {"confidence": "very low", "en": "I'm not sure, but I don't think the moon is made of cheese."}

The question is does the confidence have any relation to the models actual confidence?

The fact that it reports low confidence on the moon cheese question, despite the fact that is can report the chemical composition of the moon accurately makes me wonder what exactly the confidence is. Seems more like sentiment analysis on its own answer.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#34

Post by est31 »

yosito wrote: post #10

I haven't yet figured out how to get an LLM to accurately determine whether it actually knows something or is making it up. I wonder how they handle that. They may get to that at some point in the article, but the page eventually breaks for me on mobile and I can't read past the first code block.

FTR, people are trying to build systems to compare LLMs with each other based on how well they are at saying "I don't know" (of course knowing is still rewarded higher): https://github.com/manyoso/haltt4llm

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#35

Post by wickoff »

teruakohatu wrote: post #27
wickoff wrote: post #19

Earlier quoted context omitted.

I have the same exact problem with people. Being inaccurate is a feature, not a bug.

It is different with LLMs. Most people can give a level of uncertainty along with an answer, and often do. LLMs can't, and worse, are trained to put an emphasis on the prompts. Humans are often trained to be skeptical of prompts. If I said, "the moon is made of cheese. What type of cheese do you think it is?" most humans would automatically object, but with LLMs you can usually craft a prompt that would get it to ans…

I know it's an extreme example, but flat earthers to exist. I am sure we're all have our own "flat earth" beliefs where we are confidently incorrect.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#36

Post by ryanwaggoner »

Ozzie_osman wrote: post #23

If you (like me) were wondering how these works, the LLM is given a prompt like: Answer the following questions as best you can. You have access to the following tools: Search: Use this to search the internet. Calculator: Use this to do math. Use the following format: Question: the input question you must answer Thought: you should always think about what to do Action: the action to take, should be one of [{tool_name…

I did a manual version of this where I played a dispatch controller in a robot, relaying inputs and outputs from GPT4, which I told was the reasoning brain in this robot. It was very remarkable to watch its train of thought in considering sensor inputs and then giving me actions to take in response.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#37

Post by sr-latch »

Ozzie_osman wrote: post #23

If you (like me) were wondering how these works, the LLM is given a prompt like: Answer the following questions as best you can. You have access to the following tools: Search: Use this to search the internet. Calculator: Use this to do math. Use the following format: Question: the input question you must answer Thought: you should always think about what to do Action: the action to take, should be one of [{tool_name…

This looks similar to the WebGPT paper, is that referenced in any of langchain or haystack's publications?

Introducing the mechanism of internal thought is very interesting, I wonder if there's a way to make it implicit in the model's architecture.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#38

Post by sr-latch »

est31 wrote: post #34
yosito wrote: post #10

I haven't yet figured out how to get an LLM to accurately determine whether it actually knows something or is making it up. I wonder how they handle that. They may get to that at some point in the article, but the page eventually breaks for me on mobile and I can't read past the first code block.

FTR, people are trying to build systems to compare LLMs with each other based on how well they are at saying "I don't know" (of course knowing is still rewarded higher): https://github.com/manyoso/haltt4llm

Would be cool to try to incorporate the previous token's confidence embedding into this process, but that would make training with a triangular attention mask not possible.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#39

Post by gremlinsinc »

Ozzie_osman wrote: post #23

If you (like me) were wondering how these works, the LLM is given a prompt like: Answer the following questions as best you can. You have access to the following tools: Search: Use this to search the internet. Calculator: Use this to do math. Use the following format: Question: the input question you must answer Thought: you should always think about what to do Action: the action to take, should be one of [{tool_name…

Does observation cover reflexion? Self-observation or is that something else?

Or maybe before Final Answer, you could do: Double Check: I think I have the final answer, but does it look right? IF yes: go to final answer if no: Go back up the loop.

Re: Introducing Agents in Haystack: Make LLMs resolve complex tasks

#40

Post by azinman2 »

wickoff wrote: post #35
teruakohatu wrote: post #27

Earlier quoted context omitted.

It is different with LLMs. Most people can give a level of uncertainty along with an answer, and often do. LLMs can't, and worse, are trained to put an emphasis on the prompts. Humans are often trained to be skeptical of prompts. If I said, "the moon is made of cheese. What type of cheese do you think it is?" most humans would automatically object, but with LLMs you can usually craft a prompt that would get it to ans…

I know it's an extreme example, but flat earthers to exist. I am sure we're all have our own "flat earth" beliefs where we are confidently incorrect.

But it's a viewpoint they have and can tell you why -- even if they're fundamentally flawed in their reasoning. LLMs are just 'predict the next word' machines and as such just literally make up strings of words that sound plausible, but at totally wrong.

These are not the same thing.

Post reply on HN