Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Butter – A Behavior Cache for LLMs

butter.dev

Post reply on HN
Print view View original

21–25 of 25 posts

Re: Show HN: Butter – A Behavior Cache for LLMs

#22

Post by toobulkeh »

edunteman wrote: post #16
rajit wrote: post #15

We spoke to a number of browser agent companies who said deterministic RPA with an AI fallback was their "secret" :)

Very, very common approach! Wrote more on that here: https://blog.butter.dev/the-messy-world-of-deterministic-age...

What a great overview!

I’d love your thoughts on my addition, autolearn.dev — voyager behind MCP.

The proxy format is exactly what I needed!

Thanks

Re: Show HN: Butter – A Behavior Cache for LLMs

#23

Post by toobulkeh »

edunteman wrote: post #17
barapa wrote: post #13

We often will repeat calls to try again. Or sometimes we make the same call multiple times to get multiple answers and then score or merge them. Is this used only in cases where you assume the answer from your first call is correct?

I’d love your opinion here! Right now, we assume first call is correct, and will eagerly take the first match we find while traversing the tree. One of the worst things that could currently happen is we cache a bad run, and now instead of occasional failures you’re given 100% failures. A few approaches we’ve considered - maintain a staging tree, and only promote to live if multiple sibling nodes (messages) look simil…

I’d assume RL would be baked in to the request structure. I’m surprised OAI spec doesn’t include it, but I suppose you could hijack a conversation flow to do so
Post reply on HN