11–20 of 105 posts
Langchain & Llamaindex plus Fructose really need to skip the structure adherence work & move to chunking/KG generation since that's the next pain point to tackle.
I've done a lot of work over the last year wrangling LLM outputs - both from the OpenAI API as well as local LLMs. What are the benefits of using Fructose over LMQL, Guidance or OpenAI's function calling?
What we're mostly going for is composability vs abstraction. What's the smallest nugget of lift we can do for you, to make it feel natural to implement what you want? In this case it's treating the calls as functions and leaning on native python features like functions, docstrings, and types, so you can still use the python language like closures to do the weird things you need.
This is all handwavy, put on my wizard language design hat, so take it with a grain of salt. We're just trying things out.
TGI just integrated Guidance in 1.4.3, that by itself can support both grammar/JSON/Pydantic & tool invocation/function calling. Langchain & Llamaindex plus Fructose really need to skip the structure adherence work & move to chunking/KG generation since that's the next pain point to tackle.
Very Cool! Would it work for Pydantic out of the box? Or that's something coming along?
I love the concept, but I'd really prefer being able to use it against local llms (localai, ollama, etc).
Seems like a great feature (and honestly allows us to do smarter things for strictly structured generation). I'm curious, what's your main motivation for local llms vs hosted APIs?