Good stuff. How does this compare to Instructor? I’ve been using this extensively
Show HN: Fructose – LLM calls as strongly typed functions
31–40 of 105 posts
Re: Show HN: Fructose – LLM calls as strongly typed functions
#32> not unlike other packages such as marvin
This feels pretty much identical to Marvin? Like the entire API?
From a genuine place of curiosity: I get that your prompts are different, but like why in the name of open source would you just not contribute to these libraries instead of starting your own from scratch?
Re: Show HN: Fructose – LLM calls as strongly typed functions
#33Good stuff. How does this compare to Instructor? I’ve been using this extensively https://jxnl.github.io/instructor/
answered in different thread. tldr: not that different for now. we're likely to do some serverside optimizations, esp. given our gpu inference history.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#34fyi: LM Studio can host a server that uses the OpenAI api to whatever model you are using locally
So as long as this library can be directed to localhost or configured, it can use any LLM
Re: Show HN: Fructose – LLM calls as strongly typed functions
#35I've done a lot of work over the last year wrangling LLM outputs - both from the OpenAI API as well as local LLMs. What are the benefits of using Fructose over LMQL, Guidance or OpenAI's function calling?
Still learning about the landscape so can't give informed opinions. LMQL is a new one for me, will check it out. What we're mostly going for is composability vs abstraction. What's the smallest nugget of lift we can do for you, to make it feel natural to implement what you want? In this case it's treating the calls as functions and leaning on native python features like functions, docstrings, and types, so you can st…
Re: Show HN: Fructose – LLM calls as strongly typed functions
#36I love the concept, but I'd really prefer being able to use it against local llms (localai, ollama, etc).
as with marvin, you can just swap the base url and use any of the oss proxy libs that clone the openai api (but since they don't do function calling [except for mistral i think], its not as good afaik)
Re: Show HN: Fructose – LLM calls as strongly typed functions
#37Earlier quoted context omitted.
Still learning about the landscape so can't give informed opinions. LMQL is a new one for me, will check it out. What we're mostly going for is composability vs abstraction. What's the smallest nugget of lift we can do for you, to make it feel natural to implement what you want? In this case it's treating the calls as functions and leaning on native python features like functions, docstrings, and types, so you can st…
Mentioned packages https://github.com/eth-sri/lmql https://github.com/outlines-dev/outlines https://github.com/guidance-ai/guidance
here's an awesome post on the landscape
https://hamel.dev/blog/posts/prompt/
Re: Show HN: Fructose – LLM calls as strongly typed functions
#38So what is this actually putting into the prompt to guide generation? I dislike libraries that come with a lot of pointless abstraction.
I'm about to write something that generates typescript code from pydantic models. If this just works out of box, it would make me very happy.
I'll take a look through the repo tomorrow, sorry if my response is a little lazy, I just got off work.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#39IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…
Is the F# POC open source? Link?
Re: Show HN: Fructose – LLM calls as strongly typed functions
#40Very Cool! Would it work for Pydantic out of the box? Or that's something coming along?
currently don't have pydantic support yet, but we're not too opinionated on that. I know it's seemed to emerge as a standard, and I imagine useful in the context of running fructose in a FastAPI handler, but we led with dataclasses because they're language native and achieve much of the same thing
Pydantic can serialize both instances and classes/types to json and json schema. That seems quite helpful for this use case. How are you handling serialization to/from the llm? Role-your-own or are there additional libraries for doing this with dataclasses?