Can this be a F# Type Provider?
Good Lord that sounds terrifying - I do in general prefer my type definitions to be at least approximately deterministic.
Show HN: Fructose – LLM calls as strongly typed functions
61–70 of 105 posts
Re: Show HN: Fructose – LLM calls as strongly typed functions
#62IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…
Oh yeah, you just reminded me of this cool talk I saw at Strange Loop a while back. Not about the AI parts but re: program synthesis in Haskell:
"Type-Driven Program Synthesis" by Nadia Polikarpova https://www.youtube.com/watch?v=HnOix9TFy1A
Links to more projects and papers by Prof. Polikarpova: https://cseweb.ucsd.edu/~npolikarpova/
I think this is one of the main projects she discusses in the talk: https://github.com/nadia-polikarpova/synquid
EDIT: meant to mention this too, which I think has been around a bit longer, not that I've ever used it in production: https://ucsd-progsys.github.io/liquidhaskell/
Re: Show HN: Fructose – LLM calls as strongly typed functions
#63IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…
Oh yeah, you just reminded me of this cool talk I saw at Strange Loop a while back. Not about the AI parts but re: program synthesis in Haskell: "Type-Driven Program Synthesis" by Nadia Polikarpova https://www.youtube.com/watch?v=HnOix9TFy1A Links to more projects and papers by Prof. Polikarpova: https://cseweb.ucsd.edu/~npolikarpova/ I think this is one of the main projects she discusses in the talk: https://github.…
Lots of related work also by A. Solar-Lezama https://people.csail.mit.edu/asolar, sometimes in collaboration with N. Polikarpova.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#64Earlier quoted context omitted.
Oh yeah, you just reminded me of this cool talk I saw at Strange Loop a while back. Not about the AI parts but re: program synthesis in Haskell: "Type-Driven Program Synthesis" by Nadia Polikarpova https://www.youtube.com/watch?v=HnOix9TFy1A Links to more projects and papers by Prof. Polikarpova: https://cseweb.ucsd.edu/~npolikarpova/ I think this is one of the main projects she discusses in the talk: https://github.…
Lots of related work also by A. Solar-Lezama https://people.csail.mit.edu/asolar , sometimes in collaboration with N. Polikarpova.
TIL, nice--thanks!
Re: Show HN: Fructose – LLM calls as strongly typed functions
#65I find it grating that all of these types of things say "LLMs" when in fact they literally only work with OpenAI. There are hundreds of variations of LLM models. When it works with only gpt-4-turbo or gpt-3.5-turbo, it's inaccurate to say it's a tool for LLMs in general.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#66I find it grating that all of these types of things say "LLMs" when in fact they literally only work with OpenAI. There are hundreds of variations of LLM models. When it works with only gpt-4-turbo or gpt-3.5-turbo, it's inaccurate to say it's a tool for LLMs in general.
So you're saying they should ensure compatibility with all LLMs on Day 0 so you can avoid a personal "grating" feeling. It's called an MVP.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#67How do you guarantee output structure? Does it ever fail to conform?
It's not 100% yet. Route to that: 1. Clientside, retry strategy on failed parse. Not yet implemented, we throw an exception on parse fail right now, but soon to be implemented. Not ideal because of token burn and latency, but the best quick solution. 2. For the custom model we're building, we use strict grammar definitions to bias outputs toward the needed structure (or if there is only one structurally correct token…
Basically my experience with homegrown. If you give it a typescript template for outputs, and a ton of prose to tell it how to respond, you usually get the right responses.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#68Definitely very excited to see this be a thing. Genuinely liked the approach to make function calls strongly typed and rely on functional programming principles.
During my senior year, I worked on a research project very similar to this and I’m glad to see this out there for everyone. I’d love to connect with the team if possible!
Re: Show HN: Fructose – LLM calls as strongly typed functions
#69Earlier quoted context omitted.
Is the F# POC open source? Link?
Not yet, it's a bit rough. The LLM I am using requires a bit of extra fine-tuning to be really smooth, I need to rent a bigger GPU. Besides, I am working on some novel integration between transformers and SAT/SMT that will take me some time to finish.
Our product (phosphor) is built end-to-end in F# so this stuff is close to my heart. You might find Moonbit's approach to functional AI interesting as well https://www.moonbitlang.com/blog/moonbit-ai.
Re: Show HN: Fructose – LLM calls as strongly typed functions
#70Earlier quoted context omitted.
Seems like a great feature (and honestly allows us to do smarter things for strictly structured generation). I'm curious, what's your main motivation for local llms vs hosted APIs?
Not wanting to get dinged for $20/mo? Ability to use offline for local home automation (eg: "given a verbal input request, determine the home devices in scope and their on/off state" => "given the current devices state and the verbal request, generate a list of home assistant actions to perform"), using a custom model for the above, etc.
> Not wanting to get dinged for $20/mo?
OpenAI (& peer) API pricing is honestly quite cheap - I spend way less than 20$ a month for my needs. On the other hand, ChatGPT offers a different experience, so I pay for that too.
https://help.openai.com/en/articles/7127956-how-much-does-gp...