Show HN: Fructose – LLM calls as strongly typed functions • Hacker News Board

Re: Show HN: Fructose – LLM calls as strongly typed functions

#81

Post by nextos » Thu, Mar 07, 2024, 1:59 AM UTC

amiantos wrote: post #72

nextos wrote: post #21

IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…

> Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This is how I use copilot currently, so I might not be following on what part of this is 'future' facing or relevant to this Fructose project? Not being contrarian, I thought this was an interesting point but as I thought about it more I realized, "wait, they're d…

Do you use Copilot on a language equipped with refinement or dependent types, and use said types to constrain generation of entire functions, in a single step, that are also formally verified?

Re: Show HN: Fructose – LLM calls as strongly typed functions

#82

Post by pyinstallwoes » Thu, Mar 07, 2024, 2:16 AM UTC

nextos wrote: post #21

IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…

Oddly similar to summoning demons.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#83

Post by edunteman » Thu, Mar 07, 2024, 2:24 AM UTC

judicious wrote: post #68

Definitely very excited to see this be a thing. Genuinely liked the approach to make function calls strongly typed and rely on functional programming principles. During my senior year, I worked on a research project very similar to this and I’m glad to see this out there for everyone. I’d love to connect with the team if possible!

absolutely! My DMs are open at @erikdunteman on twitter, or erik at banana dot dev

Re: Show HN: Fructose – LLM calls as strongly typed functions

#84

Post by edunteman » Thu, Mar 07, 2024, 2:33 AM UTC

ilaksh wrote: post #77

vineetch wrote: post #66

Earlier quoted context omitted.

So you're saying they should ensure compatibility with all LLMs on Day 0 so you can avoid a personal "grating" feeling. It's called an MVP.

They should just say it works with OpenAI or ChatGPT. It's called being honest.

feel free to PR the readme if you feel it was misleading

Re: Show HN: Fructose – LLM calls as strongly typed functions

#85

Post by campers » Thu, Mar 07, 2024, 2:35 AM UTC

Nice, I've just started building something similar in TypeScript. I wasn't a big fan of the Langchain model. I wanted to develop with normal functions in an imperative manner so the code is very easy to read. I'm also using decorators to add the required functionality to workflow steps so I can support retries, and build something like LangSmith on top too.

So far I've been able to make a little workflow that can complete real simple infrastructure requests in JIRA. Pick the right repository, make the changes, compile, and push up the merge request.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#86

Post by edunteman » Thu, Mar 07, 2024, 2:35 AM UTC

jxnlco wrote: post #73

edunteman wrote: post #23

Earlier quoted context omitted.

Currently, quite comparable and obviously Instructor is more mature and feature rich. They're going the "patch the openAI client" approach which makes code written still use openAI SDK patterns which is pretty smart. Jason seems like he knows what he's doing. We're trying to make it more of a language feature with the decorated functions. Plus exploring the hosted formatting model direction. How do you feel this comp…

i do not know what i am doing. I just want `create(response_model=T) -> T` lol

the king has arrived! Instructor is a clever API on this, clean

Re: Show HN: Fructose – LLM calls as strongly typed functions

#87

Post by Jonovono » Thu, Mar 07, 2024, 2:42 AM UTC

campers wrote: post #85

Nice, I've just started building something similar in TypeScript. I wasn't a big fan of the Langchain model. I wanted to develop with normal functions in an imperative manner so the code is very easy to read. I'm also using decorators to add the required functionality to workflow steps so I can support retries, and build something like LangSmith on top too. So far I've been able to make a little workflow that can com…

open source?

Re: Show HN: Fructose – LLM calls as strongly typed functions

#88

Post by lefttoreader » Thu, Mar 07, 2024, 2:47 AM UTC

barapa wrote: post #78

lefttoreader wrote: post #71

Earlier quoted context omitted.

Wait. So why not just contribute to an existing open source project if you’re going to implement an identical API? If you run your own models as a part of it, surely you could hook up your models as a backend to whatever abstractions you’re copying here.

Wait, someone made a similar comment as this elsewhere in the thread. So why don't you just upvote that? If you have your own thoughts, surely you could just think them to yourself while upvoting.

yikes.

I was responding to them sidestepping the first commenter’s question.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#89

Post by edunteman » Thu, Mar 07, 2024, 2:49 AM UTC

icyfox wrote: post #80

Big proponent of guaranteed outputs for LLMs. I wrote a library awhile back (gpt-json) that did something similar by querying the OpenAI API. At the end of the day though while their responses are _highly likely_ to be valid JSON they're not guaranteed. There's only so much that can be done with remote calls to their model's black box. The future here really lies in compiling down context free grammars. They let you…

it seems this is in the context of "extraction" where all of the data is already present in the input text, and all that's needed is the reformatting. This is something we've been wrestling with (even today): is the role of fructose and/or our eventual formatting model to provide both intelligence + formatting (such as generating novel data, on the fly, constrained to structure), or just formatting (anything-to-json). Not sure what the answer is and not expecting one, we're just realizing the clear split between needs of users running extraction vs more generative/creative tasks.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#90

Post by pedrovhb » Thu, Mar 07, 2024, 3:04 AM UTC

babyshake wrote: post #59

Does anyone else get bothered by how this seemingly results in code that won't compile? Instead of this: @ai() def describe(animals: list[str]) -> str: """ Given a list of animals, use one word that'd describe them all. """ it would seem a lot more intuitive to do this: def describe(animals: list[str]) -> str: return ai("""Given a list of animals, use one word that'd describe them all.""", animals)

Technically a function body needs at least one statement. A docstring is just an expression statement (a string), so a function definition with just a docstring is synctatically valid Python. I've seen people say multiline string literals are Python's version of multiline comments, but that's really just convention; it's a noop expression statement. Same as doing

    def foo():
        4

Which is also an expression statement as a function body, and also does nothing. Contrast to actually using a comment as a function body; comments aren't statements (nor expressions, so they can't be used as an expression statement):

    def foo():
        # this doesn't work

> IndentationError: expected an indented block after function definition on line 1

Of course, this doesn't really matter at all, and I get that it feels strange. I've just been thinking about grammars and syntax lately, and it's been interesting to now have the vocabulary and mental model to understand these unintuitive things :)