Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Fructose – LLM calls as strongly typed functions

github.com

Post reply on HN
Print view View original

41–50 of 105 posts

Re: Show HN: Fructose – LLM calls as strongly typed functions

#41

Post by nate_nowack »

itfollowsthen wrote: post #32

> not unlike other packages such as marvin This feels pretty much identical to Marvin? Like the entire API? From a genuine place of curiosity: I get that your prompts are different, but like why in the name of open source would you just not contribute to these libraries instead of starting your own from scratch?

yeah this seems to be pretty much the same interface as `fn` from marvin, except w/o pydantic (see https://github.com/PrefectHQ/marvin?tab=readme-ov-file#-buil...)

Re: Show HN: Fructose – LLM calls as strongly typed functions

#42

Post by nextos »

obeavs wrote: post #39
nextos wrote: post #21

IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…

Is the F# POC open source? Link?

Not yet, it's a bit rough. The LLM I am using requires a bit of extra fine-tuning to be really smooth, I need to rent a bigger GPU. Besides, I am working on some novel integration between transformers and SAT/SMT that will take me some time to finish.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#43

Post by yaj54 »

nextos wrote: post #42
obeavs wrote: post #39

Earlier quoted context omitted.

Is the F# POC open source? Link?

Not yet, it's a bit rough. The LLM I am using requires a bit of extra fine-tuning to be really smooth, I need to rent a bigger GPU. Besides, I am working on some novel integration between transformers and SAT/SMT that will take me some time to finish.

Is the theory tied to a specific llm? I'm interpreting it as, e.g., the llm writes the code, the solver verifies it, repeat until correct. In this situation the two are decoupled and the llm would be a drop in and thusly could be any local or remote llm. Is there something about your approach that doesn't allow this?

(also, +1 for OS link request)

Re: Show HN: Fructose – LLM calls as strongly typed functions

#46

Post by reliableturing »

memothon wrote: post #30
edunteman wrote: post #25

Earlier quoted context omitted.

Yes. TGI is Huggingface's version of LLVM (some nuance, of course). LLVM also launched grammar support recently too, so we'll be looking into it.

All of these acronyms are so confusing. I'm assuming LLVM isn't the compiler tool, but searching "LLVM ai" doesn't give me any good results.

They probably meant vLLM https://docs.vllm.ai/en/latest/

Re: Show HN: Fructose – LLM calls as strongly typed functions

#47

Post by ravenstine »

nextos wrote: post #21

IMHO, in the future programming may look similar to this. Write a type declaration for a function with an expressive type system, e.g. refinement types. Then use LLMs + SAT/SMT to generate provably correct code. This strikes a happy medium, where machines are assisting programmers, making them much more productive. Yet the resulting code is understandable as a human has decomposed everything into functions, and also…

Yeah, functional programming and pure functions seem perfect for generative programming. Granted, I think they're perfect in general, but as long as human programmers are still stuck in the world of object-orientation (the modern sense), then they're going to be wasting the time in the LLM feedback loop. The LLM should be be able to write a unit of code however it wants in a way that is as self-contained as possible. Since an LLM can, in theory, quickly "understand" code that most software engineers would object to, then we should get out of the way of LLMs rather than expect them to be like we are.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#48

Post by yaj54 »

edunteman wrote: post #33
mattew wrote: post #31

Good stuff. How does this compare to Instructor? I’ve been using this extensively https://jxnl.github.io/instructor/

answered in different thread. tldr: not that different for now. we're likely to do some serverside optimizations, esp. given our gpu inference history.

I like your UX a lot more. Modeling the llm calls as actual python functions allows them to mesh well with existing code organization dev tooling. And using a decorator to "implement" a function just feels like a special kind of magic. I'd need more ability to use my own "prompt templates" to use this as a lib but I'm definitely going to try using this general pattern.

Re: Show HN: Fructose – LLM calls as strongly typed functions

#49

Post by nerpderp82 »

edunteman wrote: post #37
nerpderp82 wrote: post #35

Earlier quoted context omitted.

Mentioned packages https://github.com/eth-sri/lmql https://github.com/outlines-dev/outlines https://github.com/guidance-ai/guidance

here's an awesome post on the landscape https://hamel.dev/blog/posts/prompt/

I remember reading that, good stuff.

I'd like to see an injectable mitm like proxy that can rewrite payloads. Many of these frameworks are useful, but when they go off the rails, they hard to modify and introspect.

It would be nice if LLMs had a way to speak an annotated format, like XML that was able to encode higher level information in a coherent manner over "well formed" addhoc text.

LLM libraries are in a crazy state right now. It is like JS frameworks 2015, a new one that demos well every other day.

Post reply on HN