Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Magentic – Use LLMs as simple Python functions

github.com

Post reply on HN
Print view View original

31–40 of 70 posts

Re: Show HN: Magentic – Use LLMs as simple Python functions

#31

Post by BoorishBears »

I've personally found frameworks like this to get in the way of quality COT: It's rare for a prompt that takes great advantage of the LLM's reasoning to fit in the format these generators encourage

A friend mentioned how terrible most cold email generators are at actually generating natural feeling emails. It just took asking him questions about how actual people in marketing come up with emails to come up with a chain of thought that produces intentionally uncanny emails for a wide range of inputs: https://rentry.co/54hbz

It's not like you can't technically fit what I described into bunch of comments (or an obnoxiously long multiline comment), but it'd be bulky and not conducive to general happiness of anyone involved.

I much prefer repurposing Nunjucks templates to keep all of that a separate document that's easy to manage with version control

Re: Show HN: Magentic – Use LLMs as simple Python functions

#32

Post by fredoliveira »

jackmpcollins wrote: post #5
bl00p wrote: post #3

Are you familiar with https://github.com/PrefectHQ/marvin ? This looks very similar

Yes, similar ideas. Marvin [asks the LLM to mimic the python function]( https://github.com/PrefectHQ/marvin/blob/f37ad5b15e2e77dd998... ), whereas in magentic the function signature just represents the inputs/outputs to the prompt-template/LLM, so the LLM “doesn’t know” that it is pretending to be a python function - you specify all the prompts.

(Completely off-topic, but oh how I wish HN supported markdown)

Re: Show HN: Magentic – Use LLMs as simple Python functions

#33

Post by te_chris »

This is great. I hacked a smaller version of this together when I built an LLM app with Elixir. Honestly, the async by default of Ex is so much better suited to this stuff, especially as it’s just api calls.

Tempted to have a go at porting these ideas. Should be v doable with the macro system.

Re: Show HN: Magentic – Use LLMs as simple Python functions

#35

Post by js98 »

Very cool! At first the title reminded me of a project me and my colleague are working on called OpenAI-Functools [1], but your concept is quite the opposite, combining LLMs in your code rather seamlessly instead of the other way around. Quite cool, and interesting examples :)

I’ll definitely try to apply it in one of my pet projects. Good stuff

[1] https://github.com/Jakob-98/openai-functools

Re: Show HN: Magentic – Use LLMs as simple Python functions

#36

Post by jackmpcollins »

Difwif wrote: post #29

Looks great! I don't normally like these LLM libraries but this one sparks joy. I'll try it out on my next experiment. Could you highlight how you're parsing to structured objects and how it can fail? Ever since I discovered guidance's method of pattern guides I've been wanting this more and more (only works for local hugging face models though). Wish OpenAI offered a similar API.

Thanks! Currently magentic just uses OpenAI function-calling; it provides it a function schema that matches the structure of the output object. So it fails in the same ways as function-calling - struggles to match complex schemas, occasionally returns empty arrays, ...

Re: Show HN: Magentic – Use LLMs as simple Python functions

#38

Post by jackmpcollins »

BoorishBears wrote: post #31

I've personally found frameworks like this to get in the way of quality COT: It's rare for a prompt that takes great advantage of the LLM's reasoning to fit in the format these generators encourage A friend mentioned how terrible most cold email generators are at actually generating natural feeling emails. It just took asking him questions about how actual people in marketing come up with emails to come up with a cha…

With magentic you could do chain-of-thought in two or more steps: one function that generates a string output containing the chain-of-thought reasoning and answer, and a second that takes that output and converts it to the final answer object. I agree though that this is not encouraged or made obvious by the framework.

The approach I'm encouraging with this is to write many functions to achieve your goal. So in the case of your email writing example you might have some of the following prompt-functions - write key bullet points for email about xyz -> list[str] - write email based on bullet points -> str - generate feedback for email to meet criteria abc -> str - update email based on feedback -> str - does email meet all criteria abc -> bool And between these you could have regular python code check things like blacklist/whitelist of keywords, length of paragraphs, and even add hardcoded strings to the feedback based on these checks.

Re: Show HN: Magentic – Use LLMs as simple Python functions

#39

Post by brandall10 »

jstarfish wrote: post #11

This looks really useful. Langchain is not my idea of a fun time. Love the examples too. Low-effort humor is the best: > create_superhero("Garden Man") > # Superhero(name='Garden Man', age=30, power='Control over plants', enemies=['Pollution Man', 'Concrete Woman'])

FWIW, at my last company we had a section in the developer guide encouraging using humor in tests - not only did it make them more fun to write, but it engaged the readership better.
Post reply on HN