Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Magentic – Use LLMs as simple Python functions

github.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: Show HN: Magentic – Use LLMs as simple Python functions

#11

Post by jstarfish »

This looks really useful. Langchain is not my idea of a fun time.

Love the examples too. Low-effort humor is the best:

> create_superhero("Garden Man")

> # Superhero(name='Garden Man', age=30, power='Control over plants', enemies=['Pollution Man', 'Concrete Woman'])

Re: Show HN: Magentic – Use LLMs as simple Python functions

#15

Post by dragonwriter »

jedberg wrote: post #12

Curious as to why you chose to do it as a decorator instead of just a function call?

Looking at the code, it looks like it is a way to support typing; just making it a function with the string template would let you return a dynamically-defined function but I think would make it harder to get static typing.

Re: Show HN: Magentic – Use LLMs as simple Python functions

#16

Post by dragonwriter »

ElectricalUnion wrote: post #2

Is it really LLMs (plural) when you only have OpenAPI integration?

text-generation-webui offers an OpenAI API implementation, specifically to support OpenAI API clients, so you can get something more than just OpenAI support by just wrapping the OpenAI API.

You could have more flexibility by abstracting out the underlying LLM APIs, but then you also have a bigger deal with supported features of different APIs, the same conceptual feature supported with very different parameter structures, etc., etc.

Re: Show HN: Magentic – Use LLMs as simple Python functions

#17

Post by jackmpcollins »

ElectricalUnion wrote: post #2

Is it really LLMs (plural) when you only have OpenAPI integration?

Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance. If you have ideas how this should be done let me know - on a github issue would be great to make it visible to others.

Re: Show HN: Magentic – Use LLMs as simple Python functions

#18

Post by jackmpcollins »

jackmpcollins wrote: post #17
ElectricalUnion wrote: post #2

Is it really LLMs (plural) when you only have OpenAPI integration?

Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance . If you have ideas how this should be done let me know - on a github issue would be great to ma…

Oh, and some companies offer APIs that match the OpenAI API and there are some open-source projects that do this for llama running locally. Since those would be compatible with the openai python package they will work with magentic too - though some of these do not support function calling.

See for example Anyscale Endpoints https://app.endpoints.anyscale.com/landing and https://github.com/AmineDiro/cria

Re: Show HN: Magentic – Use LLMs as simple Python functions

#20

Post by AmazingTurtle »

jackmpcollins wrote: post #17
ElectricalUnion wrote: post #2

Is it really LLMs (plural) when you only have OpenAPI integration?

Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance . If you have ideas how this should be done let me know - on a github issue would be great to ma…

I tried out guidance. Encountered endless bugs
Post reply on HN