Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSPy – Programming–not prompting–LMs

dspy.ai

Post reply on HN
Print view View original

11–20 of 46 posts

Re: DSPy – Programming–not prompting–LMs

#11

Post by pizza »

dr_kiszonka wrote: post #5
behnamoh wrote: post #4

Every time I've seen a dspy article, I end up thinking: ok, but what does it do exactly? I've been using guidance, outlines, GBF grammars, etc. What advantage does dspy have over those alternatives? I've learnt that the best package to use LLMs is just Python. These "LLM packages" just make it harder to do customizations as they all make opinionated assumptions and decisions.

Question from a casual AI user, if you have a minute. It seems to me that I could get much more productive by making my own personal AI "system". For example, write a simple pipeline where Claude would scrutinize OpenAI's answers and vice versa. Are there any beginner-friendly Python packages that you would recommend to facilitate fast experimentation with such ideas?

Yes, anthropic just released model context protocol and mcp is perfect for this kind of thing. I actually wrote an mcp server for claude to call out to openai just yesterday.

Re: DSPy – Programming–not prompting–LMs

#12

Post by kingkongjaffa »

dr_kiszonka wrote: post #5
behnamoh wrote: post #4

Every time I've seen a dspy article, I end up thinking: ok, but what does it do exactly? I've been using guidance, outlines, GBF grammars, etc. What advantage does dspy have over those alternatives? I've learnt that the best package to use LLMs is just Python. These "LLM packages" just make it harder to do customizations as they all make opinionated assumptions and decisions.

Question from a casual AI user, if you have a minute. It seems to me that I could get much more productive by making my own personal AI "system". For example, write a simple pipeline where Claude would scrutinize OpenAI's answers and vice versa. Are there any beginner-friendly Python packages that you would recommend to facilitate fast experimentation with such ideas?

You could use the plain api libraries for each llm and ipython notebooks, conceptually each block could be a node or link in the prompt chain, and input/output of each block is printable and visible to check which part of the chain is the part that is failing or has sub optimal outputs.

Re: DSPy – Programming–not prompting–LMs

#13

Post by deepsquirrelnet »

I use DSPy often, and it’s the only framework that I have much interest in using professionally.

Evaluations are first class and have a natural place in optimization. I still usually spend some time adjusting initial prompts, but more time doing traditional ML things… like working with SMEs, building training sets, evaluating models and developing the pipeline. If you’re an ML engineer that’s frustrated by the “loose” nature of developing applications with LLMs, I recommend trying it out.

With assertions and suggestions, there’s also additional pathways you can use to enforce constraints on the output and build in requirements from your customer.

Re: DSPy – Programming–not prompting–LMs

#14

Post by beepbooptheory »

How does it work? Like I can see the goal and the results, but is it in fact the case that its still here "LLMs all the way down"? That is, is there a supplement bot here thats fine tuned to DSPy syntax, doing the actual work of turning the code to prompt? Trying to figure out how else it would work.. But if that is the case, this really feels like a Wizard of Oz behind the curtain thing.

Re: DSPy – Programming–not prompting–LMs

#15

Post by aaronvg »

I found it interesting how DSPy created the Signatures concept: https://dspy.ai/learn/programming/signatures/

We took this kind of concept all the way to making a DSL called BAML, where prompts look like literal functions, with input and output types.

Playground link here https://www.promptfiddle.com/

https://github.com/BoundaryML/baml

(tried pasting code but the formatting is completely off here, sorry).

We think we could run some optimizers on this as well in the future! We'll definitely use DSPy as inspiration!

Re: DSPy – Programming–not prompting–LMs

#16

Post by qeternity »

deepsquirrelnet wrote: post #13

I use DSPy often, and it’s the only framework that I have much interest in using professionally. Evaluations are first class and have a natural place in optimization. I still usually spend some time adjusting initial prompts, but more time doing traditional ML things… like working with SMEs, building training sets, evaluating models and developing the pipeline. If you’re an ML engineer that’s frustrated by the “loose…

What do you actually use it for? I've never been able to actually get it to perform on anything remotely close to what it claims. Sure, it can help optimize few shot prompting...but what else can it reliably do?

Re: DSPy – Programming–not prompting–LMs

#18

Post by flakiness »

scosman wrote: post #9

Can someone explain what DSPy does that fine tuning doesn’t? Structured IO, optimized to better results. Sure. But why just just go straight to weights, instead of trying to optimize the few-shot space?

It has multiple optimization strategies. One is optimizing the few shot list. Another is to let the model write prompts and pick the best one based on the given eval. I doubt latter much more intriguing although I have no idea how practical it is.

Re: DSPy – Programming–not prompting–LMs

#19

Post by huevosabio »

deepsquirrelnet wrote: post #13

I use DSPy often, and it’s the only framework that I have much interest in using professionally. Evaluations are first class and have a natural place in optimization. I still usually spend some time adjusting initial prompts, but more time doing traditional ML things… like working with SMEs, building training sets, evaluating models and developing the pipeline. If you’re an ML engineer that’s frustrated by the “loose…

Every time I check the docs, I feel like it obfuscates so many things that it puts me off and I decide to just not try it out.

Behind the scenes it's using LLM's to find the proper prompting. I find that it uses a terminology and abstraction that is way too complicated for what it is.

Post reply on HN