Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSPy – Programming–not prompting–LMs

dspy.ai

Post reply on HN
Print view View original

41–46 of 46 posts

Re: DSPy – Programming–not prompting–LMs

#41

Post by vergessenmir »

thatsadude wrote: post #28
byefruit wrote: post #2

I've seen a couple of talks on DSPy and tried to use it for one of my projects but the structure always feels somewhat strained. It seems to be suited for tasks that are primarily show, don't tell but what do you do when you have significant prior instruction you want to tell? e.g Tests I want applied to anything retrieved from the database. What I'd like is to optimise the prompt around those (or maybe even the test…

You can optimize prompt with MIPROv2 without examples (set the max number of examples to 0)

And have the chat history end up as an input field instead of each request response being prepared individually for the LLM endpoint?

I'm new to the library but from what I can see the Chat adapter will do this automatically if I use the forward call

Quoting the docs: "Though rarely needed, you can write custom LMs by inheriting from dspy.BaseLM. Another advanced layer in the DSPy ecosystem is that of adapters, which sit between DSPy signatures and LMs. A future version of this guide will discuss these advanced features, though you likely don't need them."

I could be wrong, I could be looking for complexity where there is none.

Have a fundamentally misunderstood how this all works, it sometimes feels like I have?

Re: DSPy – Programming–not prompting–LMs

#42

Post by aaronvg »

vergessenmir wrote: post #39
aaronvg wrote: post #15

I found it interesting how DSPy created the Signatures concept: https://dspy.ai/learn/programming/signatures/ We took this kind of concept all the way to making a DSL called BAML, where prompts look like literal functions, with input and output types. Playground link here https://www.promptfiddle.com/ https://github.com/BoundaryML/baml (tried pasting code but the formatting is completely off here, sorry). We think we…

Wouldn't try and convince the nay sayers. I have a few comments (or opinions rather) This coming from a recent dspy user hitting its pain points. - The GitHub page is very busy - A clear example should come up early on the page. It's only when I got to the fiddle I could see a motivating example i.e the extractions, functions and tests. - Then a section for running tests/evaluations -Then deployment or run with/witho…

Thanks, we also do support swapping models at runtime!

Will incorporate this feedback.

Re: DSPy – Programming–not prompting–LMs

#43

Post by vergessenmir »

aaronvg wrote: post #15

I found it interesting how DSPy created the Signatures concept: https://dspy.ai/learn/programming/signatures/ We took this kind of concept all the way to making a DSL called BAML, where prompts look like literal functions, with input and output types. Playground link here https://www.promptfiddle.com/ https://github.com/BoundaryML/baml (tried pasting code but the formatting is completely off here, sorry). We think we…

Wouldn't try and convince the nay sayers. I have a few comments (or opinions rather) This coming from a recent dspy user hitting its pain points.

- The GitHub page is very busy

- A clear example should come up early on the page. It's only when I got to the fiddle I could see a motivating example i.e the extractions, functions and tests.

- Then a section for running tests/evaluations

-Then deployment or run with/without the baml cli

- I do wonder if all the functions have to be so tightly coupled with the model. In dspy my modules are model agnostic and I can evaluate behaviour across different models. It's not so clear how to do this

Re: DSPy – Programming–not prompting–LMs

#44

Post by OxfordOutlander »

dr_kiszonka wrote: post #5
behnamoh wrote: post #4

Every time I've seen a dspy article, I end up thinking: ok, but what does it do exactly? I've been using guidance, outlines, GBF grammars, etc. What advantage does dspy have over those alternatives? I've learnt that the best package to use LLMs is just Python. These "LLM packages" just make it harder to do customizations as they all make opinionated assumptions and decisions.

Question from a casual AI user, if you have a minute. It seems to me that I could get much more productive by making my own personal AI "system". For example, write a simple pipeline where Claude would scrutinize OpenAI's answers and vice versa. Are there any beginner-friendly Python packages that you would recommend to facilitate fast experimentation with such ideas?

Also not OP but adding my 2c.

I tried many python frameworks but the lack of customizations and observability limited the utility. Now, I only use Instructor (Jason Liu's library). That, and concurrent futures for parallel processing.

Re: DSPy – Programming–not prompting–LMs

#45

Post by dr_kiszonka »

digdugdirk wrote: post #8
dr_kiszonka wrote: post #5

Earlier quoted context omitted.

Question from a casual AI user, if you have a minute. It seems to me that I could get much more productive by making my own personal AI "system". For example, write a simple pipeline where Claude would scrutinize OpenAI's answers and vice versa. Are there any beginner-friendly Python packages that you would recommend to facilitate fast experimentation with such ideas?

I've had good luck with a light "shim layer" library that handles the actual interfacing with the api and implements the plumbing on any fun new features that get introduced. I've settled on the Mirascope library ( https://mirascope.com/ ), which suits my use cases and lets me implement structured inputs/outputs via pydantic models, which is nice. I really like using it, and the team behind it is really responsive an…

Mirascope looks great. Thanks!

Re: DSPy – Programming–not prompting–LMs

#46

Post by dr_kiszonka »

d4rkp4ttern wrote: post #26
dr_kiszonka wrote: post #5

Earlier quoted context omitted.

Question from a casual AI user, if you have a minute. It seems to me that I could get much more productive by making my own personal AI "system". For example, write a simple pipeline where Claude would scrutinize OpenAI's answers and vice versa. Are there any beginner-friendly Python packages that you would recommend to facilitate fast experimentation with such ideas?

You can have a look at Langroid -- it's an agent-oriented LLM programming framework from CMU/UW-Madison researchers. We started building it in Apr 2023 out of frustration with the bloat of then-existing libs. In langroid you set up a ChatAgent class which encapsulates an LLM-interface plus any state you'd like. There's a Task class that wraps an Agent and allows inter-agent communication and tool-handling. We have de…

Neat. Thanks!
Post reply on HN