DSPy: Framework for programming with foundation models
1–10 of 56 posts
Re: DSPy: Framework for programming with foundation models
#2DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
Re: DSPy: Framework for programming with foundation models
#3DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs"
I'm having trouble understanding the value provided here.
A prompt is a string. Why abstract that string away from me like this?
My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
Re: DSPy: Framework for programming with foundation models
#4DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs" I'm having trouble understanding the value provided here. A prompt is a string. Why abstract that string away from me like this? My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
Agreed. I don’t understand the trend for abstracting away the best possible human interface.
Re: DSPy: Framework for programming with foundation models
#5DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs" I'm having trouble understanding the value provided here. A prompt is a string. Why abstract that string away from me like this? My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
"A neural network layer is just a matrix. Why abstract that matrix and learn it?" Well, because it's not your job to figure out how to hardcode delicate string or floats that work well for a given architecture & backend.
We want developers to iterate quickly on system designs: How should we break down the task? Where do we call LMs? What should they do?
---
If you can guess the right prompts right away for each LLM, tweak them well for any complex pipeline, and rarely have to change the pipeline (and hence all prompts in it), then you probably won't need this.
That said, it turns out that (a) prompts that work well are very specific to particular LMs, large & especially small ones, (b) prompts that work well change significantly when you tweak your pipeline or your data, and (c) prompts that work well may be long and time-consuming to find.
Oh, and often the prompt that works well changes for different inputs. Thinking in terms of strings is a glaring anti-pattern.
Re: DSPy: Framework for programming with foundation models
#6[flagged]
Re: DSPy: Framework for programming with foundation models
#7Just yesterday I was thinking to myself "I wonder how long it will be until we start generating prompts using code and then we're just writing code again"
Re: DSPy: Framework for programming with foundation models
#8DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs" I'm having trouble understanding the value provided here. A prompt is a string. Why abstract that string away from me like this? My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
btw read a more official answer here:
https://github.com/stanfordnlp/dspy#5a-dspy-vs-thin-wrappers...
Re: DSPy: Framework for programming with foundation models
#9Earlier quoted context omitted.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs" I'm having trouble understanding the value provided here. A prompt is a string. Why abstract that string away from me like this? My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
"A neural network layer is just a matrix. Why abstract that matrix and learn it?" Well, because it's not your job to figure out how to hardcode delicate string or floats that work well for a given architecture & backend. We want developers to iterate quickly on system designs: How should we break down the task? Where do we call LMs? What should they do? --- If you can guess the right prompts right away for each LLM,…
I agree with you on all of those points - but my conclusion is different: those are the reasons it's so important to me that the prompts are not abstracted away from me!
I'm working with Llama 2 a bunch at the moment and much of the challenge is learning how to prompt it differently from how I prompt GPT-4. I'm not yet convinced that an abstraction will solve that problem for me.
Re: DSPy: Framework for programming with foundation models
#10DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.
"Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs" I'm having trouble understanding the value provided here. A prompt is a string. Why abstract that string away from me like this? My instinct is that this will make it harder, not easier, for me to understand what's going on and make necessary changes.
Because there are magic strings that work considerably better than what you might come up with. Like “think step by step.”