Viewing profile — okhat

This is a DSPy optimizer, built by the DSPy core team. Just wait for open sourcing.

I'll admit even I sometimes wish ColBERT was more user-friendly. I'll probably start using ColBERT throught RAGatouille now.

Writing right now. This month :-)

Thanks! Lots to discuss from your excellent response, but I'll address the easy part first: DSPy is v2 of DSP (demonstrate-search-predict). The DSPy paper hasn't been released yet.…

@wokwokwok Okay now we disagree. This task is not easy, it's just easy to follow in one notebook. (If it were easy, the RAG score wouldn't be 26%.) As for "carefully crafted string…

Ah okay makes sense, yeah we'll release more examples. This is just an intro to the key concepts/modules.

What did you find underwhelming if I may ask? It shows you how it takes some ~25 Pythonic lines of code to make GPT-3.5 retrieval accuracy go from the 26-36% range to 60%. Not a ba…

Posted an answer here: https://news.ycombinator.com/item?id=37420175

Here's the key idea. You give DSPy (1) your free-form code with declarative calls to LMs, (2) a few inputs [labels optional], and (3) some validation metric [e.g., sanity checks]. …

These are answers to specific questions below.

[flagged]

just posted a top-level answer, copied from the FAQs of DSPy

See the discussion of teleprompters here: https://colab.research.google.com/github/stanfordnlp/dspy/bl...

"print out every prompt that is generated to a log" --- yes of course This Colab is full of prompts and examples of improving the quality of gpt-3.5-turbo: https://t.co/Oa1RDp3XbZ …

@simonw it sounds like we'd agree that: 1] when prototyping, it's useful to not have to tweak each prompt by hand as long as you can inspect them easily 2] when the system design i…

btw read a more official answer here: https://github.com/stanfordnlp/dspy#5a-dspy-vs-thin-wrappers...

[flagged]

"A neural network layer is just a matrix. Why abstract that matrix and learn it?" Well, because it's not your job to figure out how to hardcode delicate string or floats that work …

DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax and an automatic compiler that teaches LMs how to conduct the declarative steps i…

There’s always DSP for those who need a lightweight but powerful programming model — not a library of predefined prompts and integrations. It’s a very different experience from the…

Very cool! But for hard enough problems, prompt engineering is kind of like hyperparameter tuning. It's only a final (and relatively minor) step after building up an effective arch…