Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Velvet – Store OpenAI requests in your own DB

usevelvet.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#31

Post by OutOfHere »

DeveloperErrata wrote: post #20
OutOfHere wrote: post #19

Earlier quoted context omitted.

I strongly advise not relying on embedding distance alone for it because it'll match these two: 1. great places to check out in Spain 2. great places to check out in northern Spain Logically the two are not the same, and they could in fact be very different despite their semantic similarity. Your users will be frustrated and will hate you for it. If an LLM validates the two as being the same, then it's fine, but not…

I agree, a naive approach to approximate caching would probably not work for most use cases. I'm speculating here, but I wonder if you could use a two stage pipeline for cache retrieval (kinda like the distance search + reranker model technique used by lots of RAG pipelines). Maybe it would be possible to fine-tune a custom reranker model to only output True if 2 queries are semantically equivalent rather than just s…

Any LLM can output it, but yes, a tuned LLM can benefit with a shorter prompt.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#32

Post by OutOfHere »

knowaveragejoe wrote: post #24
OutOfHere wrote: post #6

A cache is better when it's local rather than on the web. And I certainly don't need to pay anyone to cache local request responses.

How would one achieve something similarly locally, short of just running a proxy and stuffing the request/response pairs into a DB? I'm sure it wouldn't be too terribly hard to write something, but I figure something open source already exists for OpenAI-compatible APIs.

There are any number of databases and language-specific caching libraries. A custom solution or the use of a proxy isn't necessary.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#33

Post by simple10 »

Looks cool. Just out of curiosity, how does this compare to other OpenLLMetry-type observation tools like Arize, Traceloop, LangSmith, LlamaTrace, etc.?

From personal experience, they're all pretty simple to install and use. Then mileage varies in analyzing and taking action on the logs. Does Velvet offer something the others do not?

For my client projects, I've been leaning towards open source platforms like Arize so clients have the option of pulling it inhouse if needed. Most often for HIPAA requirements.

RAG support would be great to add to Velvet. Specifically pgvector and pinecone traces. But maybe Velvet already supports it and I missed it in the quick read of the docs.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#34

Post by manojlds »

nemothekid wrote: post #30
OutOfHere wrote: post #6

A cache is better when it's local rather than on the web. And I certainly don't need to pay anyone to cache local request responses.

As I understand it, your data remains local, as it leverages your own database.

Why do I even ha e to use this saas? This should be a open source lib or just a practice that I implement myself.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#35

Post by heavensteeth »

manojlds wrote: post #34
nemothekid wrote: post #30

Earlier quoted context omitted.

As I understand it, your data remains local, as it leverages your own database.

Why do I even ha e to use this saas? This should be a open source lib or just a practice that I implement myself.

So they can charge you for it.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#36

Post by dsmurrell »

manojlds wrote: post #34
nemothekid wrote: post #30

Earlier quoted context omitted.

As I understand it, your data remains local, as it leverages your own database.

Why do I even ha e to use this saas? This should be a open source lib or just a practice that I implement myself.

Implement it yourself then and save your $$ at the expense of your time.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#37

Post by turnsout »

efriis wrote: post #14
turnsout wrote: post #10

Nice! Sort of like Langsmith without the Langchain, which will be an attractive value proposition to many developers.

Howdy Erick from LangChain here! Just a quick clarification that LangSmith is designed to work great for folks not using LangChain as well :) Check out our quickstart for an example of what that looks like! https://docs.smith.langchain.com/

TIL! LangSmith is great.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#38

Post by marcklingen »

simple10 wrote: post #33

Looks cool. Just out of curiosity, how does this compare to other OpenLLMetry-type observation tools like Arize, Traceloop, LangSmith, LlamaTrace, etc.? From personal experience, they're all pretty simple to install and use. Then mileage varies in analyzing and taking action on the logs. Does Velvet offer something the others do not? For my client projects, I've been leaning towards open source platforms like Arize s…

disclosure: founder/maintainer of Langfuse (OSS LLM application observability)

I believe proxy-based implementations like Velvet are excellent for getting started and solve for the immediate debugging use case; simply changing the base path of the OpenAI SDK makes things really simple (the other solutions mentioned typically require a few more minutes to set up).

At Langfuse (similarly to the other solutions mentioned above), we prioritize asynchronous and batched logging, which is often preferred for its scalability and zero impact on uptime and latency. We have developed numerous integrations (for openai specifically an SDK wrapper), and you can also use our SDKs and Decorators to integrate with any LLM.

> For my client projects, I've been leaning towards open source platforms like Arize so clients have the option of pulling it inhouse if needed. Most often for HIPAA requirements.

I can echo this. We observe many self-hosted deployments in larger enterprises and HIPAA-related companies, thus we made it very simple to self-host Langfuse. Especially when PII is involved, self-hosting makes adopting an LLM observability tool much easier in larger teams.

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#39

Post by beepbooptheory »

bachback wrote: post #29

interesting, seems more of an enterprise offering. its OpenAI only for and you plan to expand to other vendors? anything opensource?

I guess I don't understand what this is now. If its just proxying requests and storing in db, can't it be literally any API?

Re: Show HN: Velvet – Store OpenAI requests in your own DB

#40

Post by elawler24 »

simple10 wrote: post #33

Looks cool. Just out of curiosity, how does this compare to other OpenLLMetry-type observation tools like Arize, Traceloop, LangSmith, LlamaTrace, etc.? From personal experience, they're all pretty simple to install and use. Then mileage varies in analyzing and taking action on the logs. Does Velvet offer something the others do not? For my client projects, I've been leaning towards open source platforms like Arize s…

Velvet takes We warehouse logs directly to your DB, so you can do whatever you want with the data. Build company ops on top of the DB, run your own evals, join with other tables, hash data, etc.

We’re focusing on backend eng workflows so it’s simple to run continuous monitoring, evals, and fine-tuning with any model. Our interface will focus on surfacing data and analytics to PMs and researchers.

For pgvector/pinecone RAG traces - you can start by including meta tags in the header. Those values will be queryable in the JSON object.

Curious to learn more though - feel free to email me at emma@usevelvet.com.

Post reply on HN