51–58 of 58 posts
Earlier quoted context omitted.
Why do I even ha e to use this saas? This should be a open source lib or just a practice that I implement myself.
Implement it yourself then and save your $$ at the expense of your time.
Earlier quoted context omitted.
Why do I even ha e to use this saas? This should be a open source lib or just a practice that I implement myself.
Implement it yourself then and save your $$ at the expense of your time.
Earlier quoted context omitted.
It's a standalone DB, just for LLM logging. Since it's your DB - you can configure data retention, and migrate data to an analytics DB / warehouse if cost or latency becomes a concern. And, we're happy to support whatever DB you require (ClickHouse, Big Query, Snowflake, etc) in a managed deployment.
I guess I should have elaborated to say that even if you're spinning up a new database expressly for this purpose (which I didn't see specifically called out in your docs anywhere as a best practice), you're starting off on the wrong foot. Maybe I'm old-school, but relational databases should be for relational data. This data isn't relational, it's write-once log data, and it belongs in files on disk, or in purpose-b…
PostgreSQL (Neon) is our free self-serve offering because it’s easy to spin up quickly.
I don't understand the problem that's being solved here. At the scale you're talking about (e.g. millions of requests per day with FindAI), why would I want to house immutable log data inside a relational database, presumably alongside actual relational data that's critical to my app? It's only going to bog down the app for my users. There are plenty of other solutions (examples include Presto, Athena, Redshift, or s…
Here's a video about what we do with the data: https://www.youtube.com/watch?v=KaFkRi5ESi8
Earlier quoted context omitted.
Implement it yourself then and save your $$ at the expense of your time.
Like this is not a big thing to implement, that's my point. There are already libraries like OpenLLMetry and sink to a DB. We are doing something like this already.
Earlier quoted context omitted.
Implement it yourself then and save your $$ at the expense of your time.
If you factor in dealing with somebody's black box code 6 months into a project, you'll realise you're saving both money and time.
Earlier quoted context omitted.
I was impressed by Upstash's approach to something similar with their "Semantic Cache". https://github.com/upstash/semantic-cache "Semantic Cache is a tool for caching natural text based on semantic similarity. It's ideal for any task that involves querying or retrieving information based on meaning, such as natural language classification or caching AI responses. Two pieces of text can be similar but not identical (…
I strongly advise not relying on embedding distance alone for it because it'll match these two: 1. great places to check out in Spain 2. great places to check out in northern Spain Logically the two are not the same, and they could in fact be very different despite their semantic similarity. Your users will be frustrated and will hate you for it. If an LLM validates the two as being the same, then it's fine, but not…
Shameless plug (FOSS): https://github.com/jankovicsandras/plpgsql_bm25 Okapi BM25 search implemented in PL/pgSQL for Postgres.
Looks cool. Just out of curiosity, how does this compare to other OpenLLMetry-type observation tools like Arize, Traceloop, LangSmith, LlamaTrace, etc.? From personal experience, they're all pretty simple to install and use. Then mileage varies in analyzing and taking action on the logs. Does Velvet offer something the others do not? For my client projects, I've been leaning towards open source platforms like Arize s…
disclosure: founder/maintainer of Langfuse (OSS LLM application observability) I believe proxy-based implementations like Velvet are excellent for getting started and solve for the immediate debugging use case; simply changing the base path of the OpenAI SDK makes things really simple (the other solutions mentioned typically require a few more minutes to set up). At Langfuse (similarly to the other solutions mentione…