Viewing profile — jeadie

This is exactly what we found. Ingest rates were tough. We partitioned and ran over multiple duckdb instances too (and wrangled the complexity). We ending up building a Sqlite + vo…

You might find https://github.com/apache/datafusion and https://github.com/datafusion-contrib/datafusion-federation of interest

We’re building vector indexes into Datafusion for search (starting with S3 vectors). Open source at https://github.com/spiceai/spiceai

This is one of the ideas behind using DuckDB in github.com/spiceai/spiceai

There’s also https://github.com/spiceai/spiceai

This is a common feature now. If anything, for being so early to vector databases, Pinecone was rather late to integrating embeddings. Timescale most recently added it but, yes a b…

Why not just federate Postgres and parquet files? That way the query planner can push down as much of the query and reduce how much data has to move about?

This looks functionally similar as using http://github.com/spiceai/spiceai with a postgreSQL data accelerator.

Spice AI | Senior Software Engineer | GMT+10 (e.g. Australia) through GMT-7 (e.g. Seatle/SF/LA) | Remote | Full Time Spice AI provides building blocks for data and AI-driven applic…

And yes, Iceberg is very high up on our list

Yes! It can connect to FlightSQL compatible servers (see https://docs.spiceai.org/data-connectors/flightsql ) and its also a FlightSQL compatible server

Have you seen github.com/marqo-ai/marqo? It does all this wrapping, and you don't even need to pay for OpenAI or pinecone

I'm very glad that this has some added funding. I am building a serverless API on the cloudflare edge network using GGML as the backbone --> tryinfima.com

"AI Native" catching on

I've tried both Chroma and Qdrant. I don't think Chroma lacks that much. Definitely newer, but is also a great product. I think cloud support coming Q3 2023

(Not affiliated with hyperDB)

I've been using https://github.com/jdagdelen/hyperDB and it's been really easy to use. I think Clickhouse support is on the short-term roadmap.

Most people, like me, who end up needing to use vector DBs, are wanting to use LLMs on a specific, often private dataset/use case. Typically one starts with something like unstruct…

Not a dumb question at all! Essentially what can do Marqo, and this blog shows, is that there is alot of logic and work to do what you said (i.e. pass raw data into LLM, get embedd…

Its a great tool. Unlike vectorDBs alone, Marqo helps the full process that alot of people end up wanting to use vectorDBs for (e.g. have structured data, use LLMs to create embedd…

Being able to handle and ask questions of audio data is a pretty big field. https://www.assemblyai.com/ , for example, is a company entirely dedicated to audio intelligence. They h…

This is generally very context/use case specific. In general, if a document is a `Dict[str, Any]`, then you either have to have one (or multiple) vector(s) per field, unless you wa…

I'm skeptical about some vector databases these days, but your article misses a few import points when it comes to LLMs. 1. To use LLMs effectively, you often need to generate and …

A really interesting blog post I found using LLMs for audio search which I think is a pretty nifty/new idea. I've found it cumbersome using some of the new vector DBs (chroma, fais…