Live data from Hacker News

Vector database built for scalable similarity search

milvus.io

11–20 of 95 posts

Re: Vector database built for scalable similarity search

#11
post #5

Ah I see, they released Milvus 2.0 which is similar to Pinecone. What are the differences?

Biggest difference is Milvus is self-hosted (manage your own infra) while Pinecone is a managed service. There are other differences but that’s often a driving factor.

We (Pinecone) tend to attract customers who want to start, ship, and scale quickly and reliably without worrying about infra and ops overhead.

https://www.pinecone.io

Re: Vector database built for scalable similarity search

#12
post #10
post #6

Can these databases do fast averaging of nearest neighbors for regression or do you have to retrieve the neighbors and manually compute a mean across them?

You'll have to manually compute the mean after retrieving the nearest neighbors. Automatic fast averaging could be an interesting use case for clustering or using a vector database to generate training though.

Interesting. What systems or algorithms are you referring to by “clustering”?

Re: Vector database built for scalable similarity search

#13
post #11
post #5

Ah I see, they released Milvus 2.0 which is similar to Pinecone. What are the differences?

Biggest difference is Milvus is self-hosted (manage your own infra) while Pinecone is a managed service. There are other differences but that’s often a driving factor. We (Pinecone) tend to attract customers who want to start, ship, and scale quickly and reliably without worrying about infra and ops overhead. https://www.pinecone.io

I think this update introduces Milvus 2.0, "Managed Milvus", which seems to be not self-hosted, but in-cloud managed. You can create account and see that it's similar to Pinecone.

Re: Vector database built for scalable similarity search

#14
post #5

Ah I see, they released Milvus 2.0 which is similar to Pinecone. What are the differences?

Milvus is completely open source (https://github.com/milvus-io/milvus) and supports a variety of index types (https://milvus.io/docs/overview.md#Index-types) and support various consistency levels, scalar/metadata filtering, and time travel. We started working on Milvus back in 2018, with 2.0 being released in January 2022 (https://github.com/milvus-io/milvus/releases/tag/v2.0.0).

For those interested, here's a comparison with other open source vector databases: https://zilliz.com/comparison. For those who don't want to be burdened with installing and maintaining a local database, there's a managed service available as well: https://zilliz.com/cloud.

Re: Vector database built for scalable similarity search

#15
post #13
post #11

Earlier quoted context omitted.

Biggest difference is Milvus is self-hosted (manage your own infra) while Pinecone is a managed service. There are other differences but that’s often a driving factor. We (Pinecone) tend to attract customers who want to start, ship, and scale quickly and reliably without worrying about infra and ops overhead. https://www.pinecone.io

I think this update introduces Milvus 2.0, "Managed Milvus", which seems to be not self-hosted, but in-cloud managed. You can create account and see that it's similar to Pinecone.

Yup! Zilliz Cloud (managed Milvus) provides free credits for you to get started with a full-service (no throttling or other limitations) PoC as well.

Re: Vector database built for scalable similarity search

#17
post #14
post #5

Ah I see, they released Milvus 2.0 which is similar to Pinecone. What are the differences?

Milvus is completely open source ( https://github.com/milvus-io/milvus ) and supports a variety of index types ( https://milvus.io/docs/overview.md#Index-types ) and support various consistency levels, scalar/metadata filtering, and time travel. We started working on Milvus back in 2018, with 2.0 being released in January 2022 ( https://github.com/milvus-io/milvus/releases/tag/v2.0.0 ). For those interested, here's a…

Thanks! BTW it only compares open-sourced databases, and there's no Pinecone in it, which is a very popular cloud-managed option. Would be cool if you could benchmark/compare it.

Re: Vector database built for scalable similarity search

#20
post #8
post #7

Earlier quoted context omitted.

Does Zilliz do BYOC?

Not yet, but this functionality should be coming soon. We're currently working on adding the capability to call third party embedding APIs directly from a Zilliz Cloud instance.

I was mulling over the idea of building keyword image search (say, with CLIP based embeddings). However I'm not really sure the cost, or whether or not this is the best solution. Do you have any case studies about large deployments of this software, and what the upper limits of scale might be?
Post reply on HN