Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vector search just got up to 10x faster and vertically scalable

pinecone.io

Post reply on HN
Print view View original

31–40 of 54 posts

Re: Vector search just got up to 10x faster and vertically scalable

#31

Post by pbadams »

marginalia_nu wrote: post #30
ww520 wrote: post #23

Earlier quoted context omitted.

Bigger machine doesn't automatically mean higher performance. The code needs to scale with the increased number of cores, has share-nothing or share-very-little approach to avoid contention, and uses efficient data structure to utilize the increased memory.

Why would search queries have contention with each other? Surely it's in the domain of embarrassingly parallel.

I agree in the typical case, but they support concurrent add/delete one of their index options. Handling consistency/contention for modifying whatever graph/tree/etc structure they are using is probably nontrivial, and the resulting cache invalidations would also likely affect the QPS.

P.S. Great work on your site, by the way - it's a really inspiring project!

Re: Vector search just got up to 10x faster and vertically scalable

#32

Post by zcw100 »

opisthenar84 wrote: post #14
baobob wrote: post #13

Confused by their claim to be the 'first' vector database. These things have been around forever? For example FLANN (not a DB server, but example lib) is from 2009

I think the difference is in the layer of abstraction i.e. FLANN is just the underlying search functionality whereas vector databases are fully managed solutions. Even so, Weaviate came out in 2018, so saying that they are the "first" vector database is just flat out wrong since Pinecone was founded in 2019.

Same difference as ElasticSearch and Lucene.

re: difference in layer of abstraction.

Re: Vector search just got up to 10x faster and vertically scalable

#33

Post by faizshah »

baobob wrote: post #13

Confused by their claim to be the 'first' vector database. These things have been around forever? For example FLANN (not a DB server, but example lib) is from 2009

There’s also Vespa from yahoo that has been used at scale for years (decades?): https://docs.vespa.ai/search.html?q=Vector

Re: Vector search just got up to 10x faster and vertically scalable

#34

Post by ragingcoder »

opisthenar84 wrote: post #10

I don't understand the emphasis here on vertical scaling. Move a database to a bigger machine = more storage and faster querying. Not exactly rocket science. Horizontal scaling is the real challenge here, and the complexity of vector indexes makes it especially challenging. Milvus and Vertex AI both have horizontal scaling ANN search and the ability to do parallel indexing as well. I appreciate the post but this does…

Horizontal scaling has been a feature of Pinecone for a while now.

Re: Vector search just got up to 10x faster and vertically scalable

#35

Post by marginalia_nu »

pbadams wrote: post #31
marginalia_nu wrote: post #30

Earlier quoted context omitted.

Why would search queries have contention with each other? Surely it's in the domain of embarrassingly parallel.

I agree in the typical case, but they support concurrent add/delete one of their index options. Handling consistency/contention for modifying whatever graph/tree/etc structure they are using is probably nontrivial, and the resulting cache invalidations would also likely affect the QPS. P.S. Great work on your site, by the way - it's a really inspiring project!

Seems to me you can do that in a way that ensures low contention between consumers by using a read-biased MRSW lock. It's not free such a construction, but it really shouldn't eat into your read performance all that much. You're adding hundreds of nanoseconds to your query time by acquiring and releasing a lock. Unless you're already serving millions of queries per second per thread, this is piss in the ocean.

Re: Vector search just got up to 10x faster and vertically scalable

#36

Post by nl »

fdgsdfogijq wrote: post #22
gk1 wrote: post #17

Earlier quoted context omitted.

Pinecone doesn’t use Faiss, nor ScaNN. We love Faiss and even teach people to use it[1]. There happens to be a sizable population of engineers who need more than what Faiss provides (like live index updates and metadata filtering, for example), and can’t be bothered or aren’t being paid to customize and manage open-source libraries all day. [1] https://www.pinecone.io/learn/faiss/

So you guys developed and implemented state of the art neural network vector search from scratch? in a year? and something better than libraries with tens of contributors over years of research?

Many very competitive vector search libraries are done by small teams.

HNSW in NMSLIB[1] is mostly 3 people's work and it's very competitive[2].

[1] https://github.com/nmslib/nmslib

[2] http://ann-benchmarks.com/glove-100-angular_10_angular.html

Re: Vector search just got up to 10x faster and vertically scalable

#37

Post by fzliu »

ww520 wrote: post #23
opisthenar84 wrote: post #10

I don't understand the emphasis here on vertical scaling. Move a database to a bigger machine = more storage and faster querying. Not exactly rocket science. Horizontal scaling is the real challenge here, and the complexity of vector indexes makes it especially challenging. Milvus and Vertex AI both have horizontal scaling ANN search and the ability to do parallel indexing as well. I appreciate the post but this does…

Bigger machine doesn't automatically mean higher performance. The code needs to scale with the increased number of cores, has share-nothing or share-very-little approach to avoid contention, and uses efficient data structure to utilize the increased memory.

Larger disk space does help effortlessly scale storage in single-node systems, but I agree with you that shared nothing (and/or shared something) is a necessary step for extracting maximum performance on a larger machine. When it comes to distributed architectures, shared nothingness is important as well. The decoupling of storage from compute and stateless from stateful helps minimize resource allocation when it comes to billion-scale vector storage, indexing, and search. Milvus 2.0 implements this type of architecture - here's a link to our VLDB 2022 paper, if you're interested: https://arxiv.org/abs/2206.13843

Re: Vector search just got up to 10x faster and vertically scalable

#38

Post by paystax »

opisthenar84 wrote: post #10

I don't understand the emphasis here on vertical scaling. Move a database to a bigger machine = more storage and faster querying. Not exactly rocket science. Horizontal scaling is the real challenge here, and the complexity of vector indexes makes it especially challenging. Milvus and Vertex AI both have horizontal scaling ANN search and the ability to do parallel indexing as well. I appreciate the post but this does…

Completely true. You have to understand the economics behind this to see why their claim is hyperbole at best and flat out misleading at worst. The fundamentals of scalable vector search is that you are dealing with potentially huge dimensionality and huge datasets, that means that memory consumption will be huge even for a modest (by today's standards) datasets. This problem has garnered lots of research attention, so making such a bold claim makes you think what Pinecone has under the hood that others don't.

Pinecone is VC backed and they have taken in to the tune of $50M in funding. They have to claim the "first" in solving these challenging technical problem, otherwise they'd have to really explain that their "secret source" is not really ground-breaking but relying on a series of open-source components under the hood. VCs wouldn't want to be backing yet another donkey in the derby. The truth is that solutions like FAISS, ScaNN, Weaviate, Quadrant, ANNOY and co. are working on this problem on a much more fundamental level. Pinecone and Google vertex matching AI are working on it on a application level. If Pinecones's solution is truly groundbreaking, they'd publish it in a more scientifically rigorous way. So these claims are to be taken with a grain of salt for what they are: developer evangelism/marketing speak.

Re: Vector search just got up to 10x faster and vertically scalable

#39

Post by utopcell »

faizshah wrote: post #33
baobob wrote: post #13

Confused by their claim to be the 'first' vector database. These things have been around forever? For example FLANN (not a DB server, but example lib) is from 2009

There’s also Vespa from yahoo that has been used at scale for years (decades?): https://docs.vespa.ai/search.html?q=Vector

Interesting connection considering that some of the Pinecone founders are ex-Yahoos.

Re: Vector search just got up to 10x faster and vertically scalable

#40

Post by Sommer »

gk1 wrote: post #18
fdgsdfogijq wrote: post #11

Earlier quoted context omitted.

FAISS

This is incorrect.

Strange comments and denial coming from the person who submitted a post about pinecone wrapping Faiss a year ago https://news.ycombinator.com/item?id=27502458
Post reply on HN