Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 24 posts

Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#1

Post by wakahiu »

Hi HN. Peter here. As a machine learning engineer, I mostly think in terms of feature vectors, embeddings, and matrices. One of the most useful byproducts of deep neural networks is embeddings because they allow us to represent high-dimensional data in terms of lower-dimensional latent vectors. These feature vectors can be used for downstream applications like similarly search, recommendation systems and near duplicate detection.

As an ML engineer, I was frustrated by the lack of a datastore in which vectors are first-class citizens. As a result, most ML engineers, including myself, end up using awkward workarounds to store vectors such as arrays in SQL/NoSQL databases, stringifying vectors and storing them as text in in-memory-based caching systems such as Redis ETC. Furthermore, these systems don't allow for vector-based query operations such as nearest neighbor search. Consequently, engineers have to deploy additional approximate nearest neighbor search systems such as Facebook's FAISS or Spotify's ANNOY. These systems, while nifty and fast, are difficult to install and are costly to maintain. To address these issues, I built NNext, a managed vector datastore in which vectors are first class citizens. NNext, allows you to store vectors along with any json-blob metadata. Furthermore, NNext comes with a fast approximate nearest-neighbor (ANN) search capability.

I would love to get feedback on your experience as Data Scientist or ML engineer storing feature vectors and ANN systems. Please shoot me an email at [redacted].

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#3

Post by KhoomeiK »

What I've done for most of my research projects is just pickle a PyTorch dataset object that contains all my embeddings. The pkl file can then just be uploaded anywhere and becomes plug and play with any Torch model.

What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#4

Post by wakahiu »

phenkdo wrote: post #2

How's this different from Milvus?

Main thing is that NNext is fully managed - you don’t have to worry about provisioning servers, version upgrades and package installation/ dependencies. One of the main thing I’ve observed about ML engineers is that they typically don’t want to be encumbered by general software engineering / dev-ops tasks like platform management. They want to focus on data.

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#5

Post by wakahiu »

KhoomeiK wrote: post #3

What I've done for most of my research projects is just pickle a PyTorch dataset object that contains all my embeddings. The pkl file can then just be uploaded anywhere and becomes plug and play with any Torch model. What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?

Exactly, once you have latency-sensitive applications that read-write embeddings, then you have to rethink your vector storage system. Furthermore, when your dataset grows to millions of datapoints, having a system that scales according avoids downtime and costly query lookup operations.

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#6

Post by wakahiu »

KhoomeiK wrote: post #3

What I've done for most of my research projects is just pickle a PyTorch dataset object that contains all my embeddings. The pkl file can then just be uploaded anywhere and becomes plug and play with any Torch model. What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?

How would you handle an online ML application where the set of embeddings is changing such as an image recognition app where images are constantly being added and need to be deduplicated?

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#8

Post by vladf »

I can't wait to see the vector DBs we're going to have 5 years from now when everyone needs to serve their embeddings. It's clearly early but frothy right now.

Also check out this similar co I ran into: https://www.pinecone.io/ (the CEO, EL, has some classic sketches and feature hashing papers).

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#9

Post by wakahiu »

vladf wrote: post #8

I can't wait to see the vector DBs we're going to have 5 years from now when everyone needs to serve their embeddings. It's clearly early but frothy right now. Also check out this similar co I ran into: https://www.pinecone.io/ (the CEO, EL, has some classic sketches and feature hashing papers).

This is really cool. Thanks for the link.

Re: Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

#10

Post by malux85 »

I love this idea, but there's no way a managed service like this is going to work, for two reasons: Performance and compliance.

Compliance is pretty obvious and even if you don't store the feature dictionaries and only the vectors, that's a hard conversation to have with the compliance team. I GET IT, without the feature dictionaries the vectors are useless, I KNOW this is how it works from a technical point of view, but the compliance team still won't sign off on it. That's just the way of the world.

And much much more important is performance, uncompressed high dimensional features are huge and even if you use run-length-encoding or sparse vector storage in the protocol plus some lossless compression, I have trouble keeping the GPU fed from disk, let alone over the network, it's going to be multiple orders of magnitude too slow. If the benefits is not claimed to be streaming but are fast vector similarity, keep in mind I can do cosine similarity on literally millions of vectors a second on a single CPU core using vanilla numpy, this was fast enough for me to implement realtime face recognition vector search for Dubai airport, so pretty high scale operations.

I've love a self hosted version of this, optimised for I/O throughput to the GPU. That would be great.

Post reply on HN