Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Marqo – Vectorless Vector Search

github.com

Post reply on HN
Print view View original

1–10 of 17 posts

Show HN: Marqo – Vectorless Vector Search

#1

Post by jn2clark »

Marqo is an end-to-end vector search engine. It contains everything required to integrate vector search into an application in a single API. Here is a code snippet for a minimal example of vector search with Marqo:

mq = marqo.Client()

mq.create_index("my-first-index")

mq.index("my-first-index").add_documents([{"title": "The Travels of Marco Polo"}])

results = mq.index("my-first-index").search(q="Marqo Polo")

Why Marqo? Vector similarity alone is not enough for vector search. Vector search requires more than a vector database - it also requires machine learning (ML) deployment and management, preprocessing and transformations of inputs as well as the ability to modify search behavior without retraining a model. Marqo contains all these pieces, enabling developers to build vector search into their application with minimal effort.

Why not X, Y, Z vector database? Vector databases are specialized components for vector similarity. They are “vectors in - vectors out”. They still require the production of vectors, management of the ML models, associated orchestration and processing of the inputs. Marqo makes this easy by being “documents in, documents out”. Preprocessing of text and images, embedding the content, storing meta-data and deployment of inference and storage is all taken care of by Marqo. We have been running Marqo for production workloads with both low-latency and large index requirements.

Marqo features:

- Low-latency (10’s ms - configuration dependent), large scale (10’s - 100’s M vectors). - Easily integrates with LLM’s and other generative AI - augmented generation using a knowledge base. - Pre-configured open source embedding models - SBERT, Huggingface, CLIP/OpenCLIP. - Pre-filtering and lexical search. - Multimodal model support - search text and/or images. - Custom models - load models fine tuned from your own data. - Ranking with document meta data - bias the similarity with properties like popularity. - Multi-term multi-modal queries - allows per query personalization and topic avoidance. - Multi-modal representations - search over documents that have both text and images. - GPU/CPU/ONNX/PyTorch inference support.

See some examples here:

Multimodal search: [1] https://www.marqo.ai/blog/context-is-all-you-need-multimodal...

Refining image quality and identifying unwanted content: [2] https://www.marqo.ai/blog/refining-image-quality-and-elimina...

Question answering over transcripts of speech: [3] https://www.marqo.ai/blog/speech-processing

Question and answering over technical documents and augmenting NPC's with a backstory: [4] https://www.marqo.ai/blog/from-iron-manual-to-ironman-augmen...

Show HN: Marqo – Vectorless Vector Search
github.com

Re: Show HN: Marqo – Vectorless Vector Search

#3

Post by jn2clark »

bryanrasmussen wrote: post #2

probably stupid question - is there a way to use this to search over graph data - like some way to do graph embeddings here to map a graph to the vectors?

Good question! At the moment if you have the abstraction of data -> model -> vector then it is amenable to searching like this. It will depend a bit on the use case though.

Re: Show HN: Marqo – Vectorless Vector Search

#4

Post by loxias »

I get your larger point, but the errors and phrasing are a bit off putting.

Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right.

There might be room in the market for an integrator, an all in one platform. It won't have the best performance or functionality, I doubt it would win in _any_ category. But if you can get the business model working right I could imagine such a product having sizeable market share. Hm...

Edit: I'm also curious about the dimension and metric used. Any numbers about latency or size is kinda pointless without :).

1 point in 1536-D space (what OpenAI uses),4 byte float == 6KB, so even 100 million points is only 600G...

Re: Show HN: Marqo – Vectorless Vector Search

#5

Post by jn2clark »

loxias wrote: post #4

I get your larger point, but the errors and phrasing are a bit off putting. Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right. There might be room in the market for an integrator, an all in one…

I think it depends a bit on the definition of search here. It might satisfy a literal definition of search but not search as users would expect - which I think is the important point. IMHO vector similarity and vector search are conflated too much and solving search problems as users expect them requires more than similarity.

Re: Show HN: Marqo – Vectorless Vector Search

#6

Post by jn2clark »

loxias wrote: post #4

I get your larger point, but the errors and phrasing are a bit off putting. Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right. There might be room in the market for an integrator, an all in one…

Regarding metric and dimension - it is really problem dependent as is throughput. Recall and latency numbers reported in benchmarks are typically on very well curated and structured datasets and average across all queries. Recall is not just a function of the HNSW algorithm. I can tell you though you can do 70M vector indexes with 768 dimensions https://docs.google.com/spreadsheets/d/1ftHKf4MovnAyKhGyi05e.... If there are particular things you want to see let us know and we can add them!

Re: Show HN: Marqo – Vectorless Vector Search

#8

Post by blackkettle »

loxias wrote: post #4

I get your larger point, but the errors and phrasing are a bit off putting. Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right. There might be room in the market for an integrator, an all in one…

[dead]

Re: Show HN: Marqo – Vectorless Vector Search

#9

Post by loxias »

jn2clark wrote: post #6
loxias wrote: post #4

I get your larger point, but the errors and phrasing are a bit off putting. Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right. There might be room in the market for an integrator, an all in one…

Regarding metric and dimension - it is really problem dependent as is throughput. Recall and latency numbers reported in benchmarks are typically on very well curated and structured datasets and average across all queries. Recall is not just a function of the HNSW algorithm. I can tell you though you can do 70M vector indexes with 768 dimensions https://docs.google.com/spreadsheets/d/1ftHKf4MovnAyKhGyi05e... . If the…

> it is really problem dependent as is throughput. Recall and latency numbers reported in benchmarks are typically on very well curated and structured datasets and average across all queries

This is correct. :) Don't worry, I know enough to not trust any published benchmarks on this topic... (I'm also not your target market. I wrote my first "vector DB" in 2001 for music recognition.)

I still think it's crucial to include just a few more facts though, because otherwise the statement is meaningless.

Consider:

A. "we can find an approximate NN match, euclidean, D=768, N=70000000, under 100ms on a modern laptop"

vs

B. "we can find an approximate NN match, euclidean, D=2, N=70000000, under 100ms on a modern laptop"

vs

C. "we can find an approximate NN match, euclidean, D=768, N=70000000, under 100ms on 1000x modern laptops"

Notice how B and C aren't impressive, they're trivially beatable. :)

Re: Show HN: Marqo – Vectorless Vector Search

#10

Post by loxias »

jn2clark wrote: post #5
loxias wrote: post #4

I get your larger point, but the errors and phrasing are a bit off putting. Vector similarity alone _IS_ enough for vector search. That's literally what "search" means in this context! Finding another vector within an epsilon bound given a metric. After the 3rd read, I understand the point you're trying to make I think, and I think you might be right. There might be room in the market for an integrator, an all in one…

I think it depends a bit on the definition of search here. It might satisfy a literal definition of search but not search as users would expect - which I think is the important point. IMHO vector similarity and vector search are conflated too much and solving search problems as users expect them requires more than similarity.

I think you might be on to something, in thinking about it in terms of the platform from the perspective of the end user, and what they build on it.

I humbly posit that you might be better off, at least from a communications/marketing perspective, ditching the "vector search without vectors" verbage because that alienates the segment that, uh, for lack of a better term, loves and understands high dimensional applied math, and computers. :)

Perhaps instead find language that couches it as an entirely new category. Blue ocean. Ditch the word "vector" entirely.

-$0.02

Post reply on HN