141–144 of 144 posts
Earlier quoted context omitted.
1024 bits for a hash is pretty roomy. The embedding "just" has to be well-distributed across enough of the dimensions.
Yeah, that's what I was thinking: Did we think 32 bits across each of the 1024 dimensions would be necessary? Maybe 32768 bits is adding unnecessary precision to what is ~1024 bits of information in the first place.
We at https://github.com/tensorchord/VectorChord solved most of the pgvector issues mentioned in this blog: - We're IVF + quantization, can support 15x more updates per second comparing to pgvector's HNSW. Insert or delete an element in a posting list is a super light operation comparing to modify a graph (HNSW) - Our main branch can now index 100M 768-dim vector in 20min with 16vcpu and 32G memory. This enables user…
Earlier quoted context omitted.
Also worth mentioning that we use quantization extensively: - halfvec (16bit float) for storage - bit (binary vectors) for indexes Which makes the storage cost and on-going performance good enough that we could enable this in all our hosting.
It still amazes me that the binary trick works. For anyone who hasn't seen it yet: it turns out many embedding vectors of e.g. 1024 floating point numbers can be reduced to a single bit per value that records if it's higher or lower than 0... and in this reduced form much of the embedding math still works! This means you can e.g. filter to the top 100 using extremely memory efficient and fast bit vectors, then run a…
Earlier quoted context omitted.
In Discourse embeddings power: - Related Topics, a list of topics to read next, which uses embeddings of the current topic as the key to search for similar ones - Suggesting tags and categories when composing a new topic - Augmented search - RAG for uploaded files
Thanks for the details. Also, always appreciated Discord's engineering blog posts. Lots of interesting stories, and nice to see a company discuss using Elixir at scale.