Viewing profile — marekgalovic

We do a projection of the original vectors so that it matches one of our optimized kernel. This generally gives us better recall vs. simple padding since all bits are utilized.

We actually tried to extend DataFusion at first but ultimately decided against it since we can get most of the value by using Arrow and its compute kernels directly. DataFusion als…

Founder of TopK here. There are legit use cases for vector-based retrieval (e.g. semantic search, recommendations, multi-modal search, etc.) but that only requires supporting vecto…