Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java Panama Vector API Integrated with Apache Lucene

github.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: Java Panama Vector API Integrated with Apache Lucene

#11

Post by pjmlp »

softwaredoug wrote: post #7

This will be a big deal if Lucene got competitive on http://ann-benchmarks.com if it became a serious alternative (and more holistic) than the vector databases. But it comes with continued challenges if I understand: - Panama is an incubating API and Java has taken its time having an official way of using SIMD. It could all change in Java 22 - It only works on Java 20, with a very specific set of flags passed to the…

Great if this forces Java upgrades.

Using Solr with Java 8 is still quite common.

Re: Java Panama Vector API Integrated with Apache Lucene

#12

Post by re-thc »

invalidname wrote: post #8
dzaima wrote: post #4

Earlier quoted context omitted.

A little confusing because "vector" here (largely) refers to two different things. "Vector search" being this ANN thing, but the "Vector API" is about SIMD. SIMD provides CPU operations on a bunch of data at a time, i.e. instead of one instruction for each 32-bit float, you operate on, depending on the CPU, 128 or 256 or 512 bits worth of floats at the same time. So, over scalar code, SIMD here could get maybe a 4-16…

Great explanation. But to be clear to those who don't follow: SIMD is supported by Java out of the box but the optimizer might miss some opportunities. With this API it is far more likely that SIMD will be used if it's available and on first compilation so performance should be improved.

> SIMD is supported by Java out of the box but the optimizer might miss some opportunities

It's a little limited due to how objects are stored in memory. Might improve with Valhalla.

Re: Java Panama Vector API Integrated with Apache Lucene

#13

Post by dzaima »

re-thc wrote: post #12
invalidname wrote: post #8

Earlier quoted context omitted.

Great explanation. But to be clear to those who don't follow: SIMD is supported by Java out of the box but the optimizer might miss some opportunities. With this API it is far more likely that SIMD will be used if it's available and on first compilation so performance should be improved.

> SIMD is supported by Java out of the box but the optimizer might miss some opportunities It's a little limited due to how objects are stored in memory. Might improve with Valhalla.

Lucene here just dealing with plain float[]s, so Valhalla at least shouldn't affect it much. It seems the limiting thing here is that it has sum accumulators, which the optimizer can't reorder because addition isn't associative.

(for reference: scalar impl: https://github.com/ChrisHegarty/lucene/blob/dd4eaac9af346a21... and SIMD impl: https://github.com/ChrisHegarty/lucene/blob/dd4eaac9af346a21...)

Re: Java Panama Vector API Integrated with Apache Lucene

#14

Post by re-thc »

pjmlp wrote: post #11
softwaredoug wrote: post #7

This will be a big deal if Lucene got competitive on http://ann-benchmarks.com if it became a serious alternative (and more holistic) than the vector databases. But it comes with continued challenges if I understand: - Panama is an incubating API and Java has taken its time having an official way of using SIMD. It could all change in Java 22 - It only works on Java 20, with a very specific set of flags passed to the…

Great if this forces Java upgrades. Using Solr with Java 8 is still quite common.

The newer GC options should have already forced people to upgrade. A use case like search churns through memory.

Maybe just what I've seen but Solr usually has users sticking with older versions of Java.

Re: Java Panama Vector API Integrated with Apache Lucene

#15

Post by softwaredoug »

sandGorgon wrote: post #10

Can someone answer this - today elasticsearch/lucene does vector search using HNSWLib. Will this allow lucene to drop hnswlib and get similar performance using native java ?

Elasticsearch/Lucene do not use HNSWLib, they use their own Java implementation of the HNSW algorithm. But Java, being a virtual machine, doesn't have access to some lower level CPU instructions known to make certain calculations of vector similarity fast (aka SIMD).

Java has taken its sweet time exposing these optimizations in a consistent way. They're available by turning on a flag. But the API for using these optimizations is fairly brittle and could change in the next Java version.

But long story short, they implemented something to turn on the flag, and improve their own HNSW performance. Fingers crossed Java gets its act together.

Re: Java Panama Vector API Integrated with Apache Lucene

#16

Post by jbaiter »

re-thc wrote: post #14
pjmlp wrote: post #11

Earlier quoted context omitted.

Great if this forces Java upgrades. Using Solr with Java 8 is still quite common.

The newer GC options should have already forced people to upgrade. A use case like search churns through memory. Maybe just what I've seen but Solr usually has users sticking with older versions of Java.

From Solr 8 on you need a Java 11 runtime.

Re: Java Panama Vector API Integrated with Apache Lucene

#17

Post by pjmlp »

jbaiter wrote: post #16
re-thc wrote: post #14

Earlier quoted context omitted.

The newer GC options should have already forced people to upgrade. A use case like search churns through memory. Maybe just what I've seen but Solr usually has users sticking with older versions of Java.

From Solr 8 on you need a Java 11 runtime.

Solr 9, for Solr 8, Java 8 is still used.

https://solr.apache.org/guide/8_11/solr-system-requirements....

Which is a pain, many products that use Solr as dependency like Sitecore, still lag behind in Solr versions.

https://www.searchstax.com/docs/searchstax-cloud-sitecore-so...

While it mostly works with more recent versions, there is no support if issues cannot be replicated on officially supported versions.

Re: Java Panama Vector API Integrated with Apache Lucene

#18

Post by ohye »

softwaredoug wrote: post #7

This will be a big deal if Lucene got competitive on http://ann-benchmarks.com if it became a serious alternative (and more holistic) than the vector databases. But it comes with continued challenges if I understand: - Panama is an incubating API and Java has taken its time having an official way of using SIMD. It could all change in Java 22 - It only works on Java 20, with a very specific set of flags passed to the…

Elasticsearch is already on Java 20: https://github.com/elastic/elasticsearch/pull/95373
Post reply on HN