Generalized K-Means Clustering

#1

Post by derrickrburns » Sat, Jan 13, 2024, 12:56 AM UTC

Generalized K-Means Clustering
github.com

Re: Generalized K-Means Clustering

#2

Post by derrickrburns » Sat, Jan 13, 2024, 12:56 AM UTC

AI has sparked new interest in high dimensional embeddings for approximate nearest neighbor search. Here is a highly scalable, implementation of a companion technique, k-means clustering that uses Spark 1.1 written in Scala.

Please let me know if you fork this library and update it to the latter versions of Spark.

Re: Generalized K-Means Clustering

#3

Post by staticautomatic » Sat, Jan 13, 2024, 1:38 AM UTC

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

Re: Generalized K-Means Clustering

#4

Post by boredemployee » Sat, Jan 13, 2024, 1:40 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

knowledge graphs

Re: Generalized K-Means Clustering

#5

Post by trc001 » Sat, Jan 13, 2024, 1:40 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

Real estate price estimates would be the classic, and frankly still common, example

Re: Generalized K-Means Clustering

#6

Post by JackeJR » Sat, Jan 13, 2024, 1:52 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

SMOTE

Re: Generalized K-Means Clustering

#7

Post by theophrastus » Sat, Jan 13, 2024, 1:53 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

Additional small molecule pharmaceutical candidates via molecular descriptors

Re: Generalized K-Means Clustering

#8

Post by dilyevsky » Sat, Jan 13, 2024, 1:56 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

data segmentation (e.g Voronoi), grouping (like search query results), anomaly detection. lots of different things

Re: Generalized K-Means Clustering

#9

Post by brianleb » Sat, Jan 13, 2024, 1:59 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

So after re-reading your comment a few times, I am left with this thought: either you don't understand what k-means clustering is, or I don't understand what k-means clustering is. I wouldn't describe myself as a machine learning expert, but I have taken some grad level classes in statistics, analytics, and methods like this/related to this.

So my question is... could you elaborate?

Re: Generalized K-Means Clustering

#10

Post by derwiki » Sat, Jan 13, 2024, 1:59 AM UTC

staticautomatic wrote: post #3

What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.

Used it in college to downscale an X color image to Y number of colors. Sure, Photoshop does it, but it was informative to do it manually.