What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
Constantly in remote sensing and GIS work.
Generalized K-Means Clustering
11–20 of 85 posts
Re: Generalized K-Means Clustering
#12What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
So after re-reading your comment a few times, I am left with this thought: either you don't understand what k-means clustering is, or I don't understand what k-means clustering is. I wouldn't describe myself as a machine learning expert, but I have taken some grad level classes in statistics, analytics, and methods like this/related to this. So my question is... could you elaborate?
You have to choose the number of clusters, before using k-means.
Imagine that you have a dataset, where you think there are likely meaningful clusters, but you don't know how many, especially where it's many-dimensioned.
If you pick a k that is too small, you lump unrelated points together.
If k is too large, your meaningful clusters will be fragmented/overfitted.
There are some algorithms that try to estimate the number of clusters or try to find the k with the best fit to the data to make up for this.
Re: Generalized K-Means Clustering
#13What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
Measuring multiple physical objects with the same sensor, you can use k-means to separate the measurements from each object, given that you know how many objects are being sensed. I can't get more specific than that.
Re: Generalized K-Means Clustering
#14What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
So after re-reading your comment a few times, I am left with this thought: either you don't understand what k-means clustering is, or I don't understand what k-means clustering is. I wouldn't describe myself as a machine learning expert, but I have taken some grad level classes in statistics, analytics, and methods like this/related to this. So my question is... could you elaborate?
Not GP, but I understood their question as follows.
Assume you collect some kindergartners and top NBA players into a room and collect their heights. Now say you pass these to two hapless grad students and ask them to perform K-means clustering.
Suppose one of the grad students knew the composition of the people you measured and can guess these height should clump into 2 nice clusters. The other student who doesn't know the composition of the class - what should they guess K to be?
I understood the GP's comment to refer to the state of the second grad student. How useful is K-means clustering without knowing K in advance?
Re: Generalized K-Means Clustering
#15What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
Sometimes you can use a heuristic to estimate K, or use a variant that terminates at some distance threshold.
That said, something like hdbscan doesn’t suffer from this problem.
Re: Generalized K-Means Clustering
#16What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
k-means is good for fast unsupervised clustering on an unknown low-dimensional dataset. It's helpful for EDA.
If you want accuracy at an order of magnitude more compute, you can use something like DBSCAN.
Re: Generalized K-Means Clustering
#17What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
Used it in college to downscale an X color image to Y number of colors. Sure, Photoshop does it, but it was informative to do it manually.
Google's Material You uses this to initiate color theming
(n.b. Celebi's, note usage of Lab / notHSL, respect Cartesian / polar nature of inputs / outputs, and ask for high-K, 128 is what we went with but it's arbitrary. Can get away with as few as 32 if you're ex. Doing brand color from favicon)
Re: Generalized K-Means Clustering
#18Earlier quoted context omitted.
So after re-reading your comment a few times, I am left with this thought: either you don't understand what k-means clustering is, or I don't understand what k-means clustering is. I wouldn't describe myself as a machine learning expert, but I have taken some grad level classes in statistics, analytics, and methods like this/related to this. So my question is... could you elaborate?
You have to choose the number of clusters, before using k-means. Imagine that you have a dataset, where you think there are likely meaningful clusters, but you don't know how many, especially where it's many-dimensioned. If you pick a k that is too small, you lump unrelated points together. If k is too large, your meaningful clusters will be fragmented/overfitted. There are some algorithms that try to estimate the nu…
Couldn’t you make some educated guesses and then stop when you arrive at a K that gives you meaningful clusters that are neither too high level nor too atomized.
Re: Generalized K-Means Clustering
#19What are people using k-means for? I can count on one hand the number of times I’ve had a good a priori rationale for the value of k.
Baseball pitch types based on physics profiles.
Re: Generalized K-Means Clustering
#20I built a pipeline to automatically cluster and visualize large amounts of text documents in a completely unsupervised manner:
- Embed all the text documents.
- Project to 2D using UMAP which also creates its own emergent "clusters".
- Use k-means clustering with a high cluster count depending on dataset size.
- Feed the ChatGPT API ~10 examples from each cluster and ask it to provide a concise label for the cluster.
- Bonus: Use DBSCAN to identify arbitrary subclusters within each cluster.
It is extremely effective and I have a theoetical implementation of a more practical use case to use said UMAP dimensionality reduction for better inference. There is evidence that current popular text embedding models (e.g. OpenAI ada, which outputs 1536D embeddings) are way too big for most use cases and could be giving poorly specified results for embedding similarity as a result, in addition to higher costs for the entire pipeline.