BERTopic: The Future of Topic Modeling
pinecone.io
BERTopic: The Future of Topic Modeling
1–10 of 29 posts
Re: BERTopic: The Future of Topic Modeling
#2It is true that bertopic is a great tool. It's modern, it's modular, and it's pretty performant.
That said, I want to caution against using topic modeling as a one-fits-all-solution. As the author stresses, this is one particular approach which uses a combination of embeddings (sentence, or other), umap and hdbscan. Both umap and hdbscan can be slow, so it might be worthwhile to check out the GPU enabled versions of both from the cuml package.
In addition, topic models have a huge number of degrees of freedom, and the solution you will get depends on many (seemingly arbitrary) choices. In other words, these are not the topics, they are some topics.
That said, it's awesome, really great work by Maarten Grootendorst and a great blog post by James Briggs.
[edit] here is a link to the fast cuda version of bertopic by rapidsai: https://github.com/rapidsai/rapids-examples/tree/main/cuBERT...
Re: BERTopic: The Future of Topic Modeling
#3take a look at Graphext ( https://www.graphext.com ) it automatically creates the clustering embeddings using BERT for you + great visualization libraries to interpret the clusters :D it took us 5 years to build the product
Re: BERTopic: The Future of Topic Modeling
#4A huggingface space I wrote to let you play with BERTopic in your browser:
Re: BERTopic: The Future of Topic Modeling
#5What happens on a slightly different task where domain experts have tried to create a set of topics, not all domain experts talk to each other, and so we instead need a way to merge existing topics? I continue to see benchmarks where human expertise significantly outperforms AI on common sense reasoning tasks (most recently https://arxiv.org/abs/2112.11446).
What about an approach using directed acyclic graphs and entities?
Re: BERTopic: The Future of Topic Modeling
#6How does this compare to LDA? It doesn’t seem like there’s a huge difference here. For good reason perhaps, the BERT part is only to embed the sentences.
Re: BERTopic: The Future of Topic Modeling
#7It is true that bertopic is a great tool. It's modern, it's modular, and it's pretty performant. That said, I want to caution against using topic modeling as a one-fits-all-solution. As the author stresses, this is one particular approach which uses a combination of embeddings (sentence, or other), umap and hdbscan. Both umap and hdbscan can be slow, so it might be worthwhile to check out the GPU enabled versions of…
Its seamless to accelerate BERTOPIC on GPU's with cuML now with the latest release. (v0.10.0)
Checkout the docs at: https://maartengr.github.io/BERTopic/faq.html#can-i-use-the-...
All you need to do is below
from bertopic import BERTopic
from cuml.cluster import HDBSCAN
from cuml.manifold import UMAP
# Create instances of GPU-accelerated UMAP and HDBSCAN
umap_model = UMAP(n_components=5, n_neighbors=15, min_dist=0.0)
hdbscan_model = HDBSCAN(min_samples=10, gen_min_span_tree=True)
# Pass the above models to be used in BERTopic
topic_model = BERTopic(umap_model=umap_model, hdbscan_model=hdbscan_model)
topics, probs = topic_model.fit_transform(docs)Re: BERTopic: The Future of Topic Modeling
#8How does this compare to LDA? It doesn’t seem like there’s a huge difference here. For good reason perhaps, the BERT part is only to embed the sentences.
yeah exactly my question. LDA is probabilistic and very performant if you clean up the documents well. The approach using Bert seems pretty powerful given that you can now cluster based on semantics, not just word occurrence/frequencies as in LDA (though ngrams help). However using a clustering approach would mean that each document is a part of a single topic, rather than being made up of multiple topics. But this is a cool idea nonetheless.
[EDIT] quickly checked it out, seems like it uses some kind of soft clustering so documents can occur in many clusters (topics)
Re: BERTopic: The Future of Topic Modeling
#9take a look at Graphext ( https://www.graphext.com ) it automatically creates the clustering embeddings using BERT for you + great visualization libraries to interpret the clusters :D it took us 5 years to build the product
It sure has to be much, much better than free. Especially if your pricing is 'contact sales'.
Re: BERTopic: The Future of Topic Modeling
#10How does this compare to LDA? It doesn’t seem like there’s a huge difference here. For good reason perhaps, the BERT part is only to embed the sentences.
yeah exactly my question. LDA is probabilistic and very performant if you clean up the documents well. The approach using Bert seems pretty powerful given that you can now cluster based on semantics, not just word occurrence/frequencies as in LDA (though ngrams help). However using a clustering approach would mean that each document is a part of a single topic, rather than being made up of multiple topics. But this i…
would it make sense to preprocess with a transformer style model to produce per document semantic vectors which can then be piped into LDA to find topic mixtures of those vectors?