Next step: automatically naming clusters using few-shot GPT-3. Cluster naming is a non-trivial problem.
BERTopic: The Future of Topic Modeling
11–20 of 29 posts
Re: BERTopic: The Future of Topic Modeling
#12Next step: automatically naming clusters using few-shot GPT-3. Cluster naming is a non-trivial problem.
Absolutely. This is the point where I think we shift from astronomy to astrology!
Re: BERTopic: The Future of Topic Modeling
#13Next step: automatically naming clusters using few-shot GPT-3. Cluster naming is a non-trivial problem.
I've run into this problem at a previous employer. Do you know if anyone's working on it?
Re: BERTopic: The Future of Topic Modeling
#14Earlier quoted context omitted.
yeah exactly my question. LDA is probabilistic and very performant if you clean up the documents well. The approach using Bert seems pretty powerful given that you can now cluster based on semantics, not just word occurrence/frequencies as in LDA (though ngrams help). However using a clustering approach would mean that each document is a part of a single topic, rather than being made up of multiple topics. But this i…
would it make sense to preprocess with a transformer style model to produce per document semantic vectors which can then be piped into LDA to find topic mixtures of those vectors?
Is that not exactly what's happening in TFA?
Re: BERTopic: The Future of Topic Modeling
#15A huggingface space I wrote to let you play with BERTopic in your browser: https://huggingface.co/spaces/Hellisotherpeople/HF-BERTopic
Wow. This is great!
Re: BERTopic: The Future of Topic Modeling
#16Earlier quoted context omitted.
would it make sense to preprocess with a transformer style model to produce per document semantic vectors which can then be piped into LDA to find topic mixtures of those vectors?
Is that not exactly what's happening in TFA?
if TFA means "the forememtioned article", i don't think so. i'm not convinced that the clusters found and the frequencies in those clusters would be the same as what LDA computes with gibbs sampling or the variational calculus method would find. but i must admit it's been a while since i've played with this stuff.
if TFA is some other method, i am unfamiliar and would like to know more.
in my experience, while it's true that it's hard to score and verify these sorts of models, the hierarchical multinomial nature of LDA topic models makes it easy to generate data and then verify behavior in the fitting process by recovering generative model parameters used by the test data generation process. obviously this makes no sense for the bert frontend, but a comparison of the differing backend clustering methods could be interesting.
Re: BERTopic: The Future of Topic Modeling
#17take a look at Graphext ( https://www.graphext.com ) it automatically creates the clustering embeddings using BERT for you + great visualization libraries to interpret the clusters :D it took us 5 years to build the product
I'm not going to contact sales. I have no problem paying though.
Part of your value proposition is saving people time, but your sales model is time expensive.
Re: BERTopic: The Future of Topic Modeling
#18Next step: automatically naming clusters using few-shot GPT-3. Cluster naming is a non-trivial problem.
I've run into this problem at a previous employer. Do you know if anyone's working on it?
There is a recent discussion in r/machinelearning about it
https://www.reddit.com/r/MachineLearning/comments/umgdts/p_c...
Re: BERTopic: The Future of Topic Modeling
#19It is true that bertopic is a great tool. It's modern, it's modular, and it's pretty performant. That said, I want to caution against using topic modeling as a one-fits-all-solution. As the author stresses, this is one particular approach which uses a combination of embeddings (sentence, or other), umap and hdbscan. Both umap and hdbscan can be slow, so it might be worthwhile to check out the GPU enabled versions of…
I agree that this is a cool. That being said, the results show that we have a long, long way to go. The topics are pretty incoherent: what are "would", "should", and "use" doing in there? The words have no context, so (for example) "self" clearly refers to the python reserved keyword, but you have no way of knowing that. Not to mention (as another comment brings up), the topics aren't named so it's pretty hard to actually figure out what they're about. If we think about real-world usage this output would be practically useless - it tells you that people talk about investing in r/investing and pytorch in r/pytorch. If you want meaningful, actionable information about what people are talking about in a large corpus of unstructured text data then for the forseeable future you'll need to involve humans in the loop even if ML assistance plays a big part.
Re: BERTopic: The Future of Topic Modeling
#20What happens on a slightly different task where domain experts have tried to create a set of topics, not all domain experts talk to each other, and so we instead need a way to merge existing topics? I continue to see benchmarks where human expertise significantly outperforms AI on common sense reasoning tasks (most recently https://arxiv.org/abs/2112.11446 ). What about an approach using directed acyclic graphs and e…
In traditional qualitative research, you'd usually have a bunch of experts get together and figure out a set of topics (or import and adapt a set of topics from similar work) before you go about classifying the bulk of your data.