Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building Text Classifiers: A New Approach with Less Data

mazaal.ai

Post reply on HN
Print view View original

11–14 of 14 posts

Re: Building Text Classifiers: A New Approach with Less Data

#11

Post by janalsncm »

What model specifically are you training or fine-tuning? This article criticizes OpenAI for providing a black box only, but I don’t see any details about your model either.

Can I download the model I’ve trained? If not, we can ignore the pricing comparison since there’s no guarantee it will be the same tomorrow.

Second, as far as I can tell your metrics are comparing zero shot GPT3.5 performance with your fine-tuned model performance. If you want a fair comparison you need to compare with the fine-tuned GPT3.5 performance.

Re: Building Text Classifiers: A New Approach with Less Data

#12

Post by bugglebeetle »

tannhaeuser wrote: post #9

What's the consensus of building state of the art classifiers? Is using Llama2, Mistral, and co. really better than BERT? Fine tuning vs prompt engineering (which is what I understand the article to be about)?Text classification can also range from mere sentiment analysis, to genre/form classification, to content classification, and there are conflicting results about the relative merits of LLMs for genre vs content…

GPT-4 with JSON function calling is about on par with BERT and company for simple classification problems, but is slower and more expensive. For anything more complicated than that, you’re better off using standard ML models.

One way they can be used to build on each other, however, is synthetic training data generation from LLMs to train other kinds of models. In my experience, this works best when you already have a fairly robust dataset for it to create permutations from.

Post reply on HN