Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NLP Research in the Era of LLMs

nlpnewsletter.substack.com

Post reply on HN
Print view View original

11–18 of 18 posts

Re: NLP Research in the Era of LLMs

#11

Post by anon-feaata2311 »

anon-feaata2311 wrote: post #10
owlninja wrote: post #2

I'm still in the stone ages trying to use topic modeling on a free text field. Can LLMs help me decipher a large dataset of human text and put it in buckets?

Have a look at Using text embeddings and approximate nearest neighbour search to explore ... https://medium.com/gft-engineering/using-text-embeddings-and... You can reduce sentences to vectors and then create similarity scores to build a graph over the corpus. If you choose to create clusters then use a llm to summarise them to create labels.

Interestingly I don't see much progress on the core of vector DBs... Does anyone have so innovative references?

Re: NLP Research in the Era of LLMs

#12

Post by JacobiX »

I'm still using old-school maximum-entropy models and HMMs for NLP and integrating them into our products. The model size is less than 1MB, and the inference time is really negligible. These models are used in batch processing for billions of entries and with no access to internet. Not sure if LLMs can fit this task?

Re: NLP Research in the Era of LLMs

#13

Post by janalsncm »

JacobiX wrote: post #12

I'm still using old-school maximum-entropy models and HMMs for NLP and integrating them into our products. The model size is less than 1MB, and the inference time is really negligible. These models are used in batch processing for billions of entries and with no access to internet. Not sure if LLMs can fit this task?

If inference time is negligible, you might find that a larger model with slightly better metrics is affordable. Not that an LLM is the way to go, but perhaps if you’re already in for the squeeze maybe there’s more juice to be had.

Re: NLP Research in the Era of LLMs

#14

Post by bravura »

soultrees wrote: post #9
owlninja wrote: post #2

I'm still in the stone ages trying to use topic modeling on a free text field. Can LLMs help me decipher a large dataset of human text and put it in buckets?

I just did this today using openAI’s function calling. I have a bunch of elements in a scene and trying to classify them between various ‘buckets’ has been the challenge. The way I set it up is that in the schema expected, you take that top level free text, wrap it in quotes and it becomes the parent object with the elementCategory as a string required inside it, with a list of all category types in the description.…

Try this: https://github.com/refuel-ai/autolabel

Then the main challenge just becomes prompt design, which can sometimes be nebulous for NLP annotation.

Re: NLP Research in the Era of LLMs

#15

Post by YetAnotherNick »

sunpazed wrote: post #6

> While LLM projects typically require an exorbitant amount of resources, it is important to remind ourselves that research does not need to assemble full-fledged massively expensive systems in order to have impact. Check out TinyLlama; https://github.com/jzhang38/TinyLlama Four research students from the Singapore University of Technology and Design are pretraining a 1.1B Llama model on 3 trillion tokens using a han…

Even if they ran it without facing any issues and 0 testing, it would have taken 35k A100 hours or $70k-100k. It is not cheap to do it.

Re: NLP Research in the Era of LLMs

#16

Post by kolinko »

owlninja wrote: post #2

I'm still in the stone ages trying to use topic modeling on a free text field. Can LLMs help me decipher a large dataset of human text and put it in buckets?

With no problem. The easiest way is to go with GPT4 api for a start (you just tell it how you want it organised / what metadata to extract) - it will be costly, but you can prove that it works, and generate examples to fine tune a simpler model to do the same.

Re: NLP Research in the Era of LLMs

#17

Post by sunpazed »

YetAnotherNick wrote: post #15
sunpazed wrote: post #6

> While LLM projects typically require an exorbitant amount of resources, it is important to remind ourselves that research does not need to assemble full-fledged massively expensive systems in order to have impact. Check out TinyLlama; https://github.com/jzhang38/TinyLlama Four research students from the Singapore University of Technology and Design are pretraining a 1.1B Llama model on 3 trillion tokens using a han…

Even if they ran it without facing any issues and 0 testing, it would have taken 35k A100 hours or $70k-100k. It is not cheap to do it.

I’d agree — but would argue affordable for a sponsored dissertation program with 3 research students and an associate professor. They’re actually still training it!

Re: NLP Research in the Era of LLMs

#18

Post by YetAnotherNick »

sunpazed wrote: post #17
YetAnotherNick wrote: post #15

Earlier quoted context omitted.

Even if they ran it without facing any issues and 0 testing, it would have taken 35k A100 hours or $70k-100k. It is not cheap to do it.

I’d agree — but would argue affordable for a sponsored dissertation program with 3 research students and an associate professor. They’re actually still training it!

For one run, yes. But if they are testing new architecture or something like that, they need at least dozens of them. If they are not testing new architecture, finetuning is almost always the way to go.
Post reply on HN