A PhD student's perspective on research in NLP in the era of LLMs
1–10 of 55 posts
Re: A PhD student's perspective on research in NLP in the era of LLMs
#2I've been interested in NLP for tagging stories based on topics and themes (detectives, werewolves, murder mystery, etc.), so need accurate disambiguation of parts of speech and ways of detecting uses of metaphore, similies, etc. to describe those. I also want to be able to assess how much of the text is about a given topic, so that if I'm interested in reading a detective story from e.g. the Project Gutenberg collection, I don't want it to pick up a story where a detective is only mentioned in one paragraph.
I've looked at several existing NLP frameworks (Open NLP, Stanford NLP) and none of them are accurate enough -- they fail on things like adjectives and old english second person pronouns. This makes them practically unusable for proper sense diambiguation, lemma and part of speech based rules, etc.
The Open NLP tokenizer is also terrible at tokenizing title abbreviations ("Dr", etc.) and things like the use of "--" to delimit text, which is frequently found various Project Gutenberg texts. You can train the Open NLP tokenizer, but it works on what it has seen, so you need to give it every variation of "(Mr|Mrs|Miss|Ms|Rev|Dr|...). [A-Z]" for it to tokenize those titles; the same for other tokens.
Re: A PhD student's perspective on research in NLP in the era of LLMs
#3I've been interested in NLP for tagging stories based on topics and themes (detectives, werewolves, murder mystery, etc.), so need accurate disambiguation of parts of speech and ways of detecting uses of metaphore, similies, etc. to describe those. I also want to be able to assess how much of the text is about a given topic, so that if I'm interested in reading a detective story from e.g. the Project Gutenberg collec…
at this point the gpt api will do all of that for you! or you can use the embeddings api and build your own systems.
Re: A PhD student's perspective on research in NLP in the era of LLMs
#4I've been interested in NLP for tagging stories based on topics and themes (detectives, werewolves, murder mystery, etc.), so need accurate disambiguation of parts of speech and ways of detecting uses of metaphore, similies, etc. to describe those. I also want to be able to assess how much of the text is about a given topic, so that if I'm interested in reading a detective story from e.g. the Project Gutenberg collec…
at this point the gpt api will do all of that for you! or you can use the embeddings api and build your own systems.
The problem with GPT and other LLMs is that they don't tokenize words at a word or morpheme level, it's just blocks of up to 4 characters, so you get tokens like `!"` instead of two separate tokens. -- That makes it harder to write custom tools on top of, unlike e.g. the output/model of things like the universaldependencies project.
Re: A PhD student's perspective on research in NLP in the era of LLMs
#5Earlier quoted context omitted.
at this point the gpt api will do all of that for you! or you can use the embeddings api and build your own systems.
The problem with GPT and other LLMs is that they don't tokenize words at a word or morpheme level, it's just blocks of up to 4 characters, so you get tokens like `!"` instead of two separate tokens. -- That makes it harder to write custom tools on top of, unlike e.g. the output/model of things like the universaldependencies project.
perhaps a spacy pipeline using gpt and huggingface?
Re: A PhD student's perspective on research in NLP in the era of LLMs
#6well I see topics like NLP in ethics, healthcare, etc, which I think is a sign they don't have much to do here.
Re: A PhD student's perspective on research in NLP in the era of LLMs
#7Earlier quoted context omitted.
at this point the gpt api will do all of that for you! or you can use the embeddings api and build your own systems.
The problem with GPT and other LLMs is that they don't tokenize words at a word or morpheme level, it's just blocks of up to 4 characters, so you get tokens like `!"` instead of two separate tokens. -- That makes it harder to write custom tools on top of, unlike e.g. the output/model of things like the universaldependencies project.
Do you strictly need that level of tokenisation precision to meet your high-level goals?
Re: A PhD student's perspective on research in NLP in the era of LLMs
#8[deleted]
Re: A PhD student's perspective on research in NLP in the era of LLMs
#9Isn't the main problem with NLP research now that you'll need a ton of money to run your experiments? How can an "average" PhD researcher hope to validate their hypothesis if they need several thousand dollars per test?
Re: A PhD student's perspective on research in NLP in the era of LLMs
#10Isn't the main problem with NLP research now that you'll need a ton of money to run your experiments? How can an "average" PhD researcher hope to validate their hypothesis if they need several thousand dollars per test?
Much of the interesting pure research can be done at smaller scale, the larger models are arguably more product engineering than research. At least from a certain perspective.