Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

State-of-the-art text classification with universal language models

nlp.fast.ai

Post reply on HN
Print view View original

51–54 of 54 posts

Re: State-of-the-art text classification with universal language models

#51

Post by saulberardo »

Jeremy, after a quick first reading, I believe the following can be improved in the paper:

1. In the introduction, there seem to be contradictory statements about the status of Inductive Transfer for NLP. It is first stated that it "has had a large impact in practice", then in the next paragraph it is stated that "it has been unsuccessful for NLP". How is it possible, having a large impact and at the same time being unsuccessful?

2. In the introduction, it is stated that "Research in NLP focused mostly on transductive transfer". Perhaps this statement were valid back in 2007, but it seems to me outdated. Recently most transfer learning in NLP are related to using pre-trained embeddings in a inductive transfer setting.

3. In the beginning of the "2 Related Work" section, in the excerpt "Features in deep neural networks in CV have been observed to transition from task-specific to general from the first to the last layer", I believe the order "first to the last" should read "last to the first", since the last layers have the more task-specific features and the first layers have the more general.

Re: State-of-the-art text classification with universal language models

#52

Post by Abundnce10 »

ioot17 wrote: post #50
Abundnce10 wrote: post #30

Earlier quoted context omitted.

This method dramatically improves over previous approaches to text classification, and the code and pre-trained models allow anyone to leverage this new approach to better solve problems such as: Finding documents relevant to a legal case; Identifying spam, bots, and offensive comments; Classifying positive and negative reviews of a product; Grouping articles by political orientation; I'm starting a new project where…

CRF works quite well, it's actually what I utilize right now to approach recipe parsing on https://cookalo.com/ . It's based on CRFsuite with Python bindings for data training on already labeled recipes. If you build your own app and want to do some comparison, feel free to run some benchmarks against it.

Very cool! It sounds like you followed a similar approach that the NY Times used in their recipe parsing approach, correct?

How does your API handle ingredients with multiple options (e.g. "1 1/2 cups seedless red or green grapes")?

Re: State-of-the-art text classification with universal language models

#53

Post by ioot17 »

Abundnce10 wrote: post #52
ioot17 wrote: post #50

Earlier quoted context omitted.

CRF works quite well, it's actually what I utilize right now to approach recipe parsing on https://cookalo.com/ . It's based on CRFsuite with Python bindings for data training on already labeled recipes. If you build your own app and want to do some comparison, feel free to run some benchmarks against it.

Very cool! It sounds like you followed a similar approach that the NY Times used in their recipe parsing approach, correct? How does your API handle ingredients with multiple options (e.g. "1 1/2 cups seedless red or green grapes")?

Yes, that's correct, it's similar to the mechanisms NY Times guys were using and I've been focusing on the datasets to feed the CRF with as it's what drives the whole thing. This is the output I've got based on your example: [ { "unit": "cup", "input": "1$1/2 cups seedless red or green grapes", "name": "red grapes", "qty": "1$1/2", "comment": "seedless or green" } ]

Don't hesitate to try the API out by pasting some examples to the white box on the site and pressing the "Try it out!" button, it's interactive :)

Re: State-of-the-art text classification with universal language models

#54

Post by Abundnce10 »

ioot17 wrote: post #53
Abundnce10 wrote: post #52

Earlier quoted context omitted.

Very cool! It sounds like you followed a similar approach that the NY Times used in their recipe parsing approach, correct? How does your API handle ingredients with multiple options (e.g. "1 1/2 cups seedless red or green grapes")?

Yes, that's correct, it's similar to the mechanisms NY Times guys were using and I've been focusing on the datasets to feed the CRF with as it's what drives the whole thing. This is the output I've got based on your example: [ { "unit": "cup", "input": "1$1/2 cups seedless red or green grapes", "name": "red grapes", "qty": "1$1/2", "comment": "seedless or green" } ] Don't hesitate to try the API out by pasting some e…

Don't hesitate to try the API out by pasting some examples to the white box on the site and pressing the "Try it out!" button, it's interactive

Sweet, I didn't realize it was interactive. I'll give it a try!

Post reply on HN