Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Universal Sentence Encoder

arxiv.org

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Universal Sentence Encoder

#11

Post by tomku »

igravious wrote: post #6

“We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate performance on diverse transfer tasks. Two variants of the encoding models allow for trade-offs between accuracy and compute resources. For both variants, we investigate and report the relationship between model complexity, resource consumption…

They have an algorithm that takes sentences in textual form and produces a different representation of each sentence that (they claim) is easier for certain language-oriented machine learning tasks to work with. Previous work focused on producing that different representation at the word level, but theirs works on complete sentences.

Re: Universal Sentence Encoder

#12

Post by nl »

Interesting. There's a big need for better vector representations of things in-between words (for which Word2Vec/Glove/FastText work well) and documents (which to me seems impossible. Yes I know about Doc2Vec etc, but really.. it works ok for paragraphs).

Facebook's InferSent[1] has worked reasonably well for me for a variety of sentence level tasks, but I don't have anything I can point to to say that it is really substantially better than averaging word embeddings.

More options is good.

(Also, is Kurzweil part of Google Brain or separate. He doesn't really have nay background in NLP does he?)

[1] https://github.com/facebookresearch/InferSent

Re: Universal Sentence Encoder

#14

Post by eruditepanda »

ynniv wrote: post #5
quizotic wrote: post #4

Earlier quoted context omitted.

404?

Did you miss “internal”?

lol, although I might have to take some blame by putting a link in my comment to begin with.

Note: Keep in mind that some folks publish on Arxiv because it is far easier than going through a traditional publication process. As such, you sometimes get not-as-polished works like this, although they might update the article to fix some of those references.

Re: Universal Sentence Encoder

#15

Post by JustFinishedBSG »

1. This is more Technical Report worthy than paper worthy...

2. "by Ray Kurzweil's Team", although accurate I find that fetishization of certain stars to pretty insulting to the other authors, we already have a convention and it's "Cer et al. (2018)"

Re: Universal Sentence Encoder

#16

Post by irontoby »

igravious wrote: post #6

“We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate performance on diverse transfer tasks. Two variants of the encoding models allow for trade-offs between accuracy and compute resources. For both variants, we investigate and report the relationship between model complexity, resource consumption…

> Awesome. Now what does all that mean in English?

Well, simply put:

  [ccebb 677ce 28f77 86558 2d7cc d67b4 e8f31 8c393 ae867 13593 aa869 3c265],
  [c0021 72510 cee7a 31580 554d3 d49a6 306b9 c1f2c 60c1a 1157c f44c8 31273],
  [682f2 6a4df dc970 3c106 2107c 3dfd5 1506a 6f1b5 af428 829f8 11d06 797dc],
  [d6f84 25e73 76558 6feb0 c67d4 fcc73 b5c8d af4db 2f647 82247 852e7 fc010],
  [f08a8 2ed8f c71bb 12043 5f0f9 190c8 f2ae8 7b30a 4a574 269d0 03be0 a363c],
  [b38c2 10031 37ada 504a8 f2919 3b82b 258fc 5673f c939c a0ef1 46be5 a50d6],
  [93fcd e19f7 0558f e01a6 8beb1 d54b9 9ad20 d6185 adf9b 876a1 a1a94 c9197],
  [92b49 ed290 7a072 fdf1d a61a8 65124 a2025 27153 afa71 a27db 29a2a e5b47],
  [2793f 7171f b18c9 e1945 d31d5 edb66 a1ee0 d9982 e8442 7795d bd4e4 30b41]

Re: Universal Sentence Encoder

#17

Post by PaulHoule »

JustFinishedBSG wrote: post #15

1. This is more Technical Report worthy than paper worthy... 2. "by Ray Kurzweil's Team", although accurate I find that fetishization of certain stars to pretty insulting to the other authors, we already have a convention and it's "Cer et al. (2018)"

At least Ray has the decency to be listed last on the author list!

Personally I think the idea of this paper is pretty good, but the evaluation is weak.

Re: Universal Sentence Encoder

#18

Post by wolfgke »

PaulHoule wrote: post #17
JustFinishedBSG wrote: post #15

1. This is more Technical Report worthy than paper worthy... 2. "by Ray Kurzweil's Team", although accurate I find that fetishization of certain stars to pretty insulting to the other authors, we already have a convention and it's "Cer et al. (2018)"

At least Ray has the decency to be listed last on the author list! Personally I think the idea of this paper is pretty good, but the evaluation is weak.

> At least Ray has the decency to be listed last on the author list!

Just do it like in mathematics: Authors in alphabetical order.

Re: Universal Sentence Encoder

#19

Post by PaulHoule »

wolfgke wrote: post #18
PaulHoule wrote: post #17

Earlier quoted context omitted.

At least Ray has the decency to be listed last on the author list! Personally I think the idea of this paper is pretty good, but the evaluation is weak.

> At least Ray has the decency to be listed last on the author list! Just do it like in mathematics: Authors in alphabetical order.

One senior physicist I worked with advocated alphabetical order whenever he would come first in it!

Re: Universal Sentence Encoder

#20

Post by saas_co_de »

igravious wrote: post #6

“We present models for encoding sentences into embedding vectors that specifically target transfer learning to other NLP tasks. The models are efficient and result in accurate performance on diverse transfer tasks. Two variants of the encoding models allow for trade-offs between accuracy and compute resources. For both variants, we investigate and report the relationship between model complexity, resource consumption…

> Awesome. Now what does all that mean in English?

Singularity any day now

Post reply on HN