Show HN: Ecco – See what your NLP language model is “thinking”

Post by cmrdsprklpny » Fri, Jan 08, 2021, 10:08 PM UTC

airstrike wrote: post #15

jalammar wrote: post #2

Hi HN, Author here. I had been fascinated with Andrej Karpathy's article ( https://karpathy.github.io/2015/05/21/rnn-effectiveness/ ) -- especially where it shows neurons being activated in response to brackets and indentation. I built Ecco to enable examining neurons inside Transformer-based language models. You can use Ecco to simply interact with a language model and see its output token by token(as it's built on…

I just want to say I absolutely love the name and logo. Brings back some fond memories of an incredibly hard game from once upon a time... Having said that, IANAL, but I find it unlikely that the use of a dolphin and the word Ecco together are not trademarked, so you may want to check on that before someone bugs you about it

Post by tchalla » Fri, Jan 08, 2021, 10:54 PM UTC

ninjin wrote: post #7

jalammar wrote: post #2

Hi HN, Author here. I had been fascinated with Andrej Karpathy's article ( https://karpathy.github.io/2015/05/21/rnn-effectiveness/ ) -- especially where it shows neurons being activated in response to brackets and indentation. I built Ecco to enable examining neurons inside Transformer-based language models. You can use Ecco to simply interact with a language model and see its output token by token(as it's built on…

I can not thank you enough for your “The Illustrated Transformer” [1] that I have directed two cohorts of MSc students to – it is a true gem of an article. A few years ago my group made an interface to visualise contextual word representations [2] that looked like a primordial soup ancestor to your most recent article (no screenshots though, sadly). I hope putting these together brings you as much joy as it does to y…

Post by GistNoesis » Sun, Jan 10, 2021, 6:20 PM UTC

mcswell wrote: post #34

GistNoesis wrote: post #16

Interesting. The non-negative matrix factorization on the first level kinda highlight some semantic groupings : paragraph, verbs, auxiliaries, commas, pronouns, nominal propositions. I tried to look at higher level layers, and the grouping were indeed of higher level : for example at level 4 there was a grouping which highlighted for any punctuation (and not just comma). The grouping were also qualifying more : for e…

Linguist here. I give, what's a "nominal preposition"? (I glanced at the web page and at the paper, but neither "nominal" nor "preposition" shows up in either.)