Live data from Hacker News

Turing-NLG: A 17B-parameter language model

microsoft.com

121–130 of 149 posts

Re: Turing-NLG: A 17B-parameter language model

#121
post #11

One of the team members from Project Turing. Happy to answer any questions.

How close do you think the technology is to answering -this- question?

1) How close do you think the technology is to answering -this- question?

Four days!

2) How long in years?

Three years!

Re: Turing-NLG: A 17B-parameter language model

#122
I've always been interested in techniques to try to minimize parameters or alternate approaches to learning. Meanwhile, state of the art is over here just finding clever ways to make everything bigger. I have a feeling we're going to end up with a very different landscape in 5-10 years, much like the automotive industry never started mass producing inline 12s and instead moved to turbos and superchargers.

Re: Turing-NLG: A 17B-parameter language model

#125
post #45
post #18

Earlier quoted context omitted.

I think this is largely unnecessary, can't things like TPUs handle the inference?

Putting all your speech/text onto cloud machines runs counter to e2e encrypted messaging.

I think you can get hardware like a TPU for consumer products?

Re: Turing-NLG: A 17B-parameter language model

#126

Earlier quoted context omitted.

> So, are reasonable examples now of these models allowing semantic context? This is about where I am stuck. I'll start believing that we truly are on the cusp of a revolution as soon as I see Google Translate reliably knowing when to translate "home" into French as "domicile", "foyer", something those lines, or as "accueil." Right now it seems to very frequently choose "accueil", which is generally wrong, except whe…

Syntax and semantics were developed for human language, yet it's much easier to puzzle the difference in a computer language than in human language. With syntax and semantics so wrapped together, however, it kind of seems like you can go a long way with just capturing syntax, rhythm, word choice and etc. Which is to say the semantic side can be even worse than it seems, ie, nonexistent.

A long way for what though? The unicorn story is neat, but I don’t have a giant unmet demand for rambling.

Re: Turing-NLG: A 17B-parameter language model

#127
post #37

Earlier quoted context omitted.

How does it compare to Google’s BERT and do you have an online demo? Here’s a demo of BERT https://www.pragnakalp.com/demos/BERT-NLP-QnA-Demo/

(Similar to the response for another question.) BERT is a language representation model while Turing-NLG is a language generation model (similar to GPT). They are not directly comparable (they can potentially be massaged to mimic the other, but, not something that we have done yet.)

Google's T5 paper pretty convincingly combines the two doesn't it?

Re: Turing-NLG: A 17B-parameter language model

#130
post #129
post #118

Earlier quoted context omitted.

Yes.

While Markov chains sound like a schizophrenic, this sounds like a banker on coke. Pretty impressive progress, i guess.

That's because of the word “bearish”. It can sound like just about anything. For example, prompted by your comment, it sounds like a human interest journalist. (Journalism and fanfic, in particular HPMOR, seem to have comprised a large part of its training corpus; it composes Harry Potter porn at the slightest provocation; “Hermione moaned”, say.)

While Markov chains sound like a schizophrenic, this sounds like a spacial disjointed notebook, as if somebody was trying to write in two places at the same time. "Today is Monday, it's Saturday night, I forgot to write to my dad and he only leaves the house for a couple of hours."

Yet, sometimes Markov chains turn out to be the most beautiful art form. His wife Jennifer Neil, whom he met at a barbecue and has been married to since 1998, attributes this creative process to the constant ups and downs in his old job.

"His numbers are just insane," says Jennifer Neil. "I don't know where he keeps them, but they're very mind boggling."

Somewhat disconnected from the actual

Post reply on HN