Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noam Shazeer Joins OpenAI

twitter.com

Post reply on HN
Print view View original

81–90 of 478 posts

Re: Noam Shazeer Joins OpenAI

#81

Post by khazhoux »

markdown wrote: post #52
tmule wrote: post #26

Earlier quoted context omitted.

This understates his criticality. The author list was randomized, but the critical idea was truly his. Wonder what this says about GDM …

Even more important, I wonder what it says about HBW...

Even if we knew, we’d still fail to understand GHO

Re: Noam Shazeer Joins OpenAI

#82

Post by [deleted] »

kxxx wrote: post #45

https://old.reddit.com/r/singularity/comments/1u8xc9m/most_l... Seems like there are some insights here! edit: it seems the post has been removed but comments are viewable. 1 liner summary: To put it lightly, the dude was politically outspoken and held strong beliefs.

[deleted]

Re: Noam Shazeer Joins OpenAI

#86

Post by diegolas »

rvnx wrote: post #69
tomalbrc wrote: post #66

Earlier quoted context omitted.

[flagged]

[flagged]

talent poaching is something pretty common in tech, google knows something like this can and will happen, so does openAI

also "empty handed" is just unnecesarily dramatic, he left all the knwoledge base he helped build, that's google's IP and is worth m(b?)illions

Re: Noam Shazeer Joins OpenAI

#87

Post by DroneBetter »

rvnx wrote: post #69
tomalbrc wrote: post #66

Earlier quoted context omitted.

[flagged]

[flagged]

I think it becomes somewhat more defensible when considering that the alternative was operatiny Google's policy (before the advent of competition) of "these models would bring unknown dangers in the hands of the public, we shouldn't release them until we better understand the implications" (or perhaps more selfishly "these effectively nullify all our detectors of generated text, if released they would instantly lose us the war on SEO").

(recall that OpenAI thought GPT-2 was too powerful to release for approximately tantamount reasons)

Re: Noam Shazeer Joins OpenAI

#88

Post by flebron »

tmule wrote: post #26
petilon wrote: post #17

Noam Shazeer was one of the lead authors of the seminal paper "Attention Is All You Need", which introduced the transformer architecture. (From Wikipedia)

This understates his criticality. The author list was randomized, but the critical idea was truly his. Wonder what this says about GDM …

Source for this? The notion of attention dates to a content-addressable lookup during sequence alignment (as well as, concurrently, memory lookups in neural Turing machines). Attention had been used in other models, like GRUs and LSTMs with attention. The Vaswani et. al. paper did not introduce attention, just removed everything _but_ attention (and FFW) from the network. Are you claiming the "critical idea" of removing the GRU and LSTM parts and just keeping attention was "truly" Noam's?
Post reply on HN