Live data from Hacker News

Noam Shazeer Joins OpenAI

twitter.com

221–230 of 478 posts

Re: Noam Shazeer Joins OpenAI

#221
post #53
post #45

https://old.reddit.com/r/singularity/comments/1u8xc9m/most_l... Seems like there are some insights here! edit: it seems the post has been removed but comments are viewable. 1 liner summary: To put it lightly, the dude was politically outspoken and held strong beliefs.

[flagged]

It is possible to rationalize all sorts of irrational ideas. It's a trap many fall into.

Re: Noam Shazeer Joins OpenAI

#222
post #189

Earlier quoted context omitted.

How can an acquired dude leave after less than 2 years?

OpenAI pays for the earn out he would’ve otherwise received at Google + a new comp package. Made up numbers, if Google still owed him $10M for lasting the full two years, OpenAI can just pay him market rate +$10M.

Yes, but what about the audacity of it? Get paid a lot to join a company but then decide to get up and leave again 2 years later? He just wants to be passed around?

Re: Noam Shazeer Joins OpenAI

#223

Earlier quoted context omitted.

[flagged]

[flagged]

There are both descriptive and normative uses of gender. To use a less charged example, it's not prescriptive to identify as American. It's not prescriptive to say other people identify you that way, even if your passport says Canadian.

An example of using the category normatively would be saying someone isn't American because they burn the flag. My experience is that most of the people using "gender" normatively don't differentiate it from sex.

Re: Noam Shazeer Joins OpenAI

#224

Earlier quoted context omitted.

I loved that show. The love that went into it really shows. Sadly the gap between reality and satire has shrunk. But yes. I also wish that show would come back. Noam shazeer would be google head dreamer

The gap between reality and satire was apparently already very small back when the the show was written. The creator, Mike Judge (who also created Beavis & Butthead, and Idiocracy) had worked in Silicon Valley as a developer and based the show on what he saw. Apparently it was very popular with SV insiders precisely because it was so accurate.

Judge also consulted with various teams at places like Google; I worked with one of the guys who provided details that later showed up on the show (as well as many plushies). He didn't watch the show because "it hit too close to home"

Re: Noam Shazeer Joins OpenAI

#225

Earlier quoted context omitted.

OpenAI pays for the earn out he would’ve otherwise received at Google + a new comp package. Made up numbers, if Google still owed him $10M for lasting the full two years, OpenAI can just pay him market rate +$10M.

Yes, but what about the audacity of it? Get paid a lot to join a company but then decide to get up and leave again 2 years later? He just wants to be passed around?

You can have character and be loyal to Google (lol) or make $xx million… I’m not surprised when people choose the latter.

Re: Noam Shazeer Joins OpenAI

#226
post #26

Earlier quoted context omitted.

This understates his criticality. The author list was randomized, but the critical idea was truly his. Wonder what this says about GDM …

The architecture was Shazeer's, but the rough idea came from Jakob Uszkoreit who initiated the project. Uszkoreit wanted to build a more efficient/scalable language/seq2seq model that could take advantage of GPU parallelism (replacing RNNs which were the main approach to sequence modelling at that time). Uszkoreit's insight was that although language appears sequential, it is in fact really part parallel part hierarc…

Can you expound on the ablation process? Is that referring to a stripping down of the data or weights or something? Or a stripping down of the transformer architecture structurally? Just curious

Re: Noam Shazeer Joins OpenAI

#227
post #53

Earlier quoted context omitted.

[flagged]

Referring to this or what? Reddit post is gone, but Yahoo has something. > "I do not believe that humans have an attribute called gender," Shazeer wrote, news site the Information reported Friday. "I do not believe that G-d puts people in the wrong bodies. I do not believe that it is okay to sterilize children. You have the right to your beliefs. I do not share them." It's not dumb, and it's ridiculous if Google real…

Accusing coworkers of being antisemitic crosses the line, but accusing coworkers of sterilizing children and denying the existence of gender is ok? Surely both are bad, neither is acceptable in a workplace. Do you mean it’s not dumb because you share his views?

Re: Noam Shazeer Joins OpenAI

#229
post #47

Earlier quoted context omitted.

I think the 'models have no moat' thing is overblown. Only like 3-4 companies in the entire world have cutting edge models, that means there is some kind of moat...

Money. That's their moat. Maybe also stolen copyrighted content that cannot be found anywhere else now, so they are the only ones who can train on it.

Meta has tons of that, but no frontier contender. Clearly there’s _something_ more to the equation than money
Post reply on HN