Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noam Shazeer Joins OpenAI

twitter.com

Post reply on HN
Print view View original

241–250 of 478 posts

Re: Noam Shazeer Joins OpenAI

#241

Post by tedd4u »

senordevnyc wrote: post #226
HarHarVeryFunny wrote: post #120

Earlier quoted context omitted.

The architecture was Shazeer's, but the rough idea came from Jakob Uszkoreit who initiated the project. Uszkoreit wanted to build a more efficient/scalable language/seq2seq model that could take advantage of GPU parallelism (replacing RNNs which were the main approach to sequence modelling at that time). Uszkoreit's insight was that although language appears sequential, it is in fact really part parallel part hierarc…

Can you expound on the ablation process? Is that referring to a stripping down of the data or weights or something? Or a stripping down of the transformer architecture structurally? Just curious

You train the model then do a baseline evaluation. Then you evaluate many variants where you have removed or nulled out different layers or chunks of the model. By comparing the performance of those mutated models to the baseline you can learn a lot about the model. What parts don't have much value and can be removed, the location of "functions" or "facts." Etc. Google it.

Re: Noam Shazeer Joins OpenAI

#242

Post by frollogaston »

ignoramous wrote: post #238
frollogaston wrote: post #184

Earlier quoted context omitted.

Referring to this or what? Reddit post is gone, but Yahoo has something. > "I do not believe that humans have an attribute called gender," Shazeer wrote, news site the Information reported Friday. "I do not believe that G-d puts people in the wrong bodies. I do not believe that it is okay to sterilize children. You have the right to your beliefs. I do not share them." It's not dumb, and it's ridiculous if Google real…

> It's not dumb, and it's ridiculous if Google really has a problem with this. Google knows Shazeer's value & paid $2bn to c.ai for it: Its undesirable for anyone (regardless of their seniority) to engage in a discussion without being invited to it. Flaring up discord isn't how someone in a leadership position at a huge company is supposed to operate. It is another thing if they've got the "fuck you" money & a few fe…

I know about the Sergey Brin thing too, same issue but tbh less important of a person

Re: Noam Shazeer Joins OpenAI

#243

Post by p1necone »

wincy wrote: post #178
altmanaltman wrote: post #169

Earlier quoted context omitted.

[flagged]

This is a motte and bailey though. A regular person on the street has never seen a distinction between these two words, and common sense prevailed after years of Silicon Valley policing of speech to try to make an unpopular position seem tenable and widely agreed upon to get the average person to step in line.

Are you seriously trying to claim that e.g. wearing dresses or liking the color pink is somehow fundamentally tied the the genitals in your pants or the chromosomes you have?

The idea that gender is a social concept is so blindingly obvious that, like bbeonx I kind of assume that anyone making comments like yours about "common sense" is either blindly parroting talking points without thinking about them, or arguing in bad faith.

Re: Noam Shazeer Joins OpenAI

#245

Post by abixb »

HarHarVeryFunny wrote: post #120
tmule wrote: post #26

Earlier quoted context omitted.

This understates his criticality. The author list was randomized, but the critical idea was truly his. Wonder what this says about GDM …

The architecture was Shazeer's, but the rough idea came from Jakob Uszkoreit who initiated the project. Uszkoreit wanted to build a more efficient/scalable language/seq2seq model that could take advantage of GPU parallelism (replacing RNNs which were the main approach to sequence modelling at that time). Uszkoreit's insight was that although language appears sequential, it is in fact really part parallel part hierarc…

Curious about others' contributions, such as Vaswani, Parmar, Jones and Gomez, to the paper. What sucks about co-authorship in research papers is that you don't get a clean breakdown of who contributed what to the research paper, and the distribution (in more cases than not) is very much like a pareto distribution.

I'm talking from plenty of group project experience here.

Re: Noam Shazeer Joins OpenAI

#246

Post by VirusNewbie »

objclxt wrote: post #190
electriclove wrote: post #30

Earlier quoted context omitted.

Google and Apple both need a culling similar to what Elon did with Twitter after taking over.

> Google and Apple both need a culling similar to what Elon did with Twitter after taking over. You could cut Google's size by 40% and they'd still have more corporate employees than Apple. (Google has ~190k employees, Apple has ~160k but 50k of those are retail staff, so ~110k corporate)

Google is competing with nvidia (TPU), AWS (GCP), Netflix (youtube), Tesla (waymo self driving), OpenAI (Gemini), Microsoft (Workspace), Apple (Android)....

Re: Noam Shazeer Joins OpenAI

#247

Post by VirusNewbie »

efficax wrote: post #107
karmasimida wrote: post #98

Earlier quoted context omitted.

He is close or already a billionaire, not sure much more money will be do much heavy-lifting

you'd be surprised! people seem to have a limitless appetite for that money stuff. they just can't get enough of it, i've found

How much money do you have? or are you just commenting from the peanut gallery?

Re: Noam Shazeer Joins OpenAI

#248

Post by quantumink »

mikeyouse wrote: post #225
deadbabe wrote: post #222

Earlier quoted context omitted.

Yes, but what about the audacity of it? Get paid a lot to join a company but then decide to get up and leave again 2 years later? He just wants to be passed around?

You can have character and be loyal to Google (lol) or make $xx million… I’m not surprised when people choose the latter.

I would argue its not the millions though, but rather that sweet rare compute - OpenAI has more of it for his interests than anyone - it is understandable why an exceptional mind would prioritize access to greater capabilities above all else

Re: Noam Shazeer Joins OpenAI

#249

Post by dang »

SiempreViernes wrote: post #154
mlmonkey wrote: post #138

For those interested, Wired ran a backstory about the Attention is All You Need paper 2 years ago: https://www.wired.com/story/eight-google-employees-invented-... It gives some context on the contributions of each of the authors. About Shazeer, from the article: Shazeer’s joining the group was critical. “These theoretical or intuitive mechanisms, like self-attention, always require very careful implementation, often…

> Using what team members would later describe with words like “magic” and “alchemy” and “bells and whistles,” Ok, these peopl have all gotten extensive training on how to hype for the non-technical crowd without saying anything of substance.

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

https://news.ycombinator.com/newsguidelines.html

Re: Noam Shazeer Joins OpenAI

#250

Post by projektfu »

SecretDreams wrote: post #179
sandeepkd wrote: post #135

Earlier quoted context omitted.

I think this is a hard question if you ask people to start providing proof for things like this. A lot of such opinions are usually baked into individuals personal experience and perception. Nevertheless one has to feel very strongly to share such a take here in this manner unless they are gaining something from it.

This perspective is only relevant if we assume nobody on HN ever posts maliciously. Some of the circles here are small, incestuous, and probably have some resentment. Other times, there's clear botting - very hard to talk against Elon or his companies without a load of down votes. Needless to say, the OP could be right but they could be right without proof. Or proof would out them. Or it's malicious posting. Don't ta…

I always thought it was people who were married to their book, or just fanboys. Some of these accounts that jump in to defend are quite old.
Post reply on HN