Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noam Shazeer Joins OpenAI

twitter.com

Post reply on HN
Print view View original

331–340 of 478 posts

Re: Noam Shazeer Joins OpenAI

#331

Post by tgma »

afavour wrote: post #302
electriclove wrote: post #30

Earlier quoted context omitted.

Google and Apple both need a culling similar to what Elon did with Twitter after taking over.

Eh, what has X/Twitter delivered since the cull? It’s basically in maintenance mode. Which is fine if that’s what you want to do, but Google and Apple definitely don’t (and I’m glad for that)

> It’s basically in maintenance mode

Has been in more of a maintenance mode with a multiple of those people. If anything, the pace of the product has improved. Regardless of what you think about Musk, the company he bought was a bloated mess.

Re: Noam Shazeer Joins OpenAI

#332

Post by UltraSane »

electriclove wrote: post #30
thewebguyd wrote: post #23

Earlier quoted context omitted.

That twitter story isn't anything unique to OpenAI or Google, it's just classic "big public corp vs private startup" culture. Once you have to worry about the SEC, shareholders, antitrust, regulations, lawsuits, etc. it's very, very difficult to avoid turning into "big corp" culture. Sama, and any other founder, will always have a difficult fight against bureaucracy, and once you let a little bit in, the bureaucracy'…

Google and Apple both need a culling similar to what Elon did with Twitter after taking over.

"what Elon did with Twitter after taking over."

You mean fire the very smart people who designed the core systems AND insult them so that anyone with options would never want to work there?

Re: Noam Shazeer Joins OpenAI

#333

Post by UltraSane »

frollogaston wrote: post #329
UltraSane wrote: post #327

Earlier quoted context omitted.

> I think it's valid to use the word the way that Israel's present leadership is using it. And how is that? Israel has a population of 10 million people and a very modern military and nuclear weapons. If it's existence was ever truly threatened things would get VERY ugly.

Because that's what matters. The original Zionists aren't alive to ask what they think. Self-proclaimed Zionists are taking the West Bank and Gaza. In fact they've been kinda doing it for decades under previous governments, but more slowly. If there's some other kind of Zionism around, the most it's doing is complaining, and it's been outvoted. I have doubts about their ability to self-defend because otherwise we wou…

Israel actually LEFT Gaza. You seem biased to the point of just plain lying.

Re: Noam Shazeer Joins OpenAI

#334

Post by s1artibartfast »

freejazz wrote: post #175
janalsncm wrote: post #148

Earlier quoted context omitted.

Is this genuinely confusing for people? He helped invent the transformer, he didn’t solve content moderation.

Just from reading the threads here it seems readily apparent that he then went to start this company that did these bad things. Does not seem confusing at all?

What bad things?

Re: Noam Shazeer Joins OpenAI

#335

Post by rgreasons »

gniv wrote: post #10

Wow. What could possibly have caused him to quit so soon after coming back? I hope this is not accurate but I'm afraid it is: https://x.com/signulll/status/2067446889956430273

Thanks for this. I will be thinking about “we can create a permission working group” for a long time.

Re: Noam Shazeer Joins OpenAI

#336

Post by chvid »

SiempreViernes wrote: post #154
mlmonkey wrote: post #138

For those interested, Wired ran a backstory about the Attention is All You Need paper 2 years ago: https://www.wired.com/story/eight-google-employees-invented-... It gives some context on the contributions of each of the authors. About Shazeer, from the article: Shazeer’s joining the group was critical. “These theoretical or intuitive mechanisms, like self-attention, always require very careful implementation, often…

> Using what team members would later describe with words like “magic” and “alchemy” and “bells and whistles,” Ok, these peopl have all gotten extensive training on how to hype for the non-technical crowd without saying anything of substance.

If I use the words alchemy and magic about a piece of code, those are not flattering words.

Re: Noam Shazeer Joins OpenAI

#337

Post by tintor »

tedd4u wrote: post #241
senordevnyc wrote: post #226

Earlier quoted context omitted.

Can you expound on the ablation process? Is that referring to a stripping down of the data or weights or something? Or a stripping down of the transformer architecture structurally? Just curious

You train the model then do a baseline evaluation. Then you evaluate many variants where you have removed or nulled out different layers or chunks of the model. By comparing the performance of those mutated models to the baseline you can learn a lot about the model. What parts don't have much value and can be removed, the location of "functions" or "facts." Etc. Google it.

How come they didn’t ablate encoder? OpenAI GOT models are decoder only.

Re: Noam Shazeer Joins OpenAI

#338

Post by nrds »

dang wrote: post #299
nrds wrote: post #259

Earlier quoted context omitted.

[flagged]

It's not a question of painful - I'm happy to "admit" what's true, as best I can, and not what's not true. Let's see if we can sort that out a bit in the present case. HN is certainly curated - I've been "admitting" that since the day I got outed as a mod here: https://news.ycombinator.com/item?id=7494621 (March 2014) https://news.ycombinator.com/item?id=7507229 (April 2014) https://news.ycombinator.com/item?id=79629…

> snark and indignation

These are preference-based but you're pretending they're objective. I find _your_ comments to be full of snark and indignation more than any you respond to, but of course you won't agree. (But because you don't agree, that makes me objectively wrong, I know.)

"Tonal arguments are ways of, frankly, policing working class ways of communication, and covering them in elite preferences." - someone smarter than the average HN commenter.

Re: Noam Shazeer Joins OpenAI

#339

Post by TiredOfLife »

afavour wrote: post #302
electriclove wrote: post #30

Earlier quoted context omitted.

Google and Apple both need a culling similar to what Elon did with Twitter after taking over.

Eh, what has X/Twitter delivered since the cull? It’s basically in maintenance mode. Which is fine if that’s what you want to do, but Google and Apple definitely don’t (and I’m glad for that)

Working recommended feed

Re: Noam Shazeer Joins OpenAI

#340

Post by adonese »

I'm curious to know the hype behind the hiring for Karpathy and Noam. In the sense that did oai and anthropic do that for sort of long term and potential new directions (investing in them so they come up with the new transformer). Because it definitely cannot be just a regular filling vacant roles.

Because I think as far as running the existing models and handling whatever nuances, it must be well understood by oai and ANT -- but you don't what you don't know.

Post reply on HN