Viewing profile — slashcom

There’s lots of work on distillation, smaller models, approximations, etc. People already have simpler forms of these running on smartphones. Models seem to be growing faster than …

A DGX-2 will do just fine.

Quite a few of my colleagues came back Monday the 27th. I imagine many stay longer since the trip is onerous.

http://nsaphra.github.io/post/hands/ This one perhaps

fp16 saves a lot of memory and is worth doing. I've not had trouble fine tuning all these models with fp16.

Zipf’s law

Links to papers: https://arxiv.org/abs/1811.00207 - Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset https://arxiv.org/abs/1811.01241 - Wizard of Wik…

From the paper: "$50,000 was divided among the human participants based on their performance to incentivize them to play their best. Each player was guaranteed a minimum of $0.40 p…

That should tell you how grad students are paid.

I mean, that’s the thing though. Undergrad in state tuition plus R&B for NCSU and UTexas (programs I got in state tuition for) is at 9k and 10k per year with 5% interest for underg…

Indeed it may somewhat be a result of financial illiteracy, but today's young people are significantly better educated than previous generations ( https://en.wikipedia.org/wiki/Edu…

That is the actual plot of the SheWork episode in Broad City.

Yann LeCun did not, otherwise he’d be a coauthor. As it is, this was a collaboration between NYU and Facebook AI Research, with multiple authors working at both institutions.

I don’t know man, 20% of your annual income would be seen as a sizeable fine. That’s 20% of their yearly profits, and it wipes out most of their earnings for Q1.

And replaced by residual connections in transformers, which are absolutely dominating LSTMs now.

Interestingly, we have 3 in North America: East, West and Texas. https://en.m.wikipedia.org/wiki/Texas_Interconnection

It’s indeed a very high trials to success ratio. Again though, there’s enough papers preceding this one that you could have good confidence in the effort. Another thing that helps …

There’s a natural way to parallelize these models so that using 128 GPUs is the same as a 128x batch size. You can similarly simulate 128x batch size by accumulated gradients befor…

Top of the document says use a ballpoint pen, which doesn't smear and can't be erased like a pencil.

An infinitely sized 2 layer NN is universal in the same way a Turing machine is universal — sure you can write any program; God help you if you try.

So Russia ays they were forced to develop it after Trump withdrew from the nuclear treaty. 2 months is some impressive R&D time, huh?

An easier way to understand it is in the context morphology: word prefixes and suffixes mean things, and words have common roots. For example, polymorphism could be decomposed into…

Picking which items to show you in your newsfeed; making recommendations for places based on your friends; automatically generating blind-friendly captions for uploaded photos (rea…

Buy a dumb terminal. A very cheap, lightweight laptop that can handle a web browser, ssh, and ideally drive an external monitor or two. Then rent time on a cloud service of your ch…