Deep learning has a size problem • Hacker News Board

Re: Deep learning has a size problem

#41

Post by acollins1331 » Sun, Nov 10, 2019, 2:27 PM UTC

joe_the_user wrote: post #30

blt wrote: post #15

IMO this is not a problem. The people building insanely huge models are expanding the set of tasks that can be done by a computer. Who cares how much memory it takes? Historically, computationally expensive methods eventually become cheap. In the 1980's, researchers had access to Crays to develop physics model, graphics, etc. requiring lots of floating point math and memory. Meanwhile, for the home computers, game pr…

IMO this is not a problem. The people building insanely huge models are expanding the set of tasks that can be done by a computer. Who cares how much memory it takes? But are they? The example in the article describes an incremental improvement in a benchmark in exchange for a massive increasing in training time. Deep learning has achieved success on a number of tasks that previously computers had been unable to do.…

I disagree. There are lots of advancements that DL has yet to fully realize with even the current technology. You're focused on commercial applications but applying neural network models, especially CV models to many types of scientific research has yet to be explored due to lack of funding.

Re: Deep learning has a size problem

#42

Post by chongli » Sun, Nov 10, 2019, 2:49 PM UTC

phoboslab wrote: post #39

chongli wrote: post #38

Earlier quoted context omitted.

So what? Training model is the hardest part, then you just reuse results I doubt anyone is going to want to run a 33GB model on their phone. So what? I can't run weather simulation on my laptop. You only need to run the weather simulation once and then broadcast your forecast to everyone’s devices. You can’t do that with NLP. In order to be useful, NLP models need to run on different input data for every user. With a…

> I doubt anyone is going to want to run a 33GB model on their phone. Why not? Many modern phones have upwards of 512GB of storage. 33 GB for a useful model seems entirely reasonable to me.

That’s for one application. Phones have dozens of apps. If they all use different, giant models like this, then 512GB won’t be nearly enough.

Moreover, what is the performance going to be like? It can’t be too spectacular if your model doesn’t fit in RAM. 33GB is manageable on a beefy server with a ton of RAM. You’re not going to have the same luxury on your phone.

The other major aspect of it is memory bandwidth. If the model was designed to run on a high end GPU, with all 33GB stored in graphics memory, then it’s going to perform terribly if it has to be paged in and out of flash on a phone.

Re: Deep learning has a size problem

#43

Post by pheug » Sun, Nov 10, 2019, 7:12 PM UTC

RocketSyntax wrote: post #22

question_away wrote: post #10

Earlier quoted context omitted.

In what way does it not parallelize well? There are mounds of research in federated learning.

I've read that you can't split up large layers to be trained on separate processors either horizontally (one layer per processor) or vertically (parts of many layers).

On a shared memory system there's little need to do that - there's much more parallelism to be had from accelerating fine grained operations, like matrix multiplications to compute each layer's output.

On a distributed system, splitting up layers between machines to do distributed training is pretty much what Google initially designed Tensorflow for. Generally it scales less well due to the need to communicate massive amounts of data between nodes and much lower network throughput than what GPU/TPU memory provides.

Re: Deep learning has a size problem

#44

Post by bitL » Sun, Nov 10, 2019, 7:38 PM UTC

phreeza wrote: post #37

bitL wrote: post #32

Earlier quoted context omitted.

e.g. XLNet: https://arxiv.org/abs/1906.08237

XLnet is Bert with a bunch of additional training tricks.

BERT is a Transformer with a bunch of additional training tricks. Transformer is self-attention with a bunch of additional training tricks...

Re: Deep learning has a size problem

#45

Post by joe_the_user » Sun, Nov 10, 2019, 8:17 PM UTC

acollins1331 wrote: post #41

joe_the_user wrote: post #30

Earlier quoted context omitted.

IMO this is not a problem. The people building insanely huge models are expanding the set of tasks that can be done by a computer. Who cares how much memory it takes? But are they? The example in the article describes an incremental improvement in a benchmark in exchange for a massive increasing in training time. Deep learning has achieved success on a number of tasks that previously computers had been unable to do.…

I disagree. There are lots of advancements that DL has yet to fully realize with even the current technology. You're focused on commercial applications but applying neural network models, especially CV models to many types of scientific research has yet to be explored due to lack of funding.

I'd like to think I put my comments "as potential problems" since I can't claim to follow everything that's done as deep learning.

Still, to continue the devil's advocate position. Deep learning comes up with a lot of things that are suggestive but not tight enough in their approximation to be useful.

I would guess there are huge number of correlations that seems plausible but aren't really causations. You can apply employ a monster stream of sort of intelligent seeming claims and predictions and find they don't yield any progress in any firm scientific domain. The application of deep learning to finding cancer and related diagnosis processes has been "exciting and promising" for a long time but effectively yielded nothing so far because "quite accurate in highly controlled situations" turns out to seldom be that useful, at least not so far.