11–20 of 85 posts
The speed improvements are certainly interesting, the performance improvements seem decidedly not. This method has more than 2x the parameters of all but one of the models it was compared against. If I’m off-base here can someone explain?
The speed improvements are certainly interesting, the performance improvements seem decidedly not. This method has more than 2x the parameters of all but one of the models it was compared against. If I’m off-base here can someone explain?
I don't care how many parameters my model has per se. What I care about is how expensive it is to train in time and dollars. If this makes it cheaper to train better models despite more parameters, that's still a win.
(Yeah, none of us are anywhere near GPT-3 scale. But I spend most of my time thinking about scaling issues, and it was interesting to see your comment pop up; I would've agreed entirely with it myself, if not for seeing all the anguish caused by attempting to train and deploy billions of parameters.)
I was skeptical of the 83%-whatever top-1 accuracy on Imagenet. But someone pointed out that the accuracy increases when the model is pretrained on JFT, google's proprietary 100-million image dataset, the model's accuracy increases to 87%-whatever. That's pretty interesting. It implies that the original accuracy rating might be legit. The concern is that we're chasing the imagenet benchmark as if it's the holy grail,…
not necessarily, it may be mostly a bonus of the transfer, especially considering that JFT is that much larger - getting for example the first conv layers kernels to converge to Gabor-like takes time, yet those layers kernels are very similar across the well trained image nets (and there were some works showing that it is optimal in a sense, and that it is one of the reasons it is in our visual cortex) and thus transferrable, and can practically be treated as fixed in the new model (especially if those layers were pretrained on very large model and reached the state of generic feature extraction). I suspect the similar is applicable for the low level feature aggregating layers too.
I was skeptical of the 83%-whatever top-1 accuracy on Imagenet. But someone pointed out that the accuracy increases when the model is pretrained on JFT, google's proprietary 100-million image dataset, the model's accuracy increases to 87%-whatever. That's pretty interesting. It implies that the original accuracy rating might be legit. The concern is that we're chasing the imagenet benchmark as if it's the holy grail,…
> 8.7x faster to train This is an achievement but it would be helpful to have put "to train" in the title as this is quite different from efficiency at inference time, which is what often actually matters in deployed applications. From Table 3 on Page 7 it appears to me that NFNet is significantly heavier in the number of parameters than EfficientNet for similar accuracies. For example EffNet-B5 achieves 83.7% with 3…
It has, for larger models (F1 vs B7). See Fig 4 in the Appendix.
I was skeptical of the 83%-whatever top-1 accuracy on Imagenet. But someone pointed out that the accuracy increases when the model is pretrained on JFT, google's proprietary 100-million image dataset, the model's accuracy increases to 87%-whatever. That's pretty interesting. It implies that the original accuracy rating might be legit. The concern is that we're chasing the imagenet benchmark as if it's the holy grail,…
eli5 what SOTA image recognition is?
eli5 what SOTA image recognition is?