71–80 of 83 posts
There's some definite overfitting apparent in the completion of the bottom half of the cat (blue bkg) photo. Every completion has a funny index card covering the bottom including the "original", while autocomplete should be more robust than recreating the original photo.
Earlier quoted context omitted.
At some point it becomes linear though, there is a base cost to electricity. Google builds datacenters near coal power plants specifically to get (literally) dirt cheap electricity.
Just a correction here: we build datacenters near large sources of renewable / hydro plants (I consider hydro a borderline case). We buy renewable credits for everything and are even trying to timeshift load now to match renewable generation peaks. tl;dr: not coal plants :).
I'm pretty sure we're going to close that plant soon though.
Earlier quoted context omitted.
I'm still learning from deep learning papers and videos before dipping my toes in myself, is there a summary of why PyTorch vs TensorFlow? Does it matter for me?
Nowadays , there isn't a huge practical difference in terms of performance/tooling aside from edge cases and deployment options. It mostly depends on your syntax preference. (although there are flame wars from both sides) I noted the TensorFlow usage because the original GPT-2 release was TensorFlow 1.X, which led to issues when TensorFlow 2.0 was released soon after. For model training, I strongly recommend using th…
Earlier quoted context omitted.
Just a correction here: we build datacenters near large sources of renewable / hydro plants (I consider hydro a borderline case). We buy renewable credits for everything and are even trying to timeshift load now to match renewable generation peaks. tl;dr: not coal plants :).
Is it just a coincidence that the Eemshaven (eu-west4) datacenter is close to a coal plant then? I'm pretty sure we're going to close that plant soon though.
Presumably, since:
"Our data center in Eemshaven was the first to be powered by 100% renewable energy from day one."
https://www.google.com/about/datacenters/locations/eemshaven...
Apparently this takes "2500-V100 days" which is an insane amount of resources for images of this resolution. For context, this is equivalent to 100 $10,000 GPUs running for 25 days, 24/7. https://twitter.com/jm_alexia/status/1273349716915470340?s=2...
... for 64x64 px resolution. This is truly a brute force approach.
Earlier quoted context omitted.
... for 64x64 px resolution. This is truly a brute force approach.
But add an AI superresolution model after that and you're off to the races!
There is only so much about objects and the world you can learn from 64px thumbnails, and they mention that this is probably a reason iGPT wins on the tiny images like CIFAR but loses to the semi-supervised CNNs on ImageNet: because the CNNs are compute-efficient enough that they can train at the standard 224px and see all of the details and structure that disappears at iGPT's 64px, and learn end-to-end.
Earlier quoted context omitted.
Is it just a coincidence that the Eemshaven (eu-west4) datacenter is close to a coal plant then? I'm pretty sure we're going to close that plant soon though.
> Is it just a coincidence that the Eemshaven (eu-west4) datacenter is close to a coal plant then? Presumably, since: "Our data center in Eemshaven was the first to be powered by 100% renewable energy from day one." https://www.google.com/about/datacenters/locations/eemshaven...
I made a video explaining this paper if interested! https://youtu.be/7rFLnQdl22c
Earlier quoted context omitted.
> Is it just a coincidence that the Eemshaven (eu-west4) datacenter is close to a coal plant then? Presumably, since: "Our data center in Eemshaven was the first to be powered by 100% renewable energy from day one." https://www.google.com/about/datacenters/locations/eemshaven...
I appreciate them buying credits, but as nice as those windmills and solar panels are, when the sun doesn't shine, and the wind doesn't blow, what gets burned is coal. Anyway I suppose that's more a policy issue of The Netherlands, I think it's quite shameful we still have coal plants. Those credits hide the real cost of renewable energy.
https://www.blog.google/around-the-globe/google-europe/dutch...
And they seem pretty serious about removing non-renewables altogether.
https://storage.googleapis.com/gweb-sustainability.appspot.c...
In the design field, there’s an adage — constraints inspire creativity. This work seems so unconstrained in its use of computation, that is almost screams to me that they must be going about it the wrong way.
You might take a look at The Bitter Lesson [1], it's referenced by the article and linked around on this thread. > One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning. > T…
The idea that specialization is not as powerful as computation fails the most basic test of a proactive, rather than retroactive, theory. Can you make proactive claims about what works in any given domain? Is the solution to take the hungriest algorithm and apply it? What about feature engineering, cleaning, parameter tuning, analysis, etc.? Is the most power hungry solution still the most effective? In my opinion, part of the reason humans aren’t just giant computation blobs is that we thrive on constraints (physical, sexual, emotional).