Viewing profile — confuseshrink

Starcraft in the form of Alphastar worked in the sense that it could beat humans, at least in the short term. The problem with the whole technique is that they had to tether it to …

Interesting point. Nvidia have been improving the int performance for quantized inference on their GPUs a lot. It might be a lot of work but could it be possible to scale up this N…

The published hyperparameters are usually ridiculously conservative, for the simple games like breakout and pong you can usually converge in far fewer frames than in the papers.

> We're getting a bit off-topic here, but the #1 target by far in reducing greenhouse emissions is power generation. I readily admit I don't know any of the numbers associated with…

This is a very valid argument but it's hard to know what scaling a transformer will really do without trying (looking at you GPT-3). This is probably an issue for ML in general at …

Yes it's something I often see ignored as "common knowledge" dictates that in ML inference is way cheaper than training. But if you're running a model in production at google with …

The Strubell paper which is the origin of this "5 cars" number isn't even in the right ballpark for this stuff. What they did was take desktop GPU power consumption running the mod…

Yann is a consistently sober voice in this world of AI hype. I find it quite refreshing. Personally I see little evidence that this "just scale a transformer until sentience" hype-…

> So what changed? We aren't sure, but the speculation is that in the process of training, GPT-3 found that the best strategy to correctly predicting the continuation of arithmetic…

I would start with David Silvers (DeepMind) youtube series to get an idea of what's possible or not. Running an already trained reinforcement learning agent is relatively cheap (un…

It depends on how intensive your pre-processing pipeline is. With a really fast accelerator you can quite easily start to be bottlenecked by your CPU.

Vectorization: I'm not an expert in this area so I can only tell you what I've personally found difficult in dealing with vectorization. Usually it all comes down to alignment and …

Bias is just a scalar term that is added. You can learn it via backpropagation like all the other weights.

Since you are right that I have no idea what you are talking about, could you explain what you are talking about?

I find it very surprising that someone would rely on unvalidated mathematical models for this, that goes for the Imperial College people as well as Sweden. Are they even able to fi…

That article doesn't state that 14% of cases were asymptomatic.

I'm not a mathematician but coming from the software world if one guy wrote a massive program (I'm assuming 600 pages is massive) in "an impenetrable, idiosyncratic style" you coul…