Viewing profile — jeeceebees

You can just mask the output probabilities for each token based on which options are valid according to a grammar. There are quite a few open source implementations of this e.g. ht…

I think this is a property spheres. It seems to me that any two spheres that are touching have a straight line from one center to the other center exactly through the point of cont…

How does the performance between GPU programs written with std::par compare to those written in CUDA? Do you happen to know of any online resources that show a comparison of the ke…

There is a lot of evidence that these token-based models work with multi-modal data. In fact, several groups have proposed different multi-modal transformer architectures already (…

Having more memory to address means you need more circuits that direct your reads/writes to the right place. Travelling a longer distance / through more complicated routes means th…

The hidden state isn't storing the samples of your loop (or a compressed version of your loop). It's encoding a representation of how the output will change based on what the curre…

LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state siz…

I think most human creativity is built around a seed of inspiration from outside sources. In my experience serendipity and happy accidents are exactly what leads to the most creati…

I think the larger models get, the more incentive there is for researchers to look into pruning/distilling them for practical use. GPT-1,2,3 et al. have all shown that larger is be…

It's YYMM.ID I believe Last year may was 1905.xxxxxx

Have all novel qubits gotten nobel prizes so far? I think there's more than enough room in between bullshit and nobel prize. This looks very promising, but as always the devil is i…

I don't really understand why I'd use gin. From the example ipynbs it looks like pretty much the same amount of code but in a gin file and then it spookily fills in parameters for …

As far as I understand, it's because what it's simulating is a logical qubit which is different from the very noisy, almost instantaneously collapsing physical qubits present in cu…

Of course there is. All the building blocks that people are mix and matching in networks nowadays were introduced at some point. The paper that introduced batch norm, adaptive inst…

I think the approach is really cool but the processing time required is too much for this to be very useful at the moment. On a 1080 Ti it takes 45-90 minutes to train networks for…

I had already completed this course before Stefan started his conquest of the CS bachelors courses, however, I did have some other courses that he taught. I think he has a great kn…

Detect when there are multiple apps that are asking for new permissions simultaneously and then ask the user if they want to go to the "bulk permission editing page" to manage them…

This is the same idea that underlies style transfers and metrics like the FID (which is used to judge generative networks' outputs on their similarity to the test set). The idea is…

I think your camera example is a false equivalence that makes this seem as if it's not a computation. The camera is not running the same algorithm as the renderer and so you're com…

I don't think it's a non-event even if you're only interested in factoring. This proves the underlying principle of a quantum speedup is a physical reality. It might be something p…

Would be even nicer if it could be trained on unpaired datasets (ala CycleGAN https://arxiv.org/abs/1703.10593 ).

The author notes that the analysis is based on 1300 songs from the Billboard top 100 in the last 2 years. I think it's safe to say the effect of metal songs using phrygian is negli…

Distill[1] is another example of interactive scientific papers (with a focus on machine learning). But is there really a good reason to not just keep these in browser? I don't real…

Accompanying jupyter notebook: https://colab.research.google.com/drive/1e9o2NB2GDDjadptGr3r...