Viewing profile — nil-sec

Sorry maybe I should have added more explanation. One way to think about attention, which is the main distinguishing element in a transformer, is as an adaptable matrix. A feedforw…

I guess I was more thinking about self attention, so yes. The more general case is covered by your notation!

Feedforward: y=Wx Attention: y=W(x)x W is Matrix, x & y Are vectors. In the second case, W is a function of the input.

I had the same experience a couple of years back when people started discussing AI. I try to keep this in mind but somehow I keep forgetting. It’s genuinely difficult to filter goo…

It’s quite nice to have something irreversible though. It gives you time. Also nobody really thinks QM is the end of it, assuming semi classical physics under the hood is just odd …

It’s funny because, for me, this was one of the major confusions when I moved from Europe to the US. In Europe, there are a lot of prominent subcultures, particularly in college. T…

This isn’t true, the quality of images generated by DALL-E are really good, but they are an incremental improvement and based on a long chain of prior work. See e.g. https://github…

Insane take, the west vs Russia in open confrontation ends in a nuclear winter.

I have no background in control theory but this sounds very similar to the identifiability problem in nonlinear ICA. Are those equivalent?

This is a good point which I think stems from wrongly equating human level intelligence to AGI in the popular literature. It’s not at all clear what a general intelligence should b…

While I agree with the general point of this paper I don’t think it’s quite right to compare the current situation with the last AI spring. It’s not AGI but it’s very good narrow A…

For a given causal model it is (1) in my understanding.

For one it lets you avoid controlling for the wrong variables and causing e.g. spurious correlations by doing so. In fact this is one of the best examples of why a causal model is …

It is incomprehensible to you, because you just simply do not understand what your parent is talking about. You are the ignorant one here and indeed quite rude. Doesn't matter that…

Agreed, I trained a 3D version of b0-b2 on a classification task I worked on and besides being very slow to train they did not outperform a simple baseline VGG architecture. Intere…

Location: Zürich Remote: Yes Willing to relocate: Yes, within Europe. Technologies: Pytorch, Tensorflow, Python, C++, JS, HTML/CSS, Gurobi, MongoDB CV: https://nilsec.github.io Ema…

They could have used all negative samples for testing (and even training if they would have done it better), yes. But once your test set is large enough, whatever that means, its n…

1. Isn't an issue. They make inference on a sample by sample basis. The network has no memory so it won't expect a 50/50 distribution on the test set just because its trained like …

Would love to play around with this data but there are no seeds for this torrent. Anyone here who can provide this dataset?

Let me be very concrete: There is currently a lot of research being done in so called “contrastive learning”. This is an unsupervised technique in which you train a network to buil…

I happen to work in AI research and what you are saying isn’t true. There is theoretical machine learning and applications of it. They are distinct. The former is largely task agno…

In industry, yes. I had the impression this article was aimed at academia and research in AI.

The fraction of people in AI working on problems that need to consider diversity/fairness/etc. is rather small. Yes, those people, these specific applications, should be designed w…

This is a very one-sided view of what this article is talking about. That there is systemic racism in the US is not up for debate. It's a fact. Even if your back of the envelope ca…