> > an "open source" neural net, which is technically more accurate than the "error-prone" hand-written rules (ignoring that it supports far fewer filetypes)
> You say that like it's a contradiction but it's not.
> > and they want it to "help other software improve their file identification accuracy," which of course it can't since neural nets aren't introspectable.
> Being introspectable or not has no bearing on the accuracy of a system.
"Open source" and "neural net" is the contradiction, as I went on to write. Even if magika were a more accurate version of file, the implication that it could "help [libmagic] improve" isn't really true, because how do you distill the knowledge from it into a patch for libmagic?
My point re: their "error-prone" claim is that their comparison was disingenuous due to the functionality difference between the tools. (Also with their implication that AIs work perfectly, though this one sounds pretty good by the numbers. I of course accept that there's likely to be some bugs in code written by humans.)
> > and which is much less effective in an adversarial context,
> Is it? This seems like an assumption.
It is, one based on what I've heard about AI classifiers over the years. Other commenters here are interested in this point, but while I don't see anyone experimenting on magika (it's new after all), the fact it's not mentioned in the article leads me to believe they didn't try to attack themselves. (Or did, but with bad results, and so decided not to include that. Funnily enough they did mention adversarial attacks on manually-written classifiers...)