Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Magika: AI powered fast and efficient file type identification

opensource.googleblog.com

Post reply on HN
Print view View original

151–160 of 262 posts

Re: Magika: AI powered fast and efficient file type identification

#151

Post by chmod775 »

barrkel wrote: post #128
m0shen wrote: post #37

Earlier quoted context omitted.

Made a small test to try it out: https://gist.github.com/moshen/784ee4a38439f00b17855233617e9... hyperfine ./magika.bash ./file.bash Benchmark 1: ./magika.bash Time (mean ± σ): 706.2 ms ± 21.1 ms [User: 10520.3 ms, System: 1604.6 ms] Range (min … max): 684.0 ms … 738.9 ms 10 runs Benchmark 2: ./file.bash Time (mean ± σ): 23.6 ms ± 1.1 ms [User: 15.7 ms, System: 7.9 ms] Range (min … max): 22.4 ms … 29.0 ms 111 runs Su…

Realistically, either you're identifying one file interactively and you don't care about latency differences in the 10s of ms, or you're identifying in bulk (batch command line or online in response to requests), in which case you should measure the marginal cost and exclude Python startup and model loading times.

Going by those number it's taking almost a second to run, not 10s of ms. And going by those numbers, it's doing something massively parallel in that time. So basically all your cores will spike to 100% for almost a second during those one-shot identifications. It looks like GP has a 12-16 threads CPU, and it is using those while still being 30 times slower than single-threaded libmagic.

That tool needs 100x more CPU time just to figure out some filetypes than vim needs to open a file from a cold start (which presumably includes using libmagic to check the type).

If I had to wait a second just to open something during which that thing uses every resource available on my computer to the fullest, I'd probably break my keyboard. Try using that thing as a drop-in file replacement, open some folder in your favorite file manager, and watch your computer slow to a crawl as your file manager tries to figure out what thumbnails to render.

It's utterly unsuitable for "interactive" identifications.

Re: Magika: AI powered fast and efficient file type identification

#152

Post by m0shen »

invernizzi wrote: post #85
m0shen wrote: post #33

As someone that has worked in a space that has to deal with uploaded files for the last few years, and someone who maintains a WASM libmagic Node package ( https://github.com/moshen/wasmagic ) , I have to say I really love seeing new entries into the file type detection space. Though I have to say when looking at the Node module, I don't understand why they released it. Their docs say it's slow: https://github.com/go…

Hello! We wrote the Node library as a first functional version. Its API is already stable, but it's a bit slower than the Python library for two reasons: it loads the model at runtime, and it doesn't do batch lookups, meaning it calls the model for each file. Other than that, it's just as fast for single file lookups, which is the most common usecase.

Good to know! Thank you. I'll definitely be trying it out. Though, I might download and hardcode the model ;)

I also appreciate the use of ONNX here, as I'm already thinking about using another version of the runtime.

Do you think you'll open source your F1 benchmark?

Re: Magika: AI powered fast and efficient file type identification

#153

Post by wildrhythms »

fuzztester wrote: post #121
nindalf wrote: post #46

Earlier quoted context omitted.

> no way an "AI powered" tool can be more reliable The article provides accuracy benchmarks. > you would be better off just rejecting it completely They mention using it in gmail and Drive, neither of which have the luxury of rejecting files willy-nilly.

I have not tried it recently, but IIRC, Gmail does reject attachments which are zip files, for security reasons.

Gmail nukes zips if they contain an executable or some other 'prohibited' file type. Most email providers block executable attachments.

Re: Magika: AI powered fast and efficient file type identification

#154

Post by diffeomorphism »

vertis wrote: post #120
_puk wrote: post #105

Earlier quoted context omitted.

Whilst energy usage is indeed a small aspect this early on when using bespoke models, we do have to consider that this is a model for simply identifying a file type. What happens when we introduce more bespoke models for manipulating the data in that file? This feels like it could slowly boil to the point of programs using magnitudes higher power, at which point it'll be hard to claw it back.

That's a slippery slope argument, which is a common logical fallacy[0]. This model being inefficient compared to the best possible implementation does not mean that future additions will also be inefficient. It's the equivalent to saying many people programming in Ruby is causing all future programs to be less efficient. Which is not true. In fact, many people programming in Ruby has caused Ruby to become more effici…

That is a fallacy fallacy. Just because some slopes are not slippery that does not mean none of them are.

Re: Magika: AI powered fast and efficient file type identification

#157

Post by lopkeny12ko »

potatoman22 wrote: post #52
lopkeny12ko wrote: post #51

I don't understand why this needs to exist. Isn't file type detection inherently deterministic by nature? A valid tar archive will always have the same first few magic bytes. An ELF binary has a universal ELF magic and header. If the magic is bad, then the file is corrupted and not a valid XYZ file. What's the value in throwing in "heuristics" and probabilistic inference into a process that is black and white by desi…

This also works for formats like Python, HTML, and JSON.

I still don't see how this is useful. The only time I want to answer the question "what type of file is this" is if it is an opaque blob of binary data. If it's a plain text file like Python, HTML, or JSON, I can figure that out by just catting the file.

Re: Magika: AI powered fast and efficient file type identification

#158

Post by omni »

Can someone please help me understand why this is useful? The article mentions malware scanning applications, but if I'm sending you a malicious PDF, won't I want to clearly mark it with a .pdf extension so that you open it in your PDF app? Their examples are all very obvious based on file extensions.

Re: Magika: AI powered fast and efficient file type identification

#160

Post by michaelmior »

TomNomNom wrote: post #147
michaelmior wrote: post #134

Earlier quoted context omitted.

> the current implementation can't be relied on IMO What's your reasoning for not relying on this? (It seems to me that this would be application-dependent at the very least.)

It provided the wrong file-types for some files, so I cannot rely on its output to be correct. If you wanted to, for example, use this tool to route different files to different format-specific handlers it would sometimes send files to the wrong handlers.

Except a 100% correct implementation doesn't exist AFAIK. So if I want to do anything that makes a decision based on the type of a file, I have to pick some algorithm to do that. If I can do that correctly 99% of the time, that's better than not being able to make that decision at all, which is where I'm left if a perfect implementation doesn't exist.
Post reply on HN