F3 • Hacker News Board

Re: F3

#21

Post by outside1234 » Tue, Jun 23, 2026, 5:32 PM UTC

gavinray wrote: post #18

verdverm wrote: post #12

Earlier quoted context omitted.

Is embedding executable code into a file a security risk? My assumption is a yes

There is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.

I mean can't we say the same thing about sending around a .exe though?

Re: F3

#22

Post by skrtskrt » Tue, Jun 23, 2026, 5:32 PM UTC

largbae wrote: post #3

This could use a bit more "why". Shortcomings of Parquet are mentioned as overcome by this, which ones? Certainly not wide tool support... Why should one leave Parquet or ORC for this structure?

Yeah it seems like most of this can be handled by some more dev hours to Parquet

Re: F3

#23

Post by andrewstuart2 » Tue, Jun 23, 2026, 5:32 PM UTC

gavinray wrote: post #9

This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist: > "Each self-describing F3 file includes both the data and meta-data, as well as WebAssembly (Wasm) binaries to decode the data. Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native…

I would call it clever. I'm not sure I'd call it genius.

When I'm working with data I'm working in a specific set of languages. Usually one. Yeah, other people might be working in other languages, but no individual author really needs a language-agnostic way of accessing data beyond compile time. Add to that the likely runtime boundaries that may need to be crossed instead of e.g. inlined by the compiler because it's in-language and dealing with known offsets or tags (depends on the data format of course). To the other commenter's point, am I going to have to sandbox all data access code just to be sure it's not able to do something unexpected? There's a lot of complexity here. And the inherent risk is going to slow down the operation that should be the simplest and fastest: interpreting bytes.

Re: F3

#24

Post by arcfour » Tue, Jun 23, 2026, 5:32 PM UTC

jasonjayr wrote: post #13

gavinray wrote: post #9

This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist: > "Each self-describing F3 file includes both the data and meta-data, as well as WebAssembly (Wasm) binaries to decode the data. Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native…

So attackers don't have to craft specially corrupted files? They can just include the code to perform the attack in the data file itself?

Yes...my first thought. No way in hell anyone actually trusts this.

(And as if we didn't trust the compiler enough already!)

Re: F3

#25

Post by jastanton » Tue, Jun 23, 2026, 5:33 PM UTC

gavinray wrote: post #18

verdverm wrote: post #12

Earlier quoted context omitted.

Is embedding executable code into a file a security risk? My assumption is a yes

There is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.

gotcha, so the vulnerability will be in some common libraries that attackers force some wasm fallback path with custom wasm instructions that when executed does something nefarious.

I'd say at worst it's setup for poor security

Re: F3

#26

Post by bluejekyll » Tue, Jun 23, 2026, 5:33 PM UTC

grodes wrote: post #16

gavinray wrote: post #9

This bit is quite genius, rather than depend on a language-specific SDK/lib for working with the formats you can fallback to exported WASM methods if none exist: > "Each self-describing F3 file includes both the data and meta-data, as well as WebAssembly (Wasm) binaries to decode the data. Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native…

How is wasm better than C bindings?

WASM is platform independent.

What do you mean by C bindings? C bindings to what?

Re: F3

#27

Post by jastanton » Tue, Jun 23, 2026, 5:34 PM UTC

outside1234 wrote: post #21

gavinray wrote: post #18

Earlier quoted context omitted.

There is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.

I mean can't we say the same thing about sending around a .exe though?

exactly

Re: F3

#28

Post by jedberg » Tue, Jun 23, 2026, 5:34 PM UTC

gavinray wrote: post #18

verdverm wrote: post #12

Earlier quoted context omitted.

Is embedding executable code into a file a security risk? My assumption is a yes

There is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.

Sure, but when the standard says "read this file and execute the instructions you find at the beginning" that is more dangerous than "this is a file with data and your program needs to figure out how to read it".

Re: F3

#29

Post by ghkbrew » Tue, Jun 23, 2026, 5:34 PM UTC

krzyk wrote: post #10

File format for what? Text, graphics, compiled code?

For columnar data storage I think. The description references Parquet and they appear to benchmark against Parquet, Vortex, and Lance.

Re: F3

#30

Post by gavinray » Tue, Jun 23, 2026, 5:35 PM UTC

outside1234 wrote: post #21

gavinray wrote: post #18

Earlier quoted context omitted.

There is no concept of "executable" vs "non-executable" content in a file. A file is a bag of bytes. You can send those bytes to different things, like a text editor's content-stream, or as the input to a WASM interpreter. What you decide to do with the bytes in a file is your own prerogative. Each byte is whatever you make of it.

I mean can't we say the same thing about sending around a .exe though?

Double-clicking an ".exe" (or running it via a shell) is not the same as "bag of bytes", it's "send these bytes to an executable environment".

Doing `head foo.exe` is quite different than `run foo.exe`

If I encode executable instructions in "image.png" and then send them to an interpreter that runs those instructions, the file extension doesn't matter.