Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Executable Is a SQLite Database

fzakaria.com

Post reply on HN
Print view View original

101–110 of 116 posts

Re: Executable Is a SQLite Database

#101

Post by setheron »

catlifeonmars wrote: post #98
setheron wrote: post #62

Earlier quoted context omitted.

(author) In the post I say I did write sqlelf which is just that, sqlite + virtual tables prior to this.

Thanks for responding! I hope I didn’t come across as being overly critical. I’m curious what you would consider if designing a greenfield implementation of an object file format. What would you optimize for, and perhaps more interestingly, what would you not optimize for?

I like your phrasing; I think too often we get caught trying to optimize for everything.

For my own edification I often let performance & size slip to see if I can re-imagine something simpler from the start from a complete new direction;

This is often the greenfield hubris because it doesn't have all the same battle-scars of the full implementation so I try to make sure I take a very different approach to make it worthwhile :)

Then I just have faith that performance and size can always be improved over time.

Re: Executable Is a SQLite Database

#102

Post by setheron »

pstuart wrote: post #48
unified101 wrote: post #12

I don't think this goes far enough! Make the actually app store be the same file itself. so it's a living application and the file is constantly updated to how you use it. Copy it around, and you carry your data wiht you as well. Let's go deeper. it's a webserver app + the server code + application code + db, so pocketbase++ where it's also the deployment target. Then combine it with APE liek system, and the same fil…

Doesn't redbean do this in a more portable way (not ELF dependent)?

You might like https://fzakaria.com/2026/08/24/actually-queryable-executabl...

Re: Executable Is a SQLite Database

#103

Post by garganzol »

hedgehog wrote: post #80
setheron wrote: post #78

Earlier quoted context omitted.

I might look at this next; with Nix it's easy to explore and rebuild seamlessly ;P

I didn't understand the core issue, is it that due to non-alignment of the on-disk data you have to do a copy at load time, but if you could guarantee alignment of the actual blob content then you could use it directly?

Exactly, a guaranteed alignment and continuity of a blob makes it memory-mappable allowing to avoid copying the data back and forth.

Re: Executable Is a SQLite Database

#104

Post by account42 »

test6554 wrote: post #96
andrewl wrote: post #27

Earlier quoted context omitted.

Yes. In this interview Michael Stonebreaker, one of the original developers of PostgreSQL, discusses (among other topics) replacing the file system with a database : https://www.youtube.com/watch?v=YPObBOwIrHk

When you have a hammer, everything starts to seem like a nail. /s :)

Why the /s? There is a reason why we don't use hammers for all tasks even though we could find ways to accomplish many of them with one.

Re: Executable Is a SQLite Database

#105

Post by wduquette »

test6554 wrote: post #97
andrewl wrote: post #87

Earlier quoted context omitted.

I like a hierarchy, too. Presumably the underlying engine could be a database but it would at least appear to the user as a tree. But with the option to find files with SQL or a SQL-like language. I would love to be able to find files based on all kinds of metadata that was available for searching with the OS’s file system, and not using any special tools. I’m thinking things like EXIF data, audio file metadata, text…

The tree-view is practical for users but seems impractical for relational database performance. SELECT * FROM files p LEFT JOIN files c ON p.id = c.parent_id LEFT JOIN files gc ON c.id = gc.parent_id LEFT JOIN files ggc ON gc.id = ggc.parent_id .... where p.id = 123

Datalog could handle it, though.

Re: Executable Is a SQLite Database

#106

Post by antonvs »

dcrazy wrote: post #56

Congratulations, you’ve rediscovered mainframes.

In what way? Are you thinking of a particular mainframe?

My mainframe experience was on an IBM 370/158 using VM/CMS, and I can't see any real connection between the OP and its executables.

I suppose there's a very vague connection to how the LOADER could load TEXT files (similar to a .o file) into virtual storage, without having linked an executable (a MODULE). But that's about runtime usage, not file format, and there's certainly no SQL functionality that applies to executables on mainframes.

Re: Executable Is a SQLite Database

#107

Post by naikrovek »

pstuart wrote: post #47
andai wrote: post #32

This whole article is fantastic, but already at the start, the SQLite virtual tables thing is blowing my mind. https://www.sqlite.org/vtablist.html You can "mount" your filesystem (or anything else) as a SQL database, wtf. That's amazing. This sounds like it could be extremely useful.

Check this out: https://github.com/osquery/osquery

[dead]

Re: Executable Is a SQLite Database

#108

Post by inigyou »

dist-epoch wrote: post #61
inigyou wrote: post #41

Earlier quoted context omitted.

You'd have to at least increase the page size to allow 4kB of consecutive data without interfering with the page header. OR you restructure the code and data to work around the page headers.

Or you could write a new SQLite VFS that somehow separates the page headers from the page data.

You still have row headers. Yes in the extreme you could write a VFS that parses the blobs out of the database and writes them sequentially somewhere, but it's much more practical to set the page size to 64kB and probably get 56kB of contiguous data in the middle of each page, then add padding into your blobs so that contiguous data is exactly the data you want to mmap

Re: Executable Is a SQLite Database

#109

Post by Joker_vD »

inigyou wrote: post #100
Joker_vD wrote: post #71

Earlier quoted context omitted.

Also known as a directory.

If a directory was contiguous on disk.

Most files aren't contiguous on a disk either, you know.

Which makes inability to punch wholes in the middle of the files without having to read/write huge chunks of data quite annoying, especially when you write a linker or an archiver. Reserving enough space for the header is quite non-trivial.

Re: Executable Is a SQLite Database

#110

Post by inigyou »

Joker_vD wrote: post #109
inigyou wrote: post #100

Earlier quoted context omitted.

If a directory was contiguous on disk.

Most files aren't contiguous on a disk either, you know. Which makes inability to punch wholes in the middle of the files without having to read/write huge chunks of data quite annoying, especially when you write a linker or an archiver. Reserving enough space for the header is quite non-trivial.

Filesystems try fairly hard to make files mostly contiguous on disk, but they don't do the same to directories.
Post reply on HN