Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with Andreas Kling of Serenity OS (2022)

corecursive.com

Post reply on HN
Print view View original

81–90 of 181 posts

Re: Interview with Andreas Kling of Serenity OS (2022)

#81

Post by josephg »

electroly wrote: post #60
vacuity wrote: post #41

Earlier quoted context omitted.

I feel like it should be possible to say "I want to reserve access to this resource when something important happens on it". Or just "I want to reserve access to this resource". Hopefully eliminates TOCTOU races. I guess then there have to be different modes to balance scalability/flexibility.

When people complain about antivirus and endpoint security applications slowing their computer down, this is what they're talking about. This leads to user-visible delays (which, to them, seem to be random) as the reserver does what it wants right after the user has done something to the file. Many (most? ALL?) apparent Explorer stalls are really Windows Defender locking and scanning the file right after you did some…

Yeah good point. I don't want write stalls because of subscription requests. For things like filesystem watching (for compilers), it'd be enough to have a form of snapshot checkpointing. (There were these FS events within 10ms [..]) / (Then the file contents was XXX) / ... (Then there were these FS events) / and so on.

Its possible you'd end up in situations where the OS would need to retain old file contents if there's both a pending read from a subscriber and subsequent writes. But in most cases, this is fine.

Going back to my original comment, this is the sort of reason I'd like to try this in my own OS kernel rather than try and solve all these problems immediately in linux. It'll give me a chance to try out a few different solutions and see if I can find something that works well.

Re: Interview with Andreas Kling of Serenity OS (2022)

#82

Post by josephg »

Findecanor wrote: post #45
josephg wrote: post #2

I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. There's a few things I desperately want an operating system to do: 1. Have either database-like transactions or (at a minimum) write barriers for file operations. The current filesystem semantics are terrible for databases and other applications which need to not corrupt their data after a crash. fsync() is too heavy, because you can't…

> I want to try replacing all of that with a simple, unified API which lets you say "tell me the state of and tell me when it changes". Fuchsia's "Signals" are designed in that direction. You get a "handle" to a resource, and the resource has a 32-bit integer with signal bits that you could subscribe to. The meaning of each bit is different for each type of resource. (In general operating system terminology, "Signal"…

Oh I didn't know that! Thanks - I'll take a look.

Re: Interview with Andreas Kling of Serenity OS (2022)

#83

Post by josephg »

throw10920 wrote: post #39
josephg wrote: post #2

I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. There's a few things I desperately want an operating system to do: 1. Have either database-like transactions or (at a minimum) write barriers for file operations. The current filesystem semantics are terrible for databases and other applications which need to not corrupt their data after a crash. fsync() is too heavy, because you can't…

> It would also be interesting to try making all userspace programs be wasm bundles and run them in ring0. But I suspect that will run slower on modern hardware compared to using native binaries and context switching. If you have all of your programs reside in the same virtual memory space (which is the implication with running them in ring 0), then you get a unique performance advantage in that you don't have to inv…

Yeah - if you run things in the kernel you don't need to do TLB flushes or context switches - which can be very expensive. But thats all offset by the fact that wasm code runs slower inherently, because it needs to bounds checks everywhere.

I suspect that programs which make a lot of kernel calls would run faster in the kernel in wasm, and programs that are more compute / memory bound will run faster as native processes. It'd be fun to play around and find where the tradeoffs lie for different applications.

Re: Interview with Andreas Kling of Serenity OS (2022)

#84

Post by EvanAnderson »

rkagerer wrote: post #38
shiroiuma wrote: post #7

Earlier quoted context omitted.

Why is this "hacky"? I've long wondered why databases need to be on filesystems, other than simply convenience. If you think about it, a filesystem is, itself, basically a database (not a relational one of course). So it seems like you could improve performance a lot by eliminating the filesystem layer and going straight to direct disk access. Also, how this would be done would probably change depending on whether yo…

I'd like to go the other way, and have a filesystem that leverages a proper database for its metadata. I can search millions of records in a split second in SQL. But searching records on my hard disk takes enormously more time. The solutions out there for speeding it up (like Windows Indexing service) rely on asynchronous indexing that's patched on top of the filesystem instead of integrated realtime within it, and a…

You'd probably enjoy reading about the Be File System: https://en.wikipedia.org/wiki/Be_File_System

It has relational database-like semantics for metadata.

Re: Interview with Andreas Kling of Serenity OS (2022)

#85

Post by drannex »

jerrygenser wrote: post #52
drannex wrote: post #50

Figured I would write this here, but I would scratch the concept of a file system and replace it with a database.

Isn't that how z/os from IBM works? The API is a SQL api into files I think

AFAIK it still uses a classical file system, just with a DQL system layered on top. I am talking about completely scrapping the idea of file systems, folders, the lot of it. A central database that stores all blobs, configurations, info, and other metadata (modified, created, revision history, diffs, keys, tags, and authority). This would enable real time distributed scalable systems, potentially immutable backups, and a lot more (such as fully auditable data transfers, messaging, access, and more for rapid debugging).

If an operationg system was built today for tomorrow it would certainly be built on/as a database.

Re: Interview with Andreas Kling of Serenity OS (2022)

#86

Post by josephg »

twic wrote: post #27
josephg wrote: post #2

I'm increasingly tempted to try my hand at making a toy OS - or at least a kernel. There's a few things I desperately want an operating system to do: 1. Have either database-like transactions or (at a minimum) write barriers for file operations. The current filesystem semantics are terrible for databases and other applications which need to not corrupt their data after a crash. fsync() is too heavy, because you can't…

So soft updates and kqueue? :)

I'm not sure you mean by soft updates but - kqueue is really just a messaging queue. I want a data subscription system, so I can get a kernel object and get a stream of updates which can be applied to the snapshot in sequence.

kqueue is very limited in terms of what you can subscribe to, and things like filesystem watching usually requires subsequent filesystem calls to see whats actually changed. And that in turn introduces race conditions if the data is changed again before your fs syscalls make it through.

I think there are cleaner APIs out there. I'd like to see if I can find one.

Re: Interview with Andreas Kling of Serenity OS (2022)

#87

Post by asddubs »

cedws wrote: post #75

Serenity is one of the only recent OS projects that I think has any potential. It's a very high velocity, high quality project and I think that's down to the community culture Andreas has fostered. They aren't a load of fuddy-duddies who insist on using mailing lists and IRC. Nobody wants to collaborate with a load of grumpy greybeards.

I think ladybird has unfortunately sucked most of the momentum out of the project.

Re: Interview with Andreas Kling of Serenity OS (2022)

#88

Post by aidenn0 »

spankalee wrote: post #78
morphle wrote: post #74

“An operating system is a collection of things that don’t fit inside a programming language; there shouldn’t be one” - Dan Ingalls 1981

That's a really terrible definition that dates from before multitasking, GUIs, and window systems and desktops.

Huh? Multitasking dates back to the time-sharing systems of the '60s.

Granted, while windowing systems existed in '81, they weren't widespread yet (Wikipedia lists PERQ as the first commercially released GUI workstation in '79).

Re: Interview with Andreas Kling of Serenity OS (2022)

#89

Post by skizm »

How does one get started on a toy OS? I don’t mean like concepts, but is there like a minimum viable OS .c file somewhere I can look at, with instructions for how to run it on “bare metal”? I assume I need a bootloader of some kind first?

I’m just totally unsure where I’d even begin in a practical sense.

Re: Interview with Andreas Kling of Serenity OS (2022)

#90

Post by tonetegeatinst »

skizm wrote: post #89

How does one get started on a toy OS? I don’t mean like concepts, but is there like a minimum viable OS .c file somewhere I can look at, with instructions for how to run it on “bare metal”? I assume I need a bootloader of some kind first? I’m just totally unsure where I’d even begin in a practical sense.

I think tinycore would be your best bet.

Its linux but stripped down to the very bare essentials

Post reply on HN