81–90 of 291 posts
We need entirely new OS abstractions to replace the dated notions of hierarchical file systems built around the metaphor of file cabinets, I/O as streams of bytes, terminals, process hierarchy. Essentially, say goodbye to the Unix model after 50 years. It would open up an entirely new world of software experimentation and craftsmanship.
I could see that for some very small niches, but in general I think it would be a terrible development for the industry.
Hardware vendors don't like to share. They don't share code, they don't share common interfaces, they don't even share documentation. As it is now, these are all problems which most userland developers don't have to care about -- those problems get dealt with in the kernel, by developers who specialize in building support for uncooperative hardware.
The average application developer doesn't want to have to figure out how many queues are supported by a NIC just to open a connection on the network. Further: the average application developer isn't experienced enough to do this correctly.
Given the niche where these tradeoffs make sense, I'm not sure why the paper bothers to emphasize security at all.
Earlier quoted context omitted.
Amen to that. Unix was never a good design, and now is severely out of date. We can no longer afford to hack around it.
"Unix was never a good design" Nothing's perfect. Unix was a great design that served its purpose well for a long time, and evolved a bit along the way. Saying it was "never good" is trivializing and, in my opinion, arrogant.
"Unix went from being the worst operating system available, to being the best operating system available, without getting appreciably better."
https://news.ycombinator.com/item?id=19416485
Which isn't to say that those POV were necessarily correct, just that it isn't all hindsight.
Earlier quoted context omitted.
It's both.
No, it is not. Normalization is an artefact of Relational Algebra (a set theory being applied to tabular data). It has nothing to do with concrete implementations and their concerns (such as IO). http://wiki.c2.com/?RelationalModel . Moreover, ALL existing relational databases do denormalize data behind the scenes (think about caches and indexes) for the sake of IO optimization, which means that normalization does no…
Originally, schema normalization was just about logical consistency. But that was before query optimizers were invented, and long before they grew to be so very optimized for working with normalized schemata.
Nowadays, even someone who cares 100% about performance and 0% about data integrity (And is somehow still using a relational database, yeah. So what? It's MY hypothetical.) still has good reason to default to something like 3NF. Insert famous Knuth quote here.
Caches and indexes and all that stuff may technically count as denormalizations, but, insofar as the definition of 3NF doesn't mention them, that's a tangential issue.
At least in database kernels, we noticeably reached this threshold around five years ago with typical server hardware. This is an interesting computer science problem in that virtually all of our database literature is based on the presumption that I/O is much slower than CPU. If you cleanroom a database kernel design based on the assumption that I/O performance is not the bottleneck, you end up with an architecture…
Are you aware of any efforts by existing databaes or entirely new databases built with this realization in mind?
Michael Stonebreaker, a Turing awardee, called for complete destruction of old database order in 2008: https://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-... post which, I think, they created VoltDB (H-Store) and SciDB.
Interesting to note that Andy Pavlo worked on Stonebreaker's H-Store.
We need entirely new OS abstractions to replace the dated notions of hierarchical file systems built around the metaphor of file cabinets, I/O as streams of bytes, terminals, process hierarchy. Essentially, say goodbye to the Unix model after 50 years. It would open up an entirely new world of software experimentation and craftsmanship.
One idea I've been throwing around is to replace files with HTTP resources; everything is a resource. Effectively Plan9s idea, but the time might be right.
Earlier quoted context omitted.
You're misreading the article. Its subject is that because of the way IO stacks have been built CPUs are becoming the bottleneck in IO , this is an issue for both network and non-volatile storage IO e.g. > a 40 GbE NIC can receive a cache line sized packet every 5 ns, but the last level cache (LLC) access latency is already up to 15 ns, which means a single LLC access can already prevent the OS from keeping up with a…
Latency ≠ throughput
Earlier quoted context omitted.
I would like to see an updated file system architecture that's closer to a database, with tagging and all that. And I could see that extending to processes too. But what are you imagining as alternatives for i/o and terminals?
What if we surpass file systems completely? Imagine every process having its own "non-volatile" memory area, where all persistent state is kept between restarts, with a clean OS interface to share that state to other processes? Terminals and process hierarchy are a complete no-go in a fresh design. Every entity in the system can be identified through unique IDs which can be handed down to other processes based on dif…
I'll treat this seriously: as software development practices are just now beginning to mature, with more emphasis on test coverage being considered best practice, sure ... maybe.
But there's still a lot of software out there for which "restart the application" (or even, "restart the stinking OS") is the only practical solution when it's behaving badly.
> Every entity in the system can be identified through unique IDs which can be handed down to other processes based on different OS policies.
Okay, but how do you write a process which is capable of communicating with any other process?
The paper reads like it's suggesting moving the burden of complexity in dealing with varying hardware interfaces from the kernel to userland so that userland can take direct advantage of higher performance hardware when it's available. I could see that for some very small niches, but in general I think it would be a terrible development for the industry. Hardware vendors don't like to share. They don't share code, th…
Is there any benefit to having those specialized developers create frameworks or libraries in Userland which other developers can leverage? This way they remain the interface to uncooperative hardware, but the code is in Userland so the bold folks can try their own approach.
Earlier quoted context omitted.
Would you care to elaborate why Unix, in your opinion, was never a good design?
It was re-written on an unsafe language that completely disregarded what has happened outside Bell Labs since 1961, at Burroughs, IBM, MIT, Xerox and other research cathedrals. It only took off thanks to Bell Labs being forbidden to sell it, so it got offered for a symbolic price, alongside source code to major universities, which then decided to build on top, instead of paying OS street prices. Had UNIX been sold in…