Nicely done and thoughtfully executed. As a quibble on the suggested use cases, this platform will not work for the Internet of Things generally for two reasons. First, it lacks support for the spatial data types, including polygons, and operations, including spatial joins, that are typical of those types of data models. Second, typical commercial IoT data sources are often on the order of 100TB-1PB per day , which i…
so what kind of tool would you recommend instead to deal with 100TB-1PB per day ? I'm genuinely interested.
If you look at every company that is working in this space, one of the first things you will notice is that they all use custom storage engines that do a full operating system bypass i.e. they manage all the system resources in userspace. If you do not do this, you cannot reliably get the necessary throughput out of the system for IoT. As far as I know, no scalable storage+execution engine in open source is designed like this yet. It requires much more computer science sophistication and lines of code to implement compared to traditional storage engines, so not the kind of thing you hack together over the weekend.