11–20 of 28 posts
Would love to hear more options in that bottom left box with differential dataflow et al, because for me that's where all the really interesting work is happening.
To me the unstructured area is the future of applications development and the structured area is barren and overhyped. (e.g. 'eventually consistent' analytics is so 2021, because it looks like you are doing something quantitative,except it doesn't matter if you get the right answer. It will get you claps from a certain audience but most people lose patience pretty quick when the 'numbers don't add up'.)
If you're doing any kind of Important Reporting on eventually consistent data, you'd better make sure that you either know you're only including finalised data, or that there's a big fat warning with error bars.
To me the unstructured area is the future of applications development and the structured area is barren and overhyped. (e.g. 'eventually consistent' analytics is so 2021, because it looks like you are doing something quantitative,except it doesn't matter if you get the right answer. It will get you claps from a certain audience but most people lose patience pretty quick when the 'numbers don't add up'.)
I also disagree that unstructured is the future. I think most computations actually are structured. Otherwise sql queries wouldn’t be so useful. I think a lot of processes basically start with a big bag of foos and end up with a big bag of roughly corresponding bars, so if one foo changes then there is only a localised change to the output. That can be encoded with an unstructured system but I find they are better for cases where outputs are more scalar and the processes between inputs and outputs are myriad. While you can do structured-like things in an unstructured system, you lose out on a lot of the advantages of that structure.
To me the unstructured area is the future of applications development and the structured area is barren and overhyped. (e.g. 'eventually consistent' analytics is so 2021, because it looks like you are doing something quantitative,except it doesn't matter if you get the right answer. It will get you claps from a certain audience but most people lose patience pretty quick when the 'numbers don't add up'.)
I can’t really relate your first paragraph to your second, at least by the taxonomy in the article which has some consistent structured systems as well as inconsistent systems. I also disagree that unstructured is the future. I think most computations actually are structured. Otherwise sql queries wouldn’t be so useful. I think a lot of processes basically start with a big bag of foos and end up with a big bag of rou…
For instance a tool like this
maintains a database of timely information and can use rules to match events and could be used to manage the numerous problems of event-based architectures.
That particular tool mangles Java source code badly in the process of compiling rules so it gives error messages that make no sense at all, even if you are looking at the compiler's source code in the other window and at the running compiler in the debugger.
In the light of the interest in "low code" and the real success of "business rules" for domains they apply in I am amazed there has been less effort to apply production rules to the "update the ui when the database changes".
Even though that can look unstructured, the implementation of the rules can be done with the same relational operators that all those other methods used. A few year ago you would have had to specified the indexes by hands, but the self-optimizing database patent from Salesforce is run out now and there is no reason the system can't learn the frequent query patterns itself.
I feel like this map is missing something to tease apart systems that are not interchangeable in practice (UI state management vs database stuff). Maybe persistent vs in-memory? Like whether the data is typically completely blown away and recomputed on page reload (e.g., ui state, dom trees, scene graphs).
Would love to hear more options in that bottom left box with differential dataflow et al, because for me that's where all the really interesting work is happening.
But I've only played with souffle, which I am not sure where it lands in this taxonomy. I think it lands on the high temporal locality but compared to differential datalog, the biggest difference in my opinion is the souffle solves problems as a batch (i.e., all the inputs are known at command issue time) while differential datalog may receive inputs at runtime.
There's also incA: which I think competes with differential datalog. https://github.com/szabta89/IncA
So, how I see Datalog (which is a subset of prolog) falling into this taxonomy:
Datalog (batch-processing) would fall under structured/high-temporal locality/internally consistent.
Datalog (incremental) would fall on structured/low-temporal locality/internally consistent.
Looking a little closer on what is defined as "consistent" here, incremental Datalog might not be "consistent" because it might return an "incorrect" or "unavailable" computation for a past input if you "update" or modify the input. But if one restricts the subset of Differential Datalog to follow functional semantics, then that would be consistent. Not really super knowledgable about this projects beyond playing with them and reading some papers on Datalog/Souffle.
To me the unstructured area is the future of applications development and the structured area is barren and overhyped. (e.g. 'eventually consistent' analytics is so 2021, because it looks like you are doing something quantitative,except it doesn't matter if you get the right answer. It will get you claps from a certain audience but most people lose patience pretty quick when the 'numbers don't add up'.)
IMO the key with eventual consistency is making it known which bits are currently known to be consistent, and which are still in flux. I realize that this is vague, but how that works depends on what it is you're processing and how much time it takes for consistency to emerge. If you're doing any kind of Important Reporting on eventually consistent data, you'd better make sure that you either know you're only includi…
This isn't quite what the Helios paper talks about, but there's lots of things like async indexing in there that are kinda similar in nature
I believe Redis belongs in this map.