Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Principles of Data Oriented Programming

blog.klipse.tech

Post reply on HN
Print view View original

71–80 of 139 posts

Re: Principles of Data Oriented Programming

#71

Post by prostodata »

shwestrick wrote: post #20
conceptoriented wrote: post #14

> Data never changes, but we have the possibility to create a new version of the data. Well, it depends on what you mean by data. To avoid ambiguity it is better to talk about data values and data objects which have different properties. This can be formalized as follows [1]: o data values are modelled via mathematical tuples – tuples are immutable o data objects are modelled via mathematical functions (one field is…

What do you mean by "functions are supposed to be mutable"? Perhaps you are just pointing out that the output of the function (and therefore the value of the field...?) will change as the input changes? If mathematical tuples are immutable, then surely mathematical functions are immutable as well ;)

Here is one possible implementation of the concept-oriented model of data for data processing. It heavily relies on functions and operations with functions and is an alternative to purely set-oriented approaches like map-reduce or join-groupby (sql):

https://github.com/prostodata/prosto - Functions matter!

Re: Principles of Data Oriented Programming

#72

Post by jmull »

I don't think these principles add up to something useful. It's not complete and I think some of the principles don't align that well to the problem space.

The big one: "Data is immutable". The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data. The "source of truth" becomes muddled which creates the problem of how to keep the various instances in sync (or otherwise handle cases where they are/become out-of-sync). Immutability is a very useful tool since you have have any kind of dependency -- direct, indirect, implied, etc) -- and there's no worry. But you still need a mechanism to manage mutating data. Maybe it comes out somewhere, but the principles of DO don't cover it, which is a rather serious omission.

Also, principle 2 isn't really a principle. I think what it's getting at is that you don't really know the precise type of your data, over time, in a distributed system, so it's good to include the flexibility to handle that. That makes sense to me. but generic data structures aren't necessarily always the right way to handle that.

Re: Principles of Data Oriented Programming

#73

Post by Joker_vD »

discreteevent wrote: post #68
Joker_vD wrote: post #23

You know, sometimes I wish the programming languages had a better distinction between "immutable data pieces" and "stateful agents". Sometimes it's really nice to have a simple struct for which you (or anyone else) can write many function to act upon. Sometimes it's really nice to have an opaque "object" with methods to yank described in some public interface, but which encapsulates loads of state and other objects i…

The "stateful agents" is what OOP is all about. The objects are interpreters. I referenced a discussion elsewhere in this thread between Alan Kay and Rich Hickey where Kay talks about the value of interpreters. Rich Hickey talks about data/values coming from a seismometer or an IoT device but what he misses there I think is that it's not all just "data". There are also stateful agents. The seismometer (or the earthqu…

Right. And the problem is, quite often those "data/values" turn out to be objects too, and for no good reason. Take the whole Active Records approach, for example, re-implemented in Java in the most straightforward way possible: you have a class with data fields (well, with getters/setters) but it also has "Save()/Load()" methods on it. Ugh.

Re: Principles of Data Oriented Programming

#75

Post by adamnemecek »

skohan wrote: post #66
adamnemecek wrote: post #64

Earlier quoted context omitted.

None of these really apply in this case. In data oriented programming, the data would not be stored in heterogenous lists but each element would be stored in a different, homogenous array and you would ties these together using the same entity ID.

Even with this type of design, it's possible to implement in a typesafe way. I have seen clever ECS systems accomplish this

Maybe but ECS does in some sense circumvent types.

Re: Principles of Data Oriented Programming

#76

Post by conceptoriented »

louthy wrote: post #56
conceptoriented wrote: post #35

Earlier quoted context omitted.

Function is a mapping between two sets (of values). This mapping between values is mutable although the values are not.

Functions are a mapping between a domain and a codomain, the mapping absolutely isn’t mutable, the definition of the function is the relationship between the domains. If I have a function: int Add1(int x) => x + 1 I would expect the domain and codomain to be immutable; I would also expect that x+1 to not turn in x/2 randomly also

> the mapping absolutely isn’t mutable

Assume f: X -> Y. We can now map x_1 to y_1 f(x_1)=y_1. And then change this same function by mapping x_1 to y_2: f(x_1)=y_2. Thus we can easily modify functions. Moreover, we do it constantly when we modify object fields in OOP. It is probably easier to comprehend if a function is represented as a table which we modify.

In contrast, we cannot modify data values (mathematical tuples). Say, x=42+1 means that a new value 43 is created rather than the existing value 42 is modified.

> I would expect the domain and codomain to be immutable;

No. Domains, codomains and any set can well be modified by adding or removing tuples. What is immutable are values (in the sets).

Re: Principles of Data Oriented Programming

#78

Post by dragandj »

jll29 wrote: post #21

It's a good idea to write a book about a data-oriented development style (I'm working on a methodology, a way of working in data-intensive projects called Data2Value). However, JavaScript or Clojure are not ideal for demonstrating this methodology in the sense that industrial applications will more likely be built in C++, Java or Python. For example C++ and Java support Apache UIMA, which is an industry standard for…

FYI Clojure supports GPU processing since 5 years ago.

https://github.com/uncomplicate

Re: Principles of Data Oriented Programming

#79

Post by rellekio »

jmull wrote: post #72

I don't think these principles add up to something useful. It's not complete and I think some of the principles don't align that well to the problem space. The big one: "Data is immutable". The problem here is that data isn't actually immutable (generally) and mutability isn't actually the problem. The problem is unmanaged references or other dependencies on the mutable data. The "source of truth" becomes muddled whi…

Your right about the sync and dependency issue. Big reason why in JS land you treat each mutation of the data as immutable destruction and refresh of an object is to eliminate any old references that might not of been cleaned up by the garbage collector.

Data Oriented Programmming is both old and new. In that it does not have the same amount of programming patterns that OOP has. As it is a more bare metal means of programming without a ton of abstraction to ease most programmers into it.

Where I find the idea interesting is concurrent and parallel processes are more natural in the data oriented. And that is through immutablility and ownership as first principles.

Re: Principles of Data Oriented Programming

#80

Post by dpc_pw »

So... is Data Oriented something new?

Storing fields in a map leads me to believe this is not Data Oriented Design (DOD). And I completely reject this idea (fields in maps). The "flexibility" there is hardly useful, and could be achieved with defined shapes (types) in modern statically typed languages without all the dowsides.

"Separate code from data" is a big core belief I share with this article, but the rest doesn't seem good idea / novel / important.

Post reply on HN