Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Principles of Data Oriented Programming

blog.klipse.tech

Post reply on HN
Print view View original

21–30 of 139 posts

Re: Principles of Data Oriented Programming

#21

Post by jll29 »

It's a good idea to write a book about a data-oriented development style (I'm working on a methodology, a way of working in data-intensive projects called Data2Value).

However, JavaScript or Clojure are not ideal for demonstrating this methodology in the sense that industrial applications will more likely be built in C++, Java or Python. For example C++ and Java support Apache UIMA, which is an industry standard for data-oriented systems. UIMA (originally developed at IBM before open sourced, and use in their Watson system) manages data as immutable objects (e.g. text, videos) that are enriched with annotations (e.g. syntax graphs, topical tags, subtitles...).

Functional designs are often well-suited to data flow related processing, whereas in OOP, you end up with a pipeline object and various DataStream objects that it inputs and outputs.

In my experience, data-intensive systems often need to: - cater for distributed processing due to large-scale (which calls for Apache Spark, and then PySpark or Scala); - compute-intensive work like machine learning, which may require GPUs or other bespoke hardware (Tensorflow supports GPUs, Google's cloud has TPUs); and - special purpose data structures (e.g. Bloom filters, huge persistent graphs, R* trees...) specific to the nature of the processing (this latter point, I guess, contradicts to the author's claims).

Re: Principles of Data Oriented Programming

#23

Post by Joker_vD »

You know, sometimes I wish the programming languages had a better distinction between "immutable data pieces" and "stateful agents". Sometimes it's really nice to have a simple struct for which you (or anyone else) can write many function to act upon. Sometimes it's really nice to have an opaque "object" with methods to yank described in some public interface, but which encapsulates loads of state and other objects inside it so you don't have to think about all of that.

Re: Principles of Data Oriented Programming

#24

Post by dathinab »

viebel wrote: post #16
dathinab wrote: post #13

Earlier quoted context omitted.

Depending on how a language defines their syntax. Often times literals are only things like `"string"`, `0` and so one. But thinks like `[ 1,2]` would be a array expression where each "entry" is syntax wise an expression (and any literal is an expression itself). I don't know if JavaScript specifically does define object literals or object expressions, in the end it depends on what you define a literal as. Lastly dep…

I totally agree with "data should be creatable without explicitly doing any function calls, variable assignments or similar." Could you explain what you mean by "Creation must not depend on implicitly captured data"?

Lets say you have a language which has some form of macro system or similar to make creation of new thinks which look like literal-like expression easier.

E.g. instead of `a = [1,2]` you have `a = vec![1,2]` which de-sugars to `a = Vec::with_capacity(2); a.push(1); a.push(2);`.

Now custom data structures can define their own macros like that, e.g. `skip_list![1,2]`.

Which would be all fine. But what if now `bad_skip_list![1,2]` accesses a thread local variable (or other implicit provided data) and adds that, too?

Now `bad_skip_list![1,2]` might be not equal to `bad_skip_list![1,2]` defined somewhere else. Which is against the ideas behind the rule #5.

You should be able to copy-past the literal-like creation of data to any place (e.g. a unit-test) and get the same result.

EDIT: If I remember correctly you could override parts of `Array.prototype` and array construction in JavaScript to brake the #5 for JavaScript for thinks like `[1,2,3]` but I'm not to sure about that anymore.

Re: Principles of Data Oriented Programming

#25

Post by dudul »

Joker_vD wrote: post #23

You know, sometimes I wish the programming languages had a better distinction between "immutable data pieces" and "stateful agents". Sometimes it's really nice to have a simple struct for which you (or anyone else) can write many function to act upon. Sometimes it's really nice to have an opaque "object" with methods to yank described in some public interface, but which encapsulates loads of state and other objects i…

So like an Erlang map and a GenServer?

Re: Principles of Data Oriented Programming

#26

Post by mdm12 »

Joker_vD wrote: post #23

You know, sometimes I wish the programming languages had a better distinction between "immutable data pieces" and "stateful agents". Sometimes it's really nice to have a simple struct for which you (or anyone else) can write many function to act upon. Sometimes it's really nice to have an opaque "object" with methods to yank described in some public interface, but which encapsulates loads of state and other objects i…

C# will be getting records in .NET 5, which would be analogous to what you refer as 'immutable data pieces', in addition to the normal objects the language has always had. https://devblogs.microsoft.com/dotnet/announcing-net-5-0-rc-...

Re: Principles of Data Oriented Programming

#29

Post by danielscrubs »

pjmlp wrote: post #9

> One could argue that the complexity of the system where code and data are mixed is due to a bad design and data an experienced OO developer would have designed a simpler system, leveraging smart design patterns. Indeed he (or she) would have made use of traits/protocols/categories/whatever, to separate behavior from data, while keeping the design extensible (via polymorphism). This is something I usually find in OO…

The other parts exist in one form or another in non oop languages too. Heck, polymorphism is part of type theory, traits exists in Ocaml and Haskell... But arguing what and what isn’t oop isn’t that productive as no one will agree to any definition. That’s why you’ll get gut responses about lasagna code where the layering glue is more complex and of bigger proportions than the algorithm itself...

Re: Principles of Data Oriented Programming

#30

Post by omginternets »

Joker_vD wrote: post #23

You know, sometimes I wish the programming languages had a better distinction between "immutable data pieces" and "stateful agents". Sometimes it's really nice to have a simple struct for which you (or anyone else) can write many function to act upon. Sometimes it's really nice to have an opaque "object" with methods to yank described in some public interface, but which encapsulates loads of state and other objects i…

This is probably an imperfect solution, but isn't this kind of what Actors provide? Syntax and concurrency aside, passing a message is equivalent to calling a method.

The biggest annoyance I can foresee is discoverability. Object-orinented classes make it pretty clear what a given object's interface is (just look at its public methods). This is not true of actors in the languages I've used (not many), but in theory it should be a fairly trivial question of syntax.

Post reply on HN