An Introduction to Data Oriented Design with Rust • Hacker News Board

Re: An Introduction to Data Oriented Design with Rust

#11

Post by mdtusz » Thu, Sep 17, 2020, 5:37 PM UTC

adamnemecek wrote: post #4

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

I think oop is easier to understand in small examples but gets really inscrutable as your code base grows so you only start questioning oop if you have worked on large code bases. Also the lack of dod/ecs languages is a problem. I realized the other day that row polymorphism is the type theoretical concept that would make this work. https://en.wikipedia.org/wiki/Row_polymorphism

Partially, but DOD like in the article examples work great when you have a "rigid" architecture like a game where having a struct to contain vecs/arrays of player attributes makes sense. This can break down when your "objects" require more flexibility and are used in multiple different contexts where an aggregate `PlayerState` struct doesn't play nicely.

Not to say that DOD is bad, just that it shouldn't always be the go-to choice. Rust _does_ make this sort of design much nicer to work with because you can in many ways have your cake and eat it too using `from` and `into` and go back and forth from DOD to OOP type structures.

Re: An Introduction to Data Oriented Design with Rust

#12

Post by fsociety » Thu, Sep 17, 2020, 5:38 PM UTC

adamnemecek wrote: post #6

A lot of people in the Rust community (yours truly including) are trying to work out how to build UI frameworks in this paradigm. It's a really interesting question, if anyone has any good input, I'm alll ears.

Modern React/Relay with GraphQL fragments does this semi-well I think. Sources of data include the fragment in their GraphQL query and can blindly pass that to components which depend on it.

That lets you re-use components quickly and easily. Changing the downstream component which consumes the fragment just requires changes in that component and nothing else.

Re: An Introduction to Data Oriented Design with Rust

#13

Post by adamnemecek » Thu, Sep 17, 2020, 5:40 PM UTC

cogman10 wrote: post #10

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

DOD generally implies a high level of coupling on the system. That's it's biggest weakness. It also pushes even more data management problems onto programmers which makes it easy to get things wrong. You take those downsides and you trade them for higher performance. Now, that doesn't mean that you can't have hybrid systems and get most of the benefits of both worlds. It does, however, mean that you will end up with…

DOD couples less than OOP. What is data management in this context?

Re: An Introduction to Data Oriented Design with Rust

#14

Post by jessermeyer » Thu, Sep 17, 2020, 5:41 PM UTC

cogman10 wrote: post #10

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

DOD generally implies a high level of coupling on the system. That's it's biggest weakness. It also pushes even more data management problems onto programmers which makes it easy to get things wrong. You take those downsides and you trade them for higher performance. Now, that doesn't mean that you can't have hybrid systems and get most of the benefits of both worlds. It does, however, mean that you will end up with…

'The hardware is the platform' is the DOD mantra.

It's not a weakness to integrate more information about the problem space into the solution. It's engineering. To _ignore_ information about the problem space is certainly a weakness, and is partly responsible why the Windows boot time appears to be a cosmological constant.

Re: An Introduction to Data Oriented Design with Rust

#15

Post by jayd16 » Thu, Sep 17, 2020, 5:42 PM UTC

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

They're not really exclusive concepts and you don't want to bother with DOD if your n is low.

I would go as far as to say most DOD implementations are done along side the use of objects.

Re: An Introduction to Data Oriented Design with Rust

#16

Post by knowuh » Thu, Sep 17, 2020, 5:43 PM UTC

For a 2x speed up, I am not sure I would be willing to sacrifice legibility as in the example. The OOP method definition reads like english.

The article suggests the true benefits of DOP aren't all that great unless you understand the target architecture.

I feel like the pendulum is at its new zenith.

Re: An Introduction to Data Oriented Design with Rust

#17

Post by hu3 » Thu, Sep 17, 2020, 5:44 PM UTC

This is known and heavily used by the game industry for a long time now.

Ultimately it comes down to data locality and there's no silver bullet. Performance varies with your data access pattern and code must be adapted to keep caches as warm as possible.

There are also maintainability trade-offs. In general I'd advise this kind of optimization only for performance-critical code like games.

For more information on how game development approaches this, read about Entity Component Systems. Here's a good start: https://www.youtube.com/watch?v=JxI3Eu5DPwE

Re: An Introduction to Data Oriented Design with Rust

#18

Post by ekidd » Thu, Sep 17, 2020, 5:49 PM UTC

knowuh wrote: post #16

For a 2x speed up, I am not sure I would be willing to sacrifice legibility as in the example. The OOP method definition reads like english. The article suggests the true benefits of DOP aren't all that great unless you understand the target architecture. I feel like the pendulum is at its new zenith.

As always, it depends on the problem you're solving.

A 2x speedup in the inner loops of a game can be a very big deal. And for business workloads, I occasionally spend time babysitting clusters running batch jobs. A 2x performance increase in the right inner loop might save a couple hundred dollars per run.

So certainly, profile before you optimize, as the article demonstrates. But 2x speedups can be worth applying a "struct of arrays" transform.

Re: An Introduction to Data Oriented Design with Rust

#19

Post by adamnemecek » Thu, Sep 17, 2020, 5:51 PM UTC

fsociety wrote: post #12

adamnemecek wrote: post #6

A lot of people in the Rust community (yours truly including) are trying to work out how to build UI frameworks in this paradigm. It's a really interesting question, if anyone has any good input, I'm alll ears.

Modern React/Relay with GraphQL fragments does this semi-well I think. Sources of data include the fragment in their GraphQL query and can blindly pass that to components which depend on it. That lets you re-use components quickly and easily. Changing the downstream component which consumes the fragment just requires changes in that component and nothing else.

Modern React meaning Hooks?

Re: An Introduction to Data Oriented Design with Rust

#20

Post by mamcx » Thu, Sep 17, 2020, 5:52 PM UTC

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

Is more than OOP. Look at this as the difference between OLAP/OLTP databases: DOD is OLAP(Columnar) that make easy to do "query" by columns but awkward to do piece meal updates and OOP/Row-nar(?) code stay because frankly is the default everywhere. And far easier for make updates.

Iterate by rows is far more common that by just "a column". Also, constant update of the rows is more common that have a static, full materialization of the data that only need to be queried.

So, doing DOD only work in some scenarios, but do "by rows" is the common one.

P.D: I tried to go full columnar building a relational language, but get very obvious very fast that it not work for the general case. Is telling that kdb+ (probably the biggest proponent in the area) work in a OLAP-like niche...