An Introduction to Data Oriented Design with Rust • Hacker News Board

Re: An Introduction to Data Oriented Design with Rust

#61

Post by Rusky » Thu, Sep 17, 2020, 7:32 PM UTC

dragontamer wrote: post #47

Arelius wrote: post #31

Earlier quoted context omitted.

> DOD generally implies a high level of coupling on the system. That's it's biggest weakness. I'm not sure I understand what you mean by DOD generally implies a high level of coupling? Could you perhaps elaborate a bit?

Lets say you have "struct Point {int x; int y;};", and all things are fine and dandy. You write your OOP as such (or your DOD version), and things are going great. Suddenly, you want "struct Point3D {int x; int y; int z;}". How would you change the code? In OOP, maybe you'd compose a Point3D as such: struct Point3D { Point xy; int z; }; You can now "capture" all of the Point xy; functionality, and extended it to a 3r…

DOD has simple solutions for this, too.

A popular one is to group your entities by "archetype"- you have two arrays of Point2Ds, one for those without a Z coordinate and one for those with a Z coordinate. Now if you want all Point2Ds, you loop over both arrays; if you want all Point3Ds, you just loop over one. (This generalizes cleanly to larger numbers of "components.")

In some ways, this is actually quite a bit easier to use than OOP composition+inheritance. You don't need to build your compositions up front as classes or types- you just build them directly as values, throw them all into your archetype system that manages the arrays, and query them however you like. It feels similar to a relational database.

Re: An Introduction to Data Oriented Design with Rust

#62

Post by dragontamer » Thu, Sep 17, 2020, 7:33 PM UTC

Rusky wrote: post #61

dragontamer wrote: post #47

Earlier quoted context omitted.

Lets say you have "struct Point {int x; int y;};", and all things are fine and dandy. You write your OOP as such (or your DOD version), and things are going great. Suddenly, you want "struct Point3D {int x; int y; int z;}". How would you change the code? In OOP, maybe you'd compose a Point3D as such: struct Point3D { Point xy; int z; }; You can now "capture" all of the Point xy; functionality, and extended it to a 3r…

DOD has simple solutions for this, too. A popular one is to group your entities by "archetype"- you have two arrays of Point2Ds, one for those without a Z coordinate and one for those with a Z coordinate. Now if you want all Point2Ds, you loop over both arrays; if you want all Point3Ds, you just loop over one. (This generalizes cleanly to larger numbers of "components.") In some ways, this is actually quite a bit eas…

> A popular one is to group your entities by "archetype"- you have two arrays of Point2Ds, one for those without a Z coordinate and one for those with a Z coordinate. Now if you want all Point2Ds, you loop over both arrays; if you want all Point3Ds, you just loop over one. (This generalizes cleanly to larger numbers of "components.")

But now your Point2D code needs to be aware of the Point3D code. Violating the open-closed principle (which is an OOP principle, but one that's extremely useful in software engineering).

You NEVER want to change working code. That's the fundamental basis of the open-closed principle. Figuring out how to extend the code to new functionality ("open to extension"), WITHOUT risking any breaks on old code ("closed to modification"), is core to software engineering principles.

Any "solution" that relies upon changing Point2D is null-and-void to my software engineering brain. You cannot build libraries or reusable code if your solution is "rewrite the old code to support new functionality".

Re: An Introduction to Data Oriented Design with Rust

#63

Post by meheleventyone » Thu, Sep 17, 2020, 7:34 PM UTC

iherbig wrote: post #21

It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows. Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD. But while these patterns are interesting and of…

Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern.

DoD is about recognising data use patterns in your software and organising your software architecture around them in a way that suits the constraints of modern hardware. That’s all.

Re: An Introduction to Data Oriented Design with Rust

#64

Post by bfrydl » Thu, Sep 17, 2020, 7:36 PM UTC

meheleventyone wrote: post #63

iherbig wrote: post #21

It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows. Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD. But while these patterns are interesting and of…

Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern. DoD is about recognising data use patterns in your software and organising your software architecture around them in a…

I think this is because of Rust. In my opinion the Rust game dev community is overly fixated on ECS.

On the bright side, Rust has some damn good ECS libraries.

Re: An Introduction to Data Oriented Design with Rust

#65

Post by jmarcher » Thu, Sep 17, 2020, 7:36 PM UTC

renewiltord wrote: post #34

The cost to legibility for the vectorization seems high. I wish there were a way to get the best of both worlds, i.e. specify the type you want and have it striped so that the struct is then vectorified.

I don't know enough about rust macros, but that might be the answer.

Re: An Introduction to Data Oriented Design with Rust

#66

Post by strictfp » Thu, Sep 17, 2020, 7:38 PM UTC

I share the opinion that DoD is an optimization technique, and that it shouldn't be applied too generously from the get-go; you're essentially optimizing for specific usage patterns, which might be difficult to predict and often are contradictory.

What one can consider is to use ECS. The separation of data and logic into components allows you to apply DoD in a more predictable way. The data which is unique to a particular component can in this way be vectorized across entities.

The ECS approach doesn't solve the problem of data shared between components though, and like others in here I'm also interested in solutions which could optimize access in the same way that relational databases can optimize multiple different access patterns to the same data.

Re: An Introduction to Data Oriented Design with Rust

#67

Post by Rusky » Thu, Sep 17, 2020, 7:42 PM UTC

dragontamer wrote: post #62

Rusky wrote: post #61

Earlier quoted context omitted.

DOD has simple solutions for this, too. A popular one is to group your entities by "archetype"- you have two arrays of Point2Ds, one for those without a Z coordinate and one for those with a Z coordinate. Now if you want all Point2Ds, you loop over both arrays; if you want all Point3Ds, you just loop over one. (This generalizes cleanly to larger numbers of "components.") In some ways, this is actually quite a bit eas…

> A popular one is to group your entities by "archetype"- you have two arrays of Point2Ds, one for those without a Z coordinate and one for those with a Z coordinate. Now if you want all Point2Ds, you loop over both arrays; if you want all Point3Ds, you just loop over one. (This generalizes cleanly to larger numbers of "components.") But now your Point2D code needs to be aware of the Point3D code. Violating the open-…

> But now your Point2D code needs to be aware of the Point3D code.

Not at all! You write the archetype management code once, and then the Point2D code just asks it for Point2Ds, nothing else, and that doesn't change no matter how you use Point2D elsewhere.

Re: An Introduction to Data Oriented Design with Rust

#68

Post by meheleventyone » Thu, Sep 17, 2020, 7:43 PM UTC

bfrydl wrote: post #64

meheleventyone wrote: post #63

Earlier quoted context omitted.

Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern. DoD is about recognising data use patterns in your software and organising your software architecture around them in a…

I think this is because of Rust. In my opinion the Rust game dev community is overly fixated on ECS. On the bright side, Rust has some damn good ECS libraries.

I’d agree to a point but it really crosses the whole gamut of hobby game engine development. It’s weirdly self reinforcing even in the face of more interesting architectural choices like DOOM Eternal’s.

Re: An Introduction to Data Oriented Design with Rust

#69

Post by dragontamer » Thu, Sep 17, 2020, 7:44 PM UTC

Jtsummers wrote: post #60

dragontamer wrote: post #57

Earlier quoted context omitted.

> In the DOD version you'd create a new record containing 3 arrays instead of 2 (x,y,z) and write or extend your functions to support this new record type. But now you're violating DRY: don't repeat yourself. Bugfixes found in Point3D will have to be manually "ported" to Point functions (and vice versa). > In the OOP version (NB: blech, it's not OOP, it's just records) you'd also need new functions to support the new…

So your OOP version creates many layers which impedes understanding and extension as well. DRY is also not a law which must always be followed. You often don't even know when you're repeating yourself until you do, and the first time you see a repetition is not when you should eliminate it (in most cases). You often see people reference the rule of three here. The first repetition should make you pay attention. The s…

If you don't care about DRY in this case, that's fine. There's more than one problem with your proposed DOD solution.

Your proposed solution violates the open-closed principle. Which means you have to modify "old working code" to get anything done.

If your "solutions" to fixing code issues is "rewrite and extend the old code to cover the new case", then it becomes impossible to build libraries.

----------

Lets say ApplicationFoo has been written using Point2D, either using OOP or DOD principles. In the real world, ApplicationFoo is often written by another team outside of your direct control.

Your ApplicationBar wants Point3D, and realizes that Point2D largely implements the functionality you want (2d-distances, line-drawing, intersections, etc. etc.)

Your solution forces you to rewrite Point2D, which therefore creates a change in ApplicationFoo. (Or more likely, ApplicationFoo is going to refuse to update to the new library: ApplicationFoo is now stuck on Point2D 1.0, while ApplicationBar is using Point2D 1.1. And now your organization is no longer sharing code effectively).

Re: An Introduction to Data Oriented Design with Rust

#70

Post by SquishyPanda23 » Thu, Sep 17, 2020, 7:45 PM UTC

jessermeyer wrote: post #14

cogman10 wrote: post #10

Earlier quoted context omitted.

DOD generally implies a high level of coupling on the system. That's it's biggest weakness. It also pushes even more data management problems onto programmers which makes it easy to get things wrong. You take those downsides and you trade them for higher performance. Now, that doesn't mean that you can't have hybrid systems and get most of the benefits of both worlds. It does, however, mean that you will end up with…

'The hardware is the platform' is the DOD mantra. It's not a weakness to integrate more information about the problem space into the solution. It's engineering. To _ignore_ information about the problem space is certainly a weakness, and is partly responsible why the Windows boot time appears to be a cosmological constant.

> It's not a weakness to integrate more information about the problem space into the solution. It's engineering.

You want your solution to depend on the right abstract model of the problem, not on the particulars of the problem.

Otherwise your solution is difficult to extend, and is generally expensive and error prone to change when your problem changes, which happens with 100% probability.