Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Introduction to Data Oriented Design with Rust

jamesmcm.github.io

Post reply on HN
Print view View original

111–120 of 161 posts

Re: An Introduction to Data Oriented Design with Rust

#111

Post by TillE »

munificent wrote: post #88
bfrydl wrote: post #64

Earlier quoted context omitted.

I think this is because of Rust. In my opinion the Rust game dev community is overly fixated on ECS. On the bright side, Rust has some damn good ECS libraries.

> the Rust game dev community is overly fixated on ECS. Not just Rust. The game dev community everywhere is infatuated with ECS. It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your pe…

The problem is that software design in general is an incredibly messy field that's still basically in its infancy. Developers want simple solutions to complex design problems, or at least they want some decent architectural guidelines so they can avoid constantly reinventing the wheel, badly. Remember when MVC was all the rage?

ECS is good though, it's a perfectly solid answer to a lot of thorny design questions. Where problems frequently arise is when you try to jam absolutely everything in your game into the ECS structure. In practice you're probably going to have a lot of data which lives outside the system and is not attached to any entity.

Re: An Introduction to Data Oriented Design with Rust

#112

Post by munificent »

lukebitts wrote: post #110
munificent wrote: post #88

Earlier quoted context omitted.

> the Rust game dev community is overly fixated on ECS. Not just Rust. The game dev community everywhere is infatuated with ECS. It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your pe…

Most of what I see about ECS is how much easier it is to have dynamic behaviors without inheritance, and it is, so I don’t see why it would be bad for newcomers to use it or to have an ecs lib written in js.

> how much easier it is to have dynamic behaviors without inheritance

I think you're getting at the idea that instead of having objects differ in their behavior by overriding methods, you have them differ by having fields bound to objects that implement different behavior.

Assuming I understand you right, that's an excellent insight, but it's just the classic principle:

Favor object composition over class inheritance.

There's nothing new in that idea or anything special to ECS. Do you know how I know? Because that sentence here is directly quoted from page 20 of "Design Patterns", which ECS and DoD are often claimed to be in direct opposition to. :)

Re: An Introduction to Data Oriented Design with Rust

#113

Post by lukebitts »

munificent wrote: post #112
lukebitts wrote: post #110

Earlier quoted context omitted.

Most of what I see about ECS is how much easier it is to have dynamic behaviors without inheritance, and it is, so I don’t see why it would be bad for newcomers to use it or to have an ecs lib written in js.

> how much easier it is to have dynamic behaviors without inheritance I think you're getting at the idea that instead of having objects differ in their behavior by overriding methods, you have them differ by having fields bound to objects that implement different behavior. Assuming I understand you right, that's an excellent insight, but it's just the classic principle: Favor object composition over class inheritance…

> I think you're getting at the idea that instead of having objects differ in their behavior by overriding methods, you have them differ by having fields bound to objects that implement different behavior.

I guess I'm more getting at the idea of changing the game design at any point by adding or removing components, a way to make it easier for devs to cope with changing requirements (and they are always changing ofc), but you are right about favoring composition over inheritance, that by itself is pretty good.

I can't really talk about "things that are often claimed" and from the way you talk about this it seems like you have come across different opinions from mine on what ECS is or its value. Sad to see such a useful pattern get "corrupted", but I suppose that is inevitable.

Re: An Introduction to Data Oriented Design with Rust

#114

Post by kmeisthax »

dragontamer wrote: post #47
Arelius wrote: post #31

Earlier quoted context omitted.

> DOD generally implies a high level of coupling on the system. That's it's biggest weakness. I'm not sure I understand what you mean by DOD generally implies a high level of coupling? Could you perhaps elaborate a bit?

Lets say you have "struct Point {int x; int y;};", and all things are fine and dandy. You write your OOP as such (or your DOD version), and things are going great. Suddenly, you want "struct Point3D {int x; int y; int z;}". How would you change the code? In OOP, maybe you'd compose a Point3D as such: struct Point3D { Point xy; int z; }; You can now "capture" all of the Point xy; functionality, and extended it to a 3r…

Is there any particular case in which you'd want Point and Point3Ds to be indexed interchangably in the same array? This is just as cumbersome in the array-of-structs case: some structs are larger than others in the same array, you need some signalling mechanism to know which structs are and aren't Point3Ds (lest you invite the wrath of your optimizer), etc. In Rust the AOS case would be an Enum of Point and Point3D, and the SOA case would be two Vecs of ints and one Vec of Options.

More generally, this sounds like a language support problem: most languages are built around arrays of structs because it's easier to codegen for. If you're composing structures, just treat the composed-over struct as one very wide field and continue counting your offsets for the next fields down. Need to reference a struct? Just take a pointer to it and all the fields are a fixed offset. Want a bunch of the same struct? Put them next to one another in memory and you can go from one to the next by just bumping a pointer.

What we need is some sort of language support for "intrusive arrays", where you can declare a struct, create an intrusive array of it, and that array is laid out as structs of arrays. When you request an intrusive array, the language would need to recursively construct intrusive arrays for each non-primitive field (e.g. your IntrusiveVec would hold IntrusiveVec plus Vec, which flattens down to 3 Vecs). References would consist of a pointer to the whole array plus an index, and methods on structs (er, "classes", in this case) would need to accept this new pointer-to-intrusive-vector format.

Re: An Introduction to Data Oriented Design with Rust

#115

Post by setr »

cogman10 wrote: post #78
Arelius wrote: post #31

Earlier quoted context omitted.

> DOD generally implies a high level of coupling on the system. That's it's biggest weakness. I'm not sure I understand what you mean by DOD generally implies a high level of coupling? Could you perhaps elaborate a bit?

From what I've seen in DOD systems, there are generally common structures that hold data for the entire application. The entire application has to know about these structures in order to function. Take ECS [1] as an example. In order to create a new entity on the system you might need to talk to a location component, a health component, a velocity component all to register a brand new entity on the system. Now, you m…

I don't know about DOD in general, but something seems fundamentally wrong with your understanding of ECS -- it conflicts with almost all of the reading I've done on it so far (though I can't say much about it in practice)

> But you need to make sure as you are talking to each of the component you are doing the right thing.

I'm not clear on "doing the right thing". If you're looking at eg specs, at creation-time the only thing you're doing is constructing the entity correctly (adding values to the relevant arrays) with valid values. There's no real hooking into anything, or any intelligent behavior really, since you're just treating the entity as nothing more than the values its composed of.

The systems just pick it up "magically", and treat it as it would any other entity -- it doesn't need to know anything really about the new entity, except that all of the necessary components have been slotted, before executing. And if they aren't.. it won't execute, at least not for that entity.

Which I think is almost by definition de-coupled. The system knows nothing about the entity, and the entity knows nothing about the system -- the system just looks for anything that fulfills the correct set of components, and the entity just looks to fulfill all components needed to describe it. What behavior that will lead to... the entity itself has no idea. And of course, new systems and new entities (and new components) can be plugged in without knowing anything about the others.

>As a side note, the coupling of ECS systems is one thing that makes it hard to properly thread games. That's because you've got this globally shared mutable state being accessed through a ton of systems throughout the game. It's frankly impressive that games are threaded at all!

Something seems very wrong here -- bevy, specs, legion etc (rust ecs frameworks) all basically suggest multithreading comes for almost free, because you're being very clear about what you're accessing and when (the systems are defined with the components they rely on, and have access to). So they can represent systems as a dependency graph, and parallelize independent systems appropriately. eg https://specs.amethyst.rs/docs/tutorials/09_parallel_join.ht...

The problem is really the other way around -- since its parallelized for you, the difficulty is in defining an ordered sequence of events (when needed); I believe the simple solution is generally adding a component that's really just a flag, and the more general solution is utilizing event queues and event chaining

specs doesn't iterate the array itself in parallel though, at least not automatically, but there's nothing stopping you (since it's already locked the array on your behalf), hence it provides par_iter

Re: An Introduction to Data Oriented Design with Rust

#116

Post by kbenson »

FullyFunctional wrote: post #101
meheleventyone wrote: post #63

Earlier quoted context omitted.

Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern. DoD is about recognising data use patterns in your software and organising your software architecture around them in a…

It took me a fair amount of searching to establish that ESC refers to https://en.wikipedia.org/wiki/Entity_component_system

That's okay, now that you're familiar with the term, you'll end up noticing it a lot more (or maybe it's just that articles that happen to lend themselves to it showing up in their comments look slightly more appealing now?).

Re: An Introduction to Data Oriented Design with Rust

#117

Post by adamnemecek »

Jtsummers wrote: post #97
narwally wrote: post #93

Earlier quoted context omitted.

Thanks, I knew it was some permutation of those words.

NP. You may also want to check out Norvig's Paradigms of AI Programming if this interests you. A lot of that text involves creating data-driven programs, and some of the benefits (like creating a grammar lets you recognize sentences that satisfy it, or with little extra effort generate sentences from it).

This is a really different definition of data driven.

Re: An Introduction to Data Oriented Design with Rust

#118

Post by AlphaSite »

baryphonic wrote: post #80
iherbig wrote: post #21

It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows. Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD. But while these patterns are interesting and of…

In my understanding of DOD, I'm not sure there really is much of a basis that can be explored in a general way, beyond just adapting to whatever platform you are ultimately targeting. "The data" is supreme (this, along with an information theoretic understanding of "data," should be a hint that something is rotten in the state of DODmark; EDIT: this is extremely dismissive, which is not my intention, and I think DOD…

I think for low level high throughout systems that makes sense, but then how do you go up a level when networking gets involved, dealing with back pressure, load balancing, and lots of other things, etc.

Re: An Introduction to Data Oriented Design with Rust

#120

Post by bfrydl »

swsieber wrote: post #81
bfrydl wrote: post #64

Earlier quoted context omitted.

I think this is because of Rust. In my opinion the Rust game dev community is overly fixated on ECS. On the bright side, Rust has some damn good ECS libraries.

Rust lends itself to ECS a lot more than it lends itself to behavior hierarchies using inheritance. That's an over simplification, but a useful one.

I agree that Rust is more suited to ECS than hierarchies. However, the choice is not between ECS or inheritance. The reason I say that the community is overly fixated on it is that many of the benefits attributed to ECS aren't unique to ECS.
Post reply on HN