Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Introduction to Data Oriented Design with Rust

jamesmcm.github.io

Post reply on HN
Print view View original

101–110 of 161 posts

Re: An Introduction to Data Oriented Design with Rust

#101

Post by FullyFunctional »

meheleventyone wrote: post #63
iherbig wrote: post #21

It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows. Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD. But while these patterns are interesting and of…

Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern. DoD is about recognising data use patterns in your software and organising your software architecture around them in a…

It took me a fair amount of searching to establish that ESC refers to https://en.wikipedia.org/wiki/Entity_component_system

Re: An Introduction to Data Oriented Design with Rust

#102

Post by meheleventyone »

barrkel wrote: post #99
munificent wrote: post #88

Earlier quoted context omitted.

> the Rust game dev community is overly fixated on ECS. Not just Rust. The game dev community everywhere is infatuated with ECS. It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your pe…

I wouldn't dismiss the JS ECS frameworks without measurement. Polymorphism has costs, and while dynamic languages work hard to remove them, they still work best when they're able to monomorphize the call site, because that enables inlining without combinatoric explosion from chained polymorphic calls. Having a single type in your array means field accesses, method calls etc. have the potential to be monomorphized. Th…

I don’t believe any JS engines do any “shape” or “hidden class” caching other than for call site polymorphism?

Re: An Introduction to Data Oriented Design with Rust

#103

Post by [deleted] »

strictfp wrote: post #66

I share the opinion that DoD is an optimization technique, and that it shouldn't be applied too generously from the get-go; you're essentially optimizing for specific usage patterns, which might be difficult to predict and often are contradictory. What one can consider is to use ECS. The separation of data and logic into components allows you to apply DoD in a more predictable way. The data which is unique to a parti…

[deleted]

Re: An Introduction to Data Oriented Design with Rust

#104

Post by narwally »

Jtsummers wrote: post #97
narwally wrote: post #93

Earlier quoted context omitted.

Thanks, I knew it was some permutation of those words.

NP. You may also want to check out Norvig's Paradigms of AI Programming if this interests you. A lot of that text involves creating data-driven programs, and some of the benefits (like creating a grammar lets you recognize sentences that satisfy it, or with little extra effort generate sentences from it).

I'm actually working through SICP for the second time right now[1]. This looks like a great place to go after I finish that. I've been meaning to roll my sleeves up and finally dig into the Common Lisp ecosystem, so far it's been mostly Clojure, emacs-lisp, and a bit of Guile[2]. The incredible depth and breadth of the language, and the fact that good documentation seems to be split across all of its various implementations, makes approaching it feel like swimming into the deep end without your water wings.

[1] The first time was my introduction to lisp, so only bits of it stuck.

[2] A criminally underrated language IMHO| "Knit, Chisel, Hack: Building Programs in Guile Scheme" by Andy Wingo - https://www.youtube.com/watch?v=uwiaT3MoDVs

Re: An Introduction to Data Oriented Design with Rust

#105

Post by Haga »

baryphonic wrote: post #80
iherbig wrote: post #21

It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows. Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD. But while these patterns are interesting and of…

In my understanding of DOD, I'm not sure there really is much of a basis that can be explored in a general way, beyond just adapting to whatever platform you are ultimately targeting. "The data" is supreme (this, along with an information theoretic understanding of "data," should be a hint that something is rotten in the state of DODmark; EDIT: this is extremely dismissive, which is not my intention, and I think DOD…

Dod taken to its extremes, is to give up on alot of other concepts like avoiding data duplicates. Usually you build a oop prototype to identify the hot loop and then build datastructures optimized for this process. Imagine it like a chemical plant that no longer receives one type per train or wagon, but instead processor batch sized pre mixed ingredients on wagons, that might even be reused as storage and staging for the refinery traveling overhead. A classic oop objects attributes might be distributed over several of such data wagons and the problem here is synchronization and updating. So what if your Algo processed it fast but is bow stuck waiting for another to sync its data back to arrays of optimal accessstructs.

Re: An Introduction to Data Oriented Design with Rust

#106

Post by fsociety »

adamnemecek wrote: post #19
fsociety wrote: post #12

Earlier quoted context omitted.

Modern React/Relay with GraphQL fragments does this semi-well I think. Sources of data include the fragment in their GraphQL query and can blindly pass that to components which depend on it. That lets you re-use components quickly and easily. Changing the downstream component which consumes the fragment just requires changes in that component and nothing else.

Modern React meaning Hooks?

I like Hooks a lot but I’m not sure how it’d apply in the Rust UI sense (maybe well, maybe not). It is nice to not be forced into building classes and to also ensure that specific state changes will force partial re-renders.

I’m talking more about https://relay.dev/docs/en/fragment-container. Creating opaque types that can be fetched from a data source and passed off to a child component is more what I am thinking of.

I mention it because I recall working on a Qt product where you have to parse data from a source, then you bring it throughout your program and try to keep it in the right form for every UI component that uses it.

It was easy for the project but we had no composability of components.

With fragments, you don’t really care and duplication of data between fragments on the same source of data generally just works. That way you can pass the fragments to each component as required.

The idea behind that being that, you can have “efficient” and opaque data structure received from a data source which can be blindly passed to a component to consume.

I say data source but in this instance I mean GraphQL... so maybe it doesn’t apply to a generic UI library.

Re: An Introduction to Data Oriented Design with Rust

#107

Post by jbjbjbjb »

blovescoffee wrote: post #3

I've read quite a few articles about DOD. I get how DOD is great but there are reasons OOP is still around (surely?) I suppose I'm still hung up on, what are the benefits of OOP that I'm missing? It seems like there are reasons OOP is so ingrained. I can see a couple of reasons that are environmental. For instance, OOP is taught in schools because students are often taught to program before they're taught about compu…

OOP isn’t necessarily slower. One thing you get is lazy evaluation for free. There’s no better performance gain that no computation. It’s also much easier to extend the types than in functional programming (where it is easier to extend the functions). I find OOP is a lot better for programming in the large too, interfaces and other abstractions are really useful as a project grows.

Re: An Introduction to Data Oriented Design with Rust

#108

Post by hu3 »

hu3 wrote: post #17

This is known and heavily used by the game industry for a long time now. Ultimately it comes down to data locality and there's no silver bullet. Performance varies with your data access pattern and code must be adapted to keep caches as warm as possible. There are also maintainability trade-offs. In general I'd advise this kind of optimization only for performance-critical code like games. For more information on how…

I like that my constructive comment is currently downvoted even below another that reads "Rust is a scam". Absolute state of the language's community. And yes, it happens specially on rust related posts. I'm not the first to write about this.

Someone else's comment about jai received similar treatment. Good thing that on important technologies their communities tend to be friendly. As a niche language striving to become relevant, I'd adopt similar approach.

As a consultant I get asked almost on a daily basis: what technology could solve my problem? Community is heavily weighted in that decision because once I deliver the project, my clients will invariably have to deal with the tech communities.

Regardless, maturity problems like this currently rule it out anyway: https://www.reddit.com/r/rust/comments/iab9iv/i_must_be_doin...

Re: An Introduction to Data Oriented Design with Rust

#109

Post by munificent »

barrkel wrote: post #99
munificent wrote: post #88

Earlier quoted context omitted.

> the Rust game dev community is overly fixated on ECS. Not just Rust. The game dev community everywhere is infatuated with ECS. It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your pe…

I wouldn't dismiss the JS ECS frameworks without measurement. Polymorphism has costs, and while dynamic languages work hard to remove them, they still work best when they're able to monomorphize the call site, because that enables inlining without combinatoric explosion from chained polymorphic calls. Having a single type in your array means field accesses, method calls etc. have the potential to be monomorphized. Th…

> I wouldn't dismiss the JS ECS frameworks without measurement.

I think the burden of proof is on the part of JS ECS frameworks to show they do have better performance by virtue of DoD and, if so, why. JS engine programmers have been optimizing object-oriented code for literally forty years, all the way back to when they were making Smalltalk VMs.

If somehow a couple of folks hacking on ECS frameworks have managed to write code that runs faster on those JS engines than the kind of code they were designed for, I'd like to see it.

> Having a single type in your array means field accesses, method calls etc. have the potential to be monomorphized.

Sure, but object-oriented code does not require any more polymorphism than DoD does. Consider:

* Iterate over an array of monomorphic components and call a method on each one.

* Iterate over an array of monomorphic entities, access a monomorphic property, and call a method on the latter.

There's an extra property access in the latter (which can easily be inlined), but no polymorphic dispatch. In practice, yes, it is possible to reorganize your JavaScript code in ways that play nicer with inline and possibly even code caching. But I have never seen any evidence that JS ECS frameworks actually do that. Instead, the few I've poked around in seem like typical slow imperative dynamically-typed JS.

If someone is going to take a pattern that was invented specifically for a language like C++ that gives you precise control over memory layout and then apply it to a language that not doesn't give you that control but often uses hash tables to store an object's state, I think the burden of proof is on the framework to show that the pattern actually applies.

Re: An Introduction to Data Oriented Design with Rust

#110

Post by lukebitts »

munificent wrote: post #88
bfrydl wrote: post #64

Earlier quoted context omitted.

I think this is because of Rust. In my opinion the Rust game dev community is overly fixated on ECS. On the bright side, Rust has some damn good ECS libraries.

> the Rust game dev community is overly fixated on ECS. Not just Rust. The game dev community everywhere is infatuated with ECS. It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your pe…

Most of what I see about ECS is how much easier it is to have dynamic behaviors without inheritance, and it is, so I don’t see why it would be bad for newcomers to use it or to have an ecs lib written in js.
Post reply on HN