Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Thought Experiment: Using the ECS Pattern Outside of Game Engines

adventures.michaelfbryan.com

Post reply on HN
Print view View original

41–50 of 75 posts

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#41

Post by TeMPOraL »

BubRoss wrote: post #34
TeMPOraL wrote: post #9

Which ECS? :). Despite "clear" definition from Wikipedia, ECS as a pattern suffers from multiple personality disorder. There are several different goals that all share the same name, and different implementations pick a different goal set, ending up looking not quite the same, and getting different kinds of benefits. So outside games, if you pick "ECS as composition over inheritance", you get something like here, or…

I have gone down the exact same line of thinking. I think there is value in creating a simple relational data structure to have simple select and insert functions but not the overhead of sqlite. Many times when I think about it, the state of most programs could use a handful of the same structure and be done.

Yeah, I think so too. My goal after hopefully making a completely playable game running on SQLite is to eventually rip it out and replace with my own ECS, based on gained experience. I start with explicit SQL queries, and am slowly building an interface on top of it. Or rather, I'm moving towards the interface I abandoned prior to switching to SQL.

Before SQLite, I was working on my (yet another) own implementation, and eventually realized that I don't like the "Systems" part of the ECS, given I planned a lot of logic that required inspecting multiple components of any given entity. Thinking about it, I figured out that a better abstraction for a "System" would be a piece of code executed on a set of entities, defined by some selection logic. That is, a query. I quickly ended up with this interface:

  (select-entities 
     :components (has-position has-health (not player))
     :where ((entity-id (has-health (hit-points hp)))
             (
(The example selects non-player entities with position and health components which have less than 50 HP, and orders them by HP and, if both compared entities have a "targetable" component, also by targeting priority.)

Which, as you can see, looks essentially like an SQL query. :components is select + join, :where is filtering condition, :order-by saves me from doing an explicit sort on the results, and the :into my-array part allows this query to not allocate new storage for results each frame.

Now each time such call showed up in my code, some extra book keeping storage would be allocated, and underlying logic ensured that most of the query doesn't have to be recomputed every frame. For instance, the :where part was handled by tracking addition and deletion of components to/from an entity. Implementing similar optimizations for :order-by part, I quickly realized that most of my "book keeping" tricks are essentially equivalent to compiled queries and indices on tables. At which point I ditched it all and switched to SQLite.

(I meant to write a blog post about this at some point. I guess I just wrote half of it here.)

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#43

Post by almindor »

I've built the Texel ASCII Art Editor (https://crates.io/crates/texel) using Specs (https://crates.io/crates/specs) as my first experience with ECS.

The original goal was an ASCII game but since I needed to create the resources for it it morphed into Texel. I think the use of ECS here wasn't a bad decision but Specs proved to be just too cumbersome and overoptimizing.

As others have mentioned ECS is a bit "loosly defined" and each implementation seems to go over the line to add more specializing one way or the other. I want to switch to something more simple and elegant like DCES (https://crates.io/crates/dces) for my next refactor.

I think for "runtime resource management" ECS is fine if you need a fairly large, distinct pool of entities to handle. One of my main usage problems was the re-use of same components in the same entity.

E.g. imagine having an entity with a global world position, but also an internal position for something like "last cursor position". It's not possible to just "add another Position component" to the same entity, for good internal reasons, but still. It's pretty important people take these kind of limitations into account before planning out their entity/component maps.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#44

Post by imtringued »

TeMPOraL wrote: post #9

Which ECS? :). Despite "clear" definition from Wikipedia, ECS as a pattern suffers from multiple personality disorder. There are several different goals that all share the same name, and different implementations pick a different goal set, ending up looking not quite the same, and getting different kinds of benefits. So outside games, if you pick "ECS as composition over inheritance", you get something like here, or…

Last time I built an ECS it was effectively a bare bones relational database. A system just queries the information that it needs. The code basically looked like this. There was no need for explicit where or select queries because all I really needed to check was the presence of a component. Here is a contrived example. ecs has an inner join function that accepts all components (stored in an array) that you want to join and everything gets passed to a callback that receives all the components.

    void autoheal_system() {
         ecs.inner_join(ecs.status, ecs.inventory, ..., [](status, inventory, ...){
             if(status.health
One big advantage with this system based implementation is that you are processing all entities in batches compared to the naive OOP style where executing entity.foo() may run Dog.foo or Cat.foo and thrash the instruction cache and cause branch misprediction because of the virtual function dispatch that is constantly switching between functions. I haven't done parallelism with this pattern yet but as long as you do not cross reference other entities it shouldn't be too difficult to just slap on an OpenMP pragma and have a parallel_inner_join function that runs everything on multiple cores.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#45

Post by imtringued »

codetrotter wrote: post #16
TeMPOraL wrote: post #9

Which ECS? :). Despite "clear" definition from Wikipedia, ECS as a pattern suffers from multiple personality disorder. There are several different goals that all share the same name, and different implementations pick a different goal set, ending up looking not quite the same, and getting different kinds of benefits. So outside games, if you pick "ECS as composition over inheritance", you get something like here, or…

> If you go for "ECS as a performance optimization" - a frequently touted benefit - you'll end up storing a lot of global arrays (perhaps arranged in structures), each packing every instance of a component's property across all entities. This is basically what I am doing in the backend I am writing for an app that I am working on. (Also not a game btw.) No database, no overhead :D You really can fit a lot of data in…

You always need a memory hierarchy. Not even DRAM is fast enough for our CPUs so we have L3 caches and those are not fast enough either so we also have L2 and L1 caches and registers and physical wires which temporarily store the state of the processor by taking advantage of propagation delays.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#46

Post by skybrian »

_carl_jung wrote: post #39
spankalee wrote: post #26

So many of these kinds of OOP modeling complaints and solved by mixins.

I'd earnestly like to hear the responses to this from the down-voters. I had the same thought and suspect there is a good reason not to use mixins. One problem off the top of my head is run-time changing of components. An entity that inherits from multiple mixins can't inherit from new mixins (or lose existing ones) at runtime.

Yes, ECS is usually about runtime changes in behavior. It also uses IDs rather than hard references, like in a database. Deleting an object makes references to it invalid, rather than references keeping objects alive like in a functional or object-oriented object graph.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#47

Post by rstrduck »

TeMPOraL wrote: post #41
BubRoss wrote: post #34

Earlier quoted context omitted.

I have gone down the exact same line of thinking. I think there is value in creating a simple relational data structure to have simple select and insert functions but not the overhead of sqlite. Many times when I think about it, the state of most programs could use a handful of the same structure and be done.

Yeah, I think so too. My goal after hopefully making a completely playable game running on SQLite is to eventually rip it out and replace with my own ECS, based on gained experience. I start with explicit SQL queries, and am slowly building an interface on top of it. Or rather, I'm moving towards the interface I abandoned prior to switching to SQL. Before SQLite, I was working on my (yet another) own implementation,…

I’m intrigued by your approach! If you do end up writing that blog post and/or releasing the source, I’d be interested in learning more.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#48

Post by TeMPOraL »

rstrduck wrote: post #47
TeMPOraL wrote: post #41

Earlier quoted context omitted.

Yeah, I think so too. My goal after hopefully making a completely playable game running on SQLite is to eventually rip it out and replace with my own ECS, based on gained experience. I start with explicit SQL queries, and am slowly building an interface on top of it. Or rather, I'm moving towards the interface I abandoned prior to switching to SQL. Before SQLite, I was working on my (yet another) own implementation,…

I’m intrigued by your approach! If you do end up writing that blog post and/or releasing the source, I’d be interested in learning more.

Drop me an e-mail (address in my profile), I'll let you know when the post is up.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#49

Post by jayd16 »

fulafel wrote: post #40
jayd16 wrote: post #14

Earlier quoted context omitted.

This is just false unless you think the featured article isn't ECS. Rendering is an ordered operation that at the very least shouldn't be considered parallel by default. Simply putting it in an ECS pattern doesn't guarantee anything like that. Its probably better to say it might help you write tighter loops because (ideally) you have a small amount of code looping over a large array of data, instead of a sea of actor…

Is rendering commonly closely coupled to the ECS? Rendering involves a lot of spatial trees, sorting, and indeed dealing with the nontrivial problem of parallelizing the interdependent rendering work items. It would seem to call for more specialized data structures.

It not. That's my point. You can't just throw things in the pattern and make them parallelized.

Re: A Thought Experiment: Using the ECS Pattern Outside of Game Engines

#50

Post by jayd16 »

kevingadd wrote: post #28
jayd16 wrote: post #14

Earlier quoted context omitted.

This is just false unless you think the featured article isn't ECS. Rendering is an ordered operation that at the very least shouldn't be considered parallel by default. Simply putting it in an ECS pattern doesn't guarantee anything like that. Its probably better to say it might help you write tighter loops because (ideally) you have a small amount of code looping over a large array of data, instead of a sea of actor…

Rendering on modern hardware is fundamentally parallel by default, even if the commands you issue appear to be sequential. In practice multiple commands can be issued in parallel by a modern GPU and fragments are rasterized in parallel as well (divide and conquer), see https://youtu.be/Nc6R1hwXhL8?t=465 and note how it's chunking many triangles up into groups and rasterizing them in parallel (there's a predictable sp…

Ehh...I see what you mean but... A lot of tricks have gone into the render pipeline to get pixels drawn in parallel but a lot of rendering is fundamentally ordered. Rasters, blends, grab passes etc. and other non communicative operations are done in sequence even if they can be done in parallel per pixel. And these are just the required cases. Plenty of time you want to order things for performance reasons.

>Most of my current parallelization is explicit ordering of scene elements which allows me to prepare buffers/draw commands in parallel, and filling GPU buffers in parallel. If

That's not naturally parallel. You did all that work to know the order of the object before hand. Its not naturally parallel like, say, functional programming with no side effects is naturally parallel.

And you've hyper focused on the 3D render pipeline when many 2D frameworks have a single Layout/UI thread.

And the major point is none of that has to do with ECS!

Post reply on HN