Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases are the endgame for data-oriented design

spacetimedb.com

Post reply on HN
Print view View original

21–30 of 157 posts

Re: Databases are the endgame for data-oriented design

#21

Post by crazygringo »

deadbabe wrote: post #19
crazygringo wrote: post #15

Earlier quoted context omitted.

> But it is just that SQL is the most terse and standard way so express logic. As someone who has written a ton of complex SQL... I couldn't disagree more. Trying to shoehorn things that can trivially and intuitively be expressed in a couple of for-loops with a couple of variables, into SQL expressions making use of joins and aggregate functions and GROUP BY's and (god forbid) correlated subqueries... having to repli…

Are you using Postgres? It seems like you haven’t discovered CTEs or lateral joins.

Oh, I've worked with all the major engines. To which we can add: trying to remember which seemingly basic features are supported by which engines, and which aren't (e.g. percentiles are great in Postgres, but you're SOL in MySQL). That there's virtually nothing "standard" about SQL syntax or functionality at all, except for the simplest of queries and the general basic concepts.

Re: Databases are the endgame for data-oriented design

#23

Post by munificent »

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff.

At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and make your game loop faster? Yes. Will it make your teeth whiter and your partner love you more? No.

(There are ECS frameworks in JavaScript, which gives you absolutely no control over memory layout and thus completely defeats one of the primary purposes of the pattern.)

Like any pattern, it exists to solve concrete problems. It shouldn't be the One True Way To Think About All Programming Henceforth and Forever.

When the author says things like:

> For example, how would you model chat messages in your game? I suppose you’d have to represent that as an entity in your game. How would you represent a constraint that would prevent a health component from being added to your chat message erroneously? In ECS it’s straightforward to create a system which operates on chat messages individually, but how would you query your chat messages so that you can display them in order?

To me, that just means "Don't use ECS for those." I have a really nice coffee mug that is just perfect for holding coffee. It does its job very well. That does not mean I feel any need to use that coffee mug for digging holes in my garden.

Databases and the relational model are great. ECS is great. Object-oriented programming is great. Functional programming is great. But treat them all as tools that should be used for the right job.

Re: Databases are the endgame for data-oriented design

#24

Post by da_chicken »

hitpointdrew wrote: post #12

Nope, nope and nope. Went to the github page for spacetimedb, it does everything that is terrible. >Instead of deploying a web or game server that sits in between your clients and your database, your clients connect directly to the database and execute your application logic inside the database itself. You can write all of your permission and authorization logic right inside your module just as you would in a normal…

> Databases should never, ever, ever, be used to perform logic, they are datastores, that is it.

I wouldn't go that far. Relational algebra is performing logic. Constraints and foreign keys are logic, as well.

I'm not going to argue that you should go back in time 15-20 years and start shredding XML strings in stored procedures again. But thinking of the database as one step above a flat file is similarly backward thinking.

More than that, the concept of putting application logic adjacent to the data store is sound. That's exactly what a web API or a microservice is doing. They allow a uniform mechanism of requests and responses to a data store. At a concept level, that's exactly the same thing. The concept of keeping logic at or immediately adjacent to the data layer so that a whole range of disparate applications can request and maintain data from the source is what the design goal of "database side logic" is.

Re: Databases are the endgame for data-oriented design

#25

Post by acdha »

deadbabe wrote: post #17
hitpointdrew wrote: post #12

Nope, nope and nope. Went to the github page for spacetimedb, it does everything that is terrible. >Instead of deploying a web or game server that sits in between your clients and your database, your clients connect directly to the database and execute your application logic inside the database itself. You can write all of your permission and authorization logic right inside your module just as you would in a normal…

Can you keep an open mind? We’ve used stored procedures for years. It has worked wonderfully for creating a single source and producer of truth for business data. Instead of potentially having business logic across multiple repos and deployments, everything exists in one place, with absolute unquestionable authority. It’s not difficult to debug at all, you might just be unskilled.

> It’s not difficult to debug at all, you might just be unskilled.

I agree with your larger point but this seems too harsh: it’s definitely harder to debug simply because, as with microservices, understanding how the app is functioning now requires you to understand different code in multiple languages and locations, you’re highly likely to hit non-portable behavior across databases for authoring and debugging, and you’re never going to get a debugger with the whole flow in context.

That doesn’t mean there aren’t benefits as well and it could be especially useful as a way to force distinctions about contracts for common operations, but I wouldn’t say it’s right for all or even most projects. The sweet spot is going to vary widely.

Re: Databases are the endgame for data-oriented design

#26

Post by jerf »

munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s."

I appreciate that most of the ECS hype has been around specific use cases, though.

OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded off the original proposals (particularly the idea that objects should only and exactly match real world objects, an idea which I believe in hindsight was accidentally carried over from a simulation worldview in Simula where maybe it worked into the general programming world where it didn't), but it certainly has failed to be the one true programming methdology.

I haven't seen anyone claiming everything should be rewritten in ECS.

Re: Databases are the endgame for data-oriented design

#27

Post by meheleventyone »

munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

This. To me a well engineered game from a data perspective is a set of seperate datastores optimized for the job they are doing referencing one another through handles. I can see the drive to represent this in a general purpose way but you nearly always lose performance and/or flexibility. Ironically the “where do I put chat messages in my ECS” example illustrates that nicely.

Re: Databases are the endgame for data-oriented design

#28

Post by fifilura »

crazygringo wrote: post #15
fifilura wrote: post #4

My colleagues hate me, but I also found that SQL is The way to write business logic. Lots of caveats about difficulty to test and weird syntax. But it is just that SQL is the most terse and standard way so express logic. And that in itself is the most important factor to avoid bugs. Not what testing strategy you choose.

> But it is just that SQL is the most terse and standard way so express logic. As someone who has written a ton of complex SQL... I couldn't disagree more. Trying to shoehorn things that can trivially and intuitively be expressed in a couple of for-loops with a couple of variables, into SQL expressions making use of joins and aggregate functions and GROUP BY's and (god forbid) correlated subqueries... having to repli…

I am happy for all the disagreeing posts and that HN is a place for it.

It also helps me understand my colleagues.

It may be how our brains are wired. For me whenever I see a for loop I see bugs.

It is also about that you get the concurrency from map/reduce-type jobs for free without having to think about it.

But yeah there are a couple of places you need to be careful. Avoiding duplicates and handling of NULL.

Re: Databases are the endgame for data-oriented design

#29

Post by hyperpape »

jerf wrote: post #26
munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s." I appreciate that most of the ECS hype has been around specific use cases, though. OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded…

> I appreciate that most of the ECS hype has been around specific use cases, though.

I recently watched the first 30 minutes of Mike Acton's 2014 talk, and while that portion of the talk wasn't about ECS specifically, it very much presented an absolutist perspective.

Re: Databases are the endgame for data-oriented design

#30

Post by stvltvs »

deadbabe wrote: post #19
crazygringo wrote: post #15

Earlier quoted context omitted.

> But it is just that SQL is the most terse and standard way so express logic. As someone who has written a ton of complex SQL... I couldn't disagree more. Trying to shoehorn things that can trivially and intuitively be expressed in a couple of for-loops with a couple of variables, into SQL expressions making use of joins and aggregate functions and GROUP BY's and (god forbid) correlated subqueries... having to repli…

Are you using Postgres? It seems like you haven’t discovered CTEs or lateral joins.

I agree that CTEs are a solution to some of the pain listed above, but CTEs aren't exotic. Lots of the major RDBMSs support them.
Post reply on HN