Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases are the endgame for data-oriented design

spacetimedb.com

Post reply on HN
Print view View original

121–130 of 157 posts

Re: Databases are the endgame for data-oriented design

#121

Post by sarreph »

gorgoiler wrote: post #112

Has anyone here seen a database where your users are users in the db itself? Not just a user(id, name, email, password) table but actual db users with GRANTed permissions and ACLs etc set appropriately, and open access to the DB for these users. It seems like it would solve a lot of problems by eliminating the need for data broker apps / endpoints that simply put POST or GET parameters into different SQL queries base…

Yes, Firebase / Firestore has this capability. You grant access to collections (and even fields) using rulesets[0].

Firestore is a NoSQL db so it's a quite different paradigmatically though.

[0] - https://firebase.google.com/docs/firestore/security/get-star...

Re: Databases are the endgame for data-oriented design

#122

Post by matesz »

munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

I dedicated years to full-time work on unifying the architecture for general software development.

ECS, OOP, functional programming, and others serve as methods to organize software — the developer interface. However that role should belong to IDEs — an actual user interface for developers.

Everybody has different hierarchy of their software, esentially developing their own software engine which sucks, because doing it right is extremely hard and time consuming.

What Steve Jobs told long time ago is still true today, "Paradoxically, we need more sophisticated software to make it more easy for the user" (paraphrasing). This is what we need - trully general software engine with IDE made right, what will unify organization of all sofotware and beyond.

For some deeper insights, in the context of the Web, I suggest reading Graydon Hoare's post [1].

[1] https://types.pl/@graydon/110561036237098295

Re: Databases are the endgame for data-oriented design

#123

Post by camgunz »

manicennui wrote: post #68
camgunz wrote: post #60

Earlier quoted context omitted.

You can define functions in all major databases

And how many non-technical users do you know who do this?

I think SWEs think of databases only as a kind of generic persistence layer and aren't super interested in a lot of the details (more than one has said to me databases are just an implementation detail) or additional capabilities, which I think is a very limited view. Databases are essentially interpreters, just like Python or Ruby. They come with access to a multiuser persistence (etc) service or engine (super handy!) but offer all the affordances of other interpreters. In fact, plpython on PostgreSQL will just use the Python packages on your system.

Do you have to read about it? Sure, just like all programming languages and platforms. But, interestingly, SQL was designed for non-technical analysts, so (to your point) it's more accessible than those others.

Re: Databases are the endgame for data-oriented design

#124

Post by tlarkworthy »

munificent wrote: post #105
tlarkworthy wrote: post #86

Earlier quoted context omitted.

I don't think memory layout is the sole draw, it's also about incrementally building up programs live.

You don't need ECS for that and it doesn't necessarily buy you much. Composition over inheritance can often accomplish the same goal.

Of course it buys a lot, you don't blow away program state when you change the program. That's huge for highly stateful programs.

Also ecs is a prime example of composition over inheritance

Re: Databases are the endgame for data-oriented design

#125

Post by slooonz »

> I think that people perceive databases to be slow, not because they are slow, but because they often interact with them in the context of persisting data to disk, from across a network, by passing strings back and forth which must be parsed, compiled and executed. If you’ve ever optimized a program, you know that I just listed pretty much the slowest operations you can do on a computer.

I’m sorry, but the difference between "databases are slow" and "the only way I can interact with databases is slow" is only pedantics. I understand that, as a database developer working hard to squeeze every bit of performance possible this is frustrating, but as a developer I couldn’t care less about the difference.

Re: Databases are the endgame for data-oriented design

#127

Post by jalk »

jerf wrote: post #26
munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s." I appreciate that most of the ECS hype has been around specific use cases, though. OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded…

When you have a hammer, everything looks like thumbs

Re: Databases are the endgame for data-oriented design

#128

Post by masfoobar »

I started learning programming in the mid-to-late 90's.. a teenager learning with Turbo C, Turbo Pascal, and VB6... eventually to Visual C++... to then attempting to jump on the OOP bandwagon with Java, I began to dislike coding.

I was questioning whether this was the career but, after a few years, decided to give it a go.

Job interviews, particulary then, were about "OOP this" and "OOP that" and I would purposely be agreeable but was unahppy with the code I was writing. Eventually I would come out of my shell with views to other devs.

Roll on to 2014 watching Mike Actons "Data Oriented Design" - and I immediately felt at home, not because I consider myself an 'expert' or on the same league as seasoned game programmers but I felt I had the right mindset all along!

In my 20 years, primarily a C# developer to pay my bills but at home my personal projects are written in C, and tried D.. and now Odin. I just prefer the control they give.

(Scheme as well, I do like Scheme)

Whenever someone asks me (a developer) what Data Oriented Design is, I try to explain to them it is like building a decent database. You are not thinking about the code (so much) but the representation of data. With databases, you create tables that have relations with other tables, with keys and indexes. You are setting up "lookups" etc.

Once they grasp the idea of building a database (which most developers can easily understand) - it is a case of transferring that energy not to tables in a database, but to data structures in your programming language.

Of course, this idea is easier to understand if you have experience in languages like C. For those coming from Python or Java or C#, etc, can be a little trickier but only if they think purely in the OOP mindset.

However, if they struggle to grasp this, then ECS is a great way for understanding in that OOP-ish way. The penny drops when I discuss Entities and Components. You are not building classes and inheritence, you are building entities and components. It is more flexible creating an Entity and "attaching" a Component for it to do something. Then you don't have some kind of Update method in a class, you just have one function which passes in all of it's type.

Anyway - it all comes back full circle now reading this article. Great reading! Now I can just refer people to this link instead.

Re: Databases are the endgame for data-oriented design

#129

Post by masfoobar »

jerf wrote: post #26
munificent wrote: post #23

As an ex-game developer and software architecture nerd, I'm very excited about data-oriented design and ECS. It really is a cool pattern, and it's a very common one in shipping games today. It's not just architecture astronaut stuff. At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s. Can ECS be a better way to structure your game entities and…

"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s." I appreciate that most of the ECS hype has been around specific use cases, though. OOP was claimed as not a specific useful tool, but the answer to all programming, a billing it has not lived up to. It achieved "useful tool", no question, especially as some of the very rough bits were sanded…

"At the same time, the level of hype about ECS today reminds me an awful lot of the amount of hype surrounding OOP in the 90s."

I try not to focus too much on the ECS side. It is all about understand the problem you need to solve. If you are making a game, and you know exactly what the game needs to do from a programming perspective... write it.

Yes, while I would keep the "data oriented" viewpoint... but do you really need to spent time on some ECS layer? If you know exactly what each character in the game does, write it and solve it. Are you making a game... or trying to create the next Unreal Engine?

Re: Databases are the endgame for data-oriented design

#130

Post by randomdata »

hot_gril wrote: post #34
randomdata wrote: post #6

Earlier quoted context omitted.

> And that in itself is the most important factor to avoid bugs. I'm not sure about that. SQL was the first language I learned and the language that has always been there throughout the decades, and is also where I make the most mistakes.

If you were to write the equivalent of a SQL query in some other language, you'll probably make a lot more mistakes. Especially if you're trying to achieve the same performance. And I mean a read-only query, not even something with multi-writes and locking.

I don't think that is true as I could use a better relational calculus that exhibits all the of the same positive attributes that SQL brings to the table. SQL's traps are not all fundamental, just straight up poor language design. SQL's biggest trap is where it grossly deviates from Codd's relational algebra model. A relational system that is more "pure" would solve many bug vectors right out of the gate.
Post reply on HN