Live data from Hacker News

Databases are the endgame for data-oriented design

spacetimedb.com

41–50 of 157 posts

Re: Databases are the endgame for data-oriented design

#41
post #4

My colleagues hate me, but I also found that SQL is The way to write business logic. Lots of caveats about difficulty to test and weird syntax. But it is just that SQL is the most terse and standard way so express logic. And that in itself is the most important factor to avoid bugs. Not what testing strategy you choose.

Have you written an ORM in SQL to help make querying easier?

Re: Databases are the endgame for data-oriented design

#42
post #14
post #10

Earlier quoted context omitted.

that's how you end up with multi-thousand LOC sql files that contain all the historical business logic and edge cases of a company and it just keeps being piled up on and its complexity keeps growing because not everything is expressible in SQL. So you end up having a big monster that someone needs to support and one that's not very testable easily

Glad that doesn't happen any where else!

Most other programming languages are more composable than a SQL query. You can make functions, split logic into different classes, extract common logic, etc. Much harder with big sql queries

Re: Databases are the endgame for data-oriented design

#43
post #30

Earlier quoted context omitted.

Are you using Postgres? It seems like you haven’t discovered CTEs or lateral joins.

I agree that CTEs are a solution to some of the pain listed above, but CTEs aren't exotic. Lots of the major RDBMSs support them.

CTEs are an absolute must to get the right "do-this-then-that" approach.

Re: Databases are the endgame for data-oriented design

#44
post #40

Earlier quoted context omitted.

> But it is just that SQL is the most terse and standard way so express logic. As someone who has written a ton of complex SQL... I couldn't disagree more. Trying to shoehorn things that can trivially and intuitively be expressed in a couple of for-loops with a couple of variables, into SQL expressions making use of joins and aggregate functions and GROUP BY's and (god forbid) correlated subqueries... having to repli…

The types of issues you're enumerating are really part of a learning curve that every language and environment will have. > not having basic functions for calculating percentiles or even a basic median I don't think this is really true anymore; windowing functions are pretty prolific, and I think every major database will have some percentile functions

> The types of issues you're enumerating are really part of a learning curve that every language and environment will have.

They're not though. That's why excellent coders often despise having to learn and use SQL, and basically just refuse to. SQL is uniquely terrible -- and I say this as someone with a career spanning from Win32 C++ to every major web technology.

> every major database will have some percentile functions

MySQL still doesn't. MariaDB only added it in 2018, and I've been writing queries for a lot longer than that. (And MySQL only added window functions in 2018 as well.)

And remember that, for various reasons, you're often (usually?) stuck using a database engine in production that's several years old, since upgrading a database engine is not something to be taken lightly.

Re: Databases are the endgame for data-oriented design

#45

Nope, nope and nope. Went to the github page for spacetimedb, it does everything that is terrible. >Instead of deploying a web or game server that sits in between your clients and your database, your clients connect directly to the database and execute your application logic inside the database itself. You can write all of your permission and authorization logic right inside your module just as you would in a normal…

> you are just asking for a hard time debugging, troubleshooting, and increasing the chance that you will fuck up the most valuable part of your system, your data.

This sounds like a tooling problem. One could imagine a database that doesn't have these issues.

> It also means you have a single point of failure, no read-replicas or redundancy.

What? Why would any of this be the case?

Re: Databases are the endgame for data-oriented design

#46
post #40

Earlier quoted context omitted.

The types of issues you're enumerating are really part of a learning curve that every language and environment will have. > not having basic functions for calculating percentiles or even a basic median I don't think this is really true anymore; windowing functions are pretty prolific, and I think every major database will have some percentile functions

> The types of issues you're enumerating are really part of a learning curve that every language and environment will have. They're not though. That's why excellent coders often despise having to learn and use SQL, and basically just refuse to. SQL is uniquely terrible -- and I say this as someone with a career spanning from Win32 C++ to every major web technology. > every major database will have some percentile fun…

> They're not though. That's why excellent coders often despise having to learn and use SQL, and basically just refuse to. SQL is uniquely terrible -- and I say this as someone with a career spanning from Win32 C++ to every major web technology.

This is basically a strawman + appeal to authority argument. I've no desire to bicker. "Uniquely terrible" is quite subjective, we'll have to agree to disagree on if it applies to sql.

Re: Databases are the endgame for data-oriented design

#47
post #14

Earlier quoted context omitted.

Glad that doesn't happen any where else!

Most other programming languages are more composable than a SQL query. You can make functions, split logic into different classes, extract common logic, etc. Much harder with big sql queries

Use views and stored procedures if you need composability.

Re: Databases are the endgame for data-oriented design

#48
post #10
post #4

My colleagues hate me, but I also found that SQL is The way to write business logic. Lots of caveats about difficulty to test and weird syntax. But it is just that SQL is the most terse and standard way so express logic. And that in itself is the most important factor to avoid bugs. Not what testing strategy you choose.

that's how you end up with multi-thousand LOC sql files that contain all the historical business logic and edge cases of a company and it just keeps being piled up on and its complexity keeps growing because not everything is expressible in SQL. So you end up having a big monster that someone needs to support and one that's not very testable easily

And in any other language this wouldn’t result in multi-thousand lines of code with historical business logic and edge cases?

Re: Databases are the endgame for data-oriented design

#49
post #31

SpacetimeDB looks great for game dev. I've been hearing chitter chatter in the FaaS world about connecting clients directly to the DB as a means of reducing number of components and complexity in simple-ish CRUD apps. Not sure how well that design holds up there, time will tell. For those seeing this (SpacetimeDB) and immediatelly conjuring images of nightmares to be, consider the following: if you had an extremely l…

I agree!

I first learned about SpacetimeDB from a HN thread posted three months ago: https://news.ycombinator.com/item?id=37146952

Re: Databases are the endgame for data-oriented design

#50
post #46

Earlier quoted context omitted.

> The types of issues you're enumerating are really part of a learning curve that every language and environment will have. They're not though. That's why excellent coders often despise having to learn and use SQL, and basically just refuse to. SQL is uniquely terrible -- and I say this as someone with a career spanning from Win32 C++ to every major web technology. > every major database will have some percentile fun…

> They're not though. That's why excellent coders often despise having to learn and use SQL, and basically just refuse to. SQL is uniquely terrible -- and I say this as someone with a career spanning from Win32 C++ to every major web technology. This is basically a strawman + appeal to authority argument. I've no desire to bicker. "Uniquely terrible" is quite subjective, we'll have to agree to disagree on if it appli…

Excuse me, but I'm certainly not "bickering" and I resent the insinuation. And you don't seem to understand what a strawman argument is.

I'm not going to write an article on this for you, but you can certainly see the sentiment frequently expressed on HN that the rise of NoSQL and of ORM's is a direct result of otherwise capable programmers not wanting to deal with SQL and its conceptual building blocks such as joins. Take that for what you will, but you might suspect there's a reason for it, and that it's not merely the learning curve for "every language".

Post reply on HN