Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Announcing SurrealDB 1.0

surrealdb.com

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Announcing SurrealDB 1.0

#41

Post by CyberDildonics »

ThinkBeat wrote: post #17

This feels like satire. and written by someone who should have spent more time studying the history of database systems. > Imagine a world where the majority of your backend logic is seamlessly embedded within the database itself This is not a good idea. It has been done many times and never quite caught on because it is not a good idea. From a security perspective it is a nightmare. Or if you do put all the correct…

It's a nightmare from a lot of angles, especially debugging. Mashing everything together is a recipe for disaster. To debug something you want to isolate a small unit of execution and be able to replay the same data through it continually until you get the right results back.

Re: Announcing SurrealDB 1.0

#42

Post by foobiekr »

anonzzzies wrote: post #26
ThinkBeat wrote: post #17

This feels like satire. and written by someone who should have spent more time studying the history of database systems. > Imagine a world where the majority of your backend logic is seamlessly embedded within the database itself This is not a good idea. It has been done many times and never quite caught on because it is not a good idea. From a security perspective it is a nightmare. Or if you do put all the correct…

> never quite caught on It was quite normal in 70s 80s and even 90s; all the ms sql, db2, oracle and as400 systems I encountered in those days had all or almost all logic as stored procs. Very large ones.

There's also a ton of optimization savings in doing this. your DB already has to move from media to cache, so why not do the pipelined processing in place where it's cheapest?

Instead, we haul that shit to the NIC, then across the network, then copy it into memory on some server (probably inefficiently), do the operations there, or we have to reinvent this with pushdown functions for distributed databases.

There are many cases where moving function-to-data is the right answer.

Re: Announcing SurrealDB 1.0

#43

Post by jddj »

10000truths wrote: post #36
ThinkBeat wrote: post #17

This feels like satire. and written by someone who should have spent more time studying the history of database systems. > Imagine a world where the majority of your backend logic is seamlessly embedded within the database itself This is not a good idea. It has been done many times and never quite caught on because it is not a good idea. From a security perspective it is a nightmare. Or if you do put all the correct…

> Or if you do put all the correct isolation around the code to protect the database, then you have basically created an "app server" (old term) inside a database, and it would happily, run outside of the database since in essence it is already doing so. You can drastically cut down on end-to-end latency by eliminating the network hop from your app server to your DB, especially in cases where you are forced to make m…

It's possible and even relatively straightforward to eliminate that network hop without moving all of the application logic into the database.

Re: Announcing SurrealDB 1.0

#44

Post by foobiekr »

threeseed wrote: post #31
anonzzzies wrote: post #26

Earlier quoted context omitted.

> never quite caught on It was quite normal in 70s 80s and even 90s; all the ms sql, db2, oracle and as400 systems I encountered in those days had all or almost all logic as stored procs. Very large ones.

During the Java era the standard quickly became ORM instead of stored procedures. Largely thanks to NeXT showing the way with EOF (the original ORM).

ORMs basically exist because people doing coding refuse to learn the relational model and want to program with OO instead. Most of the time they are, at best, irrelevant but eventually become expensive, slow, complex and painful. Everyone in coding knows, once they get a lot of experience, that translation/mapping layers are a waste of cycles, memory, and are complex and where assumptions mismatch results in tons of bugs, and that applies to ORMs in partcular.

Re: Announcing SurrealDB 1.0

#45

Post by foobiekr »

jddj wrote: post #43
10000truths wrote: post #36

Earlier quoted context omitted.

> Or if you do put all the correct isolation around the code to protect the database, then you have basically created an "app server" (old term) inside a database, and it would happily, run outside of the database since in essence it is already doing so. You can drastically cut down on end-to-end latency by eliminating the network hop from your app server to your DB, especially in cases where you are forced to make m…

It's possible and even relatively straightforward to eliminate that network hop without moving all of the application logic into the database.

"all" is doing heavy lifting here in a way that distorts the observation the person you are replying to was making.

Re: Announcing SurrealDB 1.0

#46

Post by sa46 »

ThinkBeat wrote: post #17

This feels like satire. and written by someone who should have spent more time studying the history of database systems. > Imagine a world where the majority of your backend logic is seamlessly embedded within the database itself This is not a good idea. It has been done many times and never quite caught on because it is not a good idea. From a security perspective it is a nightmare. Or if you do put all the correct…

This line in particular is a head-scratcher:

> Advanced inter-document relations and analysis. No JOINs. No pain. […] queries allow for multi-table, multi-depth document retrieval, efficiently in the database, without the use of complicated JOINs

That sounds like they have a networked database, not a relational database. That might be fine, but as I understand it, relational databases won because they offer more flexible access patterns and it’s easier to write correct queries.

Re: Announcing SurrealDB 1.0

#47

Post by jddj »

foobiekr wrote: post #45
jddj wrote: post #43

Earlier quoted context omitted.

It's possible and even relatively straightforward to eliminate that network hop without moving all of the application logic into the database.

"all" is doing heavy lifting here in a way that distorts the observation the person you are replying to was making.

Fair enough. It wasn't my intention, perhaps just being careless.

I think these conversations usually boil down to some kind of pragmatism versus idealism, so let me be frank about that. If everything could be done as close to the data as possible, that would be faster than not. Every boundary / transformation is a few burned cycles and delays waiting for the arrival of photons/electrons. The ideal here, really, is to have everything occur in one process on one machine without hitting the disk. Ideally the User's machine, for whatever value of User.

However, there are other considerations which cause us frequently to draw the boundaries differently. Often just as valid.

Where certain things happen is relatively flexible, it's software and we can come up with whatever architecture we can come up with. If you're moving more logic into the database process, then there are costs to that but they're not measured in cycles. Others if you do it by moving the database into the application process.

Maybe those costs make sense, in which case go for it. We've had the means to do arbitrary work in the database for many decades, and for some periods it was even fashionable.

Re: Announcing SurrealDB 1.0

#48

Post by pryelluw »

“Imagine a world where the majority of your backend logic is seamlessly embedded within the database itself.”

Don’t need to. Majority of my work is de-coupling systems “designed” around this idea.

Why can’t the database be the database, the backend be the backend, etc ? You can’t have a toolbox full of multi use tools. Sometimes you just need a fucking Phillips head screwdriver.

Re: Announcing SurrealDB 1.0

#49

Post by jitl »

vlovich123 wrote: post #33
jitl wrote: post #20

Earlier quoted context omitted.

One of the guys behind Convex explained the rough idea; I hope I do it justice. The strategy is to break the subscription up into listens based on the read-set ranges of the query. Then you put the individual read-set ranges into a system table that you index. Finally when writes happen, you notify all queries who's read-set intersects with the write-set. For example, say I have a query `SELECT * from block WHERE par…

Is there a talk you can point me to? There’s several challenges with this approach that come up for me (which unless I’m mistaken is the naive approach of checking each write against a notification set). The first is that maintaining the read set seems very expensive since it scales with the number of live queries installed. In a multi tenant DB, that seems tricky to manage in terms of memory used. The second is that…

See https://news.ycombinator.com/item?id=31836545

To the specifics of my solution, you’re going to visit many less than O(subscriptions) rows using a btree index on min & max as I suggested, and I’m sure there’s use case specific data structures that could do an even better job.

You also don’t need to stall the write transaction until subscriptions are notified; you can batch up that work into big chunks and process it on a background queue after the transaction commits.

Finally, you don’t need to have subscribe to read sets that exactly match the query. You can simplify them a lot for those kinds of complex cases because you can tolerate over-subscription, re-running the query even if it hasn’t changed, checking the new result, and then deciding to deliver it to the external subscriber or not.

Re: Announcing SurrealDB 1.0

#50

Post by dgudkov »

While I applaud any attempts to innovate in this area, I'd be more interested in seeing the opposite approach - integrating persistent storage in a programming language. If we already have default implementations of dictionaries, linked lists, and other in-memory data structures in programming languages, why not have default implementations of permanent data structures such as object collections and KV-stores?
Post reply on HN