Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases in 2024: A Year in Review

cs.cmu.edu

Post reply on HN
Print view View original

141–150 of 220 posts

Re: Databases in 2024: A Year in Review

#141

Post by to11mtm »

antirez wrote: post #43
nicoritschel wrote: post #40

Earlier quoted context omitted.

With all due respect, the linked video was pretty fair. It didn't imply not to use Redis, just not as a primary datastore. I don't think folks work with Redis out of fondness for the model, but because it's the least worst datastore for caching, lightweight message broker, and simple realtime things like counters.

Talking about the broken API argument here. Also Redis is particularly useful exactly in other situations compared to what OP says. Leaderboards style use cases with sorted sets are killer applications (super hard to model with SQL) of the data structure server thing. Apparently OP does not understand this and says "simple GET/SET" is what you should use Redis for. Redis has probabilistic data structures, the ability…

> I re-read this comment and it's embarassing I ever have to write this, because after 15 years of Redis history at such scale and popularity, pretty much everybody that was seriously exposed to Redis knows those stuff. Is tech culture really degraded so much that we have to restate the obvious?

I know you wrote OG and did a lot of Redis but...

Yes, Tech culture is that fucked.

In my past job I was hired semi-specifically to deal with a concern, namely that their use case 'fit well with Kafka' but the latency for their case sucked at least as much as the API pain. (Yes I did something better. No, IDK if folks will ever see it).

Now?

Now I spend my days trying to 'paper-over' patterns that drive me to insanity just trying to make it work from a 'people need to learn why things work on a starship' level [0].

On a real level you didn't fail, Redis has lots of great patterns. On a -practical- level it's a shitshow because you now have lots of folks 'glue-gunning' the Redis API on use cases that probably need tweaking or aren't the right fit at all, alas they all worked off the same example on GH/SO/etc and then did their own "this wasn't even the right way to do this so I'm adding glue, what could possibly go wrong" case.

(That said, Nats has decent stuff for this in form of KV CompareExchange style APIs, and I see the inspiration there, so that's something to feel good about.)

[0] - Namely, if anyone has a good prompt for taking a photo of someone and doing img2img of 'Person in astronaut uniform preaching from an ivory tower', that would be a plus

Re: Databases in 2024: A Year in Review

#142

Post by 1egg0myegg0 »

yencabulator wrote: post #114
roark_howard wrote: post #74

DuckDB dominating over DataFusion could fuel the ongoing language war with a great half baked argument!

Every time I've tried to use DuckDB it has segfaulted on me, so yeah I'm betting on DataFusion..

That is very surprising to hear! If you can reproduce it, could you please file a bug report on GitHub? That would be a huge help!!

Re: Databases in 2024: A Year in Review

#143

Post by DennisZ_89 »

leeoniya wrote: post #6
samanthasu wrote: post #4

would love to see what Andy's take on GreptimeDB https://github.com/GreptimeTeam/greptimedb

made same comment recently in https://news.ycombinator.com/item?id=42330055#42331927

Glad to see people start trying on GreptimeDB. We're committed to building a fully open-source and cost-effective, unified time series database for metrics, logs, and events. To this point, it may still be too small to be included in the year database summary. But hope we'll grow fast in 2025 :)

Re: Databases in 2024: A Year in Review

#144

Post by thayne »

zzzeek wrote: post #79
antirez wrote: post #69

Earlier quoted context omitted.

You totally miss that Redis is more like a remote interpreter with a DSL that manipulates data structures stored at global variables (keys): you (hopefully) would never complain about languages having this semantics. I don't think you understood how Redis collections work. The items are just strings, they can't be mixed like integers or strings together or whatever, nor collections can be nested. The Redis commands d…

> You totally miss that Redis is more like a remote interpreter with a DSL that manipulates data structures stored at global variables (keys): I think he makes the point that these "global variables" are dynamically typed; you can have "listX" and then write a non-list into that same name; statically typed systems would not allow this. He makes the fairly non-controversial point that a statically typed system (SQL, o…

> you can have "listX" and then write a non-list into that same name; statically typed systems would not allow this

Well, that depends. In most SQL databases there are many cases where supplying the wrong type of value will implicitly convert to the expected type, often in unexpected ways that can result in subtle bugs.

Re: Databases in 2024: A Year in Review

#145

Post by mistrial9 »

somat wrote: post #133
antirez wrote: post #43

Earlier quoted context omitted.

Talking about the broken API argument here. Also Redis is particularly useful exactly in other situations compared to what OP says. Leaderboards style use cases with sorted sets are killer applications (super hard to model with SQL) of the data structure server thing. Apparently OP does not understand this and says "simple GET/SET" is what you should use Redis for. Redis has probabilistic data structures, the ability…

Now I have never used redis in this capacity but these sorted sets(a data structure that maintains it's sort as data is entered I assume) how is that different from "create index on player_score (score)"? that is, an index on the score column. which will create a data structure that maintains it's sorted nature as data is entered. My naive view is that you create a sorted set every time you define an index. that is,…

maybe it was a reference to two or three or N level deep nested tree structures. Those dont map as naturally using simple SQL relations.

Re: Databases in 2024: A Year in Review

#146

Post by thayne »

emmelaich wrote: post #123
olavgg wrote: post #104

Earlier quoted context omitted.

I would use it if it supported backup/restore over unix pipes / ssh.

If it supports backup to a file, you can have it write to a named pipe and from there to wherever. I used this hack for backing up Oracle 30 years ago. Something like 'mknod p backup.dmp; oradump .... file=backup; dd if=backup.dmp | ssh othermachine receiver-process'

Not necessarily. That won't work if the backup uses apis that a pipe doesn't support, like seek or reading back from the file.

Re: Databases in 2024: A Year in Review

#147

Post by riku_iki »

1egg0myegg0 wrote: post #142
yencabulator wrote: post #114

Earlier quoted context omitted.

Every time I've tried to use DuckDB it has segfaulted on me, so yeah I'm betting on DataFusion..

That is very surprising to hear! If you can reproduce it, could you please file a bug report on GitHub? That would be a huge help!!

I personally filed bunch of bugs, they mostly were autoclosed because of no activity after 3 months. This is very discouraging, and I rather invest effort in looking for workarounds in the future.

Re: Databases in 2024: A Year in Review

#148

Post by hdesh »

> The upcoming year is going to be the test of strength for many database startups. Nobody wants to be the next MariaDB Corporation,

The link for "MariaDB corporation" points to an empty image with white colour background. Can anyone explain the context here?

Re: Databases in 2024: A Year in Review

#149

Post by josephg »

antirez wrote: post #86
jsnell wrote: post #81

Earlier quoted context omitted.

> 1. GET is the retrieve-the-key-of-string-type operation. That's a tautological argument. The question isn't what the definition of GET is, but whether the design is good. > 2. Having GET doing something like LRANGE 0 -1 would have many side effects. Getting for error a huge list and returning a huge data set without any reason, creating latency issues. If this really were the reason, you'd have separate operations…

The examples I made are just a subset of the protection that this provides. Similarly you can't LRANGE a set type, and so forth. So this in general makes certain errors evident ASAP (command mismatch with the key type). This does not meant that Redis would not work having generic LEN, INSERT, RANGE commands. But such commands would end also having type-specific options, that I have the feeling is not very clean. Anyw…

Eh. What people are really arguing about here is redis’s type system. Redis’s approach has some pros and some cons. I think dismissing redis’s approach out of hand for its choices is too simple a treatment.

Most sql databases (like Postgres) require all types to be declared once, and then they do type checking on mutation. In that sense, sql is like a static language like C. But weirdly, the results returned from a sql query are always dynamically typed values, expressed in a table. Applications reading data from sql will still typically need to know what kind of data they expect back - but they usually do that type checking at runtime.

Redis flips both of those choices. It’s dynamically typed - so it won’t check your mutations. But also, you don’t need schema migrations and all the complexity they bring. And rather than having a single “table” type, redis queries can return scalar values, lists or maps. What kind of return value you get back depends on the query function. (Eg GET vs LRANGE).

If you think of a database as the foundation underneath your house, static typing & type checking is a wonderful way to make that foundation more stable. There’s a reason Postgres is the default, after all. But redis isn’t best used like that. Instead, it’s a Swiss Army knife which is best used in small, specific situations in which a real database would be complex overkill. Stuff like caching, event processing, data processing, task queues, configuration, and on and on. Places where you want some of the advantages of a database (fast, concurrent network-accessible storage) but you don’t want to stress about tables and schema migrations.

If you really hate redis, maybe say the same thing I say about Java when I teach it to my students. “I hate this, and I’ll tell you why. But there are smart people out there who disagree with me.”

If you ask me, I wish sql looked more like redis in some ways. I think it’s quite awkward that every sql query returns exactly one “table”. I’d much rather if queries could return scalar values or even multiple tables, depending on your query.

Re: Databases in 2024: A Year in Review

#150

Post by Foobar8568 »

rplnt wrote: post #110
stackskipton wrote: post #78

Earlier quoted context omitted.

SRE who deals with some .Net stuff that uses MSSQL but is converting to MySQL. so I feel somewhat qualified to talk about MSSQL. TL;DR: Nothing interesting going on. There is nothing to talk about here. It's boring database engine that powers boring business applications. It's pretty efficient and can scale vertically pretty well. With state of modern hardware, that vertical limit is high enough most people won't enc…

> It's boring database engine that powers boring business applications. FWIW, it also powered the most popular (in terms of player base) MMORPG before WoW took over. And I wouldn't be surprised to find it in aviation, railways, powerplants, grid control, etc...

Before wow , there was either lineage, and then EverQuest.

I guess it was Lineage as Korean used mainly MSFT softwares?

Post reply on HN