Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databases in 2024: A Year in Review

cs.cmu.edu

Post reply on HN
Print view View original

201–210 of 220 posts

Re: Databases in 2024: A Year in Review

#201

Post by rockwotj »

mike_hearn wrote: post #194
menaerus wrote: post #159

Earlier quoted context omitted.

> It’s C++ 20/23 without virtual memory Not sure what you mean by this? Virtual memory is implied by the CPU MMU and consequently OS kernel. Perhaps you meant they use a lot of custom memory allocation schemes? Otherwise, I agree that the bar is quite high since (1) the problem at hand is already too complex (scalable LSM), and (2) pretty much anything in the code is custom made, e.g. avoiding the OS kernel as much a…

Seastar runs everything in kernel mode.

It tries to bypass the kernel as much as possible. It runs in userspace.

Re: Databases in 2024: A Year in Review

#202

Post by nevf1 »

codeulike wrote: post #17

Weird how SQL Server and its Azure variants gets no mention. It dominates in certain sectors. DBEngines ranks it third most popular overall https://db-engines.com/en/ranking

I have used a lot of RDBMS vs NoSQL solutions and I love SQL Server. I have used and written services consuming/reporting/processing thousands of transactions per second and billions of euro per year.

The profiling abilities of SQL Server Management Studio (SSMS) and its query execution insights, the overall performance and scalability, T-SQL support, in-memory OLTP, and temporal tables - I just love SQL Server.

I'm not sure if it's just that I learned SQL Server better in college than MySQL, Mongo or Postgres but it's just been an amazing UX dev experience throughout the years.

Granted, there's some sticky things in SQL Server, like backups/restores aren't as simple as I'd like, things like distributed transactions aren't straightforward, and obviously the additional licensing cost is a burden particularly for newer/smaller projects, but the juice is worth the squeeze IMHO.

Re: Databases in 2024: A Year in Review

#203

Post by bdangubic »

breadwinner wrote: post #199
bdangubic wrote: post #195

Earlier quoted context omitted.

if it is good for SO it should be good for most :) https://stackoverflow.blog/2008/09/21/what-was-stack-overflo...

You don't believe that web visitors are directly querying SQL Server, right? I can believe they are storing their employee database in SQL Server... they have hundreds of employees.

do some research and then come back here… coming with shit like “you don’t believe they are querying sql server directly” is childish and unprofessional.

Re: Databases in 2024: A Year in Review

#204

Post by kwillets »

downsplat wrote: post #167
kwillets wrote: post #44

I spent the past year puzzling over the DB market as well, but I don't feel like I'm much closer to understanding it. It appears that a lot of attention is now directed at the folks doing 100 MB queries, and the high end has moved past everybody's radar. My idea of an exciting product is Ocient, who have skipped over Cloud and gone for hyperscale on-prem hardware. Yellowbrick is also a contender here. I have a lot of…

> There's a crossover point where self-managed is cheaper than cloud Single data point here: before cloud managed dbs were a thing our smallish startup was running mysql on virtual servers by installing it from the linux package manager. Always worked great, runs without needing manual attention for years at a time once set up, so I've never felt the need to change. So at least in some cases the crossover point is "r…

Zero is certainly on my scale. Bonus points if you build a server and keep it under your desk.

Re: Databases in 2024: A Year in Review

#205

Post by kwillets »

senderista wrote: post #196
kwillets wrote: post #44

I spent the past year puzzling over the DB market as well, but I don't feel like I'm much closer to understanding it. It appears that a lot of attention is now directed at the folks doing 100 MB queries, and the high end has moved past everybody's radar. My idea of an exciting product is Ocient, who have skipped over Cloud and gone for hyperscale on-prem hardware. Yellowbrick is also a contender here. I have a lot of…

I was seriously considering applying to Ocient (had an internal referral), but there's no way I could live on their salary ranges ($145K-185K quoted for senior SWE roles), given that I live in a HCOL area.

I don't know much about the financial side of the company, but it seems like a client-led effort by telco's etc. against the dreck that tech VC's keep pushing on them. That can't translate into decent salaries unfortunately.

Silicon Valley doesn't have a good record in the DB/DWH space; producing a fully-featured DBMS doesn't seem to fit the VC model.

Re: Databases in 2024: A Year in Review

#206

Post by mike_hearn »

rockwotj wrote: post #201
mike_hearn wrote: post #194

Earlier quoted context omitted.

Seastar runs everything in kernel mode.

It tries to bypass the kernel as much as possible. It runs in userspace.

I'm probably thinking of when it's combined with OSv or something. It's been years since I looked at this.

Re: Databases in 2024: A Year in Review

#208

Post by CRConrad »

josephg wrote: post #177
CRConrad wrote: post #175

Earlier quoted context omitted.

> I’d much rather if queries could return scalar values Since when can't they?

I mean, they can - but they’re always wrapped up as pseudo-tables. Not everything is best described as a table, y’know?

> they’re always wrapped up as pseudo-tables.

Are they??? Not as I understand it.

Re: Databases in 2024: A Year in Review

#209

Post by menaerus »

rockwotj wrote: post #200
menaerus wrote: post #197

Earlier quoted context omitted.

Makes no sense. Source?

I mean memory is completely allocated up front and fragmentation is something for application developers to deal with.

I occasionally read their blogs, and I haven't looked into the source for a while, but application developers having to deal with the OOM because one is using custom memory allocation policy that doesn't deal with or suffers from fragmention would be a strange design choice.

jmalloc IME works really well.

Re: Databases in 2024: A Year in Review

#210

Post by Icathian »

atombender wrote: post #189
Icathian wrote: post #186

Earlier quoted context omitted.

This is mostly correct, but it's worth mentioning that cloudberry substantially predates Greenplum going closed source. It just got quite a boost from that change happening. Different dev team too, afaik none of the original Greenplum team was involved with Cloudberry until very recently. Also, Greenplum 7 tracks postgres 14. Which is still old at this point, but not so bad as 12.... I also don't think I'd call the a…

Thanks for the context. In what way would you say Cloudberry lags behind Greenplum technology-wise? I see newer Greenplum versions have a lot of planner improvements. Greenplum 7 is listed as tracking Postgres 12 in the release announcement [1], and the release notes for later 7.x versions don't mention anything. Is there a newer release with higher compatibility? When I say ancient, I mean that it's a "classical" sh…

Cloudberry, last I checked, took their snapshot of all the Greenplum utilities way before the repos got archived and development went private. The backup/restore, DR, Upgrade, and other such seem to leave a lot on the table. I haven't checked in a bit, it's possible they've picked back up some of that progress.

You're completely right, I had the wrong PG version in my memory. Embarrassing, thanks for catching that.

Post reply on HN