Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

risingwave-labs.com

Post reply on HN
Print view View original

111–120 of 307 posts

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#111

Post by criddell »

ZephyrBlu wrote: post #90
josephg wrote: post #80

Earlier quoted context omitted.

Yep. The one big beginner “mistake” I see people make in rust is overusing Box / String / Vec. Rust code that allocates everywhere can be even slower than javascript. The reason? Malloc is slower than you think. Slower than short allocations in V8 or Go. If you want performance, make friends with &str, &[], >, bumpalo, SmartString and SmallVec. (Or similar crates). Removing allocations from the hot path can improve p…

I slightly disagree. Using heap allocated types is perfectly fine. The biggest thing I have to keep reminding myself coming from higher level languages is to re-use data structures , and to architect things in a way that this is possible. Allocating a new String/Vec every single time you do something is killer for performance, but if you do it once up front then clear the data structure for the next use it should be…

> re-use data structures

That's funny. I've been going mostly the other direction. I'm avoiding mutable structures whenever possible. I have a much easier time reasoning about stuff when I know that things aren't going to change mid-life.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#112

Post by flohofwoe »

nu11ptr wrote: post #91
flohofwoe wrote: post #76

Earlier quoted context omitted.

> since C++11 there is no reason for not using smart pointers. Smart pointers lure you into a dark alley where each tiny object lives in its own tiny heap allocation, and before you know it you have so much memory management sprinkled decentralized all over your code base that the memory management overhead becomes a performance problem, but at the same time it's too late to do anything about it because it would mean…

I'm sure it is the same for C++, but in my Rust code things that are put in a Box/Arc etc. are carefully considered and typically my top level business objects. I even wrote my own inline String struct a while back (flexstr) to ensure I don't do allocations for strings smaller than 22 bytes. I don't use allocations "all over the place" without thought and like any language feature, design and placement is important.

That's much more thought put into memory management than what I was used to in the C++ world from 5..15 years ago where many people seemed to have the impression that allocating and freeing memory or the overhead for refcounting is free.

If this is starting to change then it's a good thing.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#113

Post by odo1242 »

mgaunard wrote: post #51
mzimbres wrote: post #11

> But as more and more engineers joined us, some shortcomings > of C++ came to bite us: unreadable coding style, memory leak, > segmentation fault, and more. * unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers. The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safet…

Rewriting stuff is actually quite good and expected of a startup. As the system grows, you realize all that was wrong with your previous version, and can write a new better one. That doesn't mean you need to switch to another language to do it though.

It’s good from a technical perspective but not a business perspective. Startups only get a limited amount of time/money investment in order to prove their profitability. “Writing a better version” can come later, when the startup isn’t trying to come up with a version in the first place.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#114

Post by spoiler »

hamilyon2 wrote: post #88
fooker wrote: post #4

Earlier quoted context omitted.

I disagree. Rust is not difficult because of lifetimes, it just gets in the way of freely prototyping what you want. This situation is slowly improving with the compiler getting better and better. Then there is some annoying macro usage. Some Rust code looks truly alien.

Is rust code heavily seasoned with unsafe keyword really that hard to prototype in? Is it meaningfully harder than c++ in this regard?

I assume they didn't mean compared to C++. If anything, saying prototyping in C++ is easier than Rust is beyond ludicrous even if you're a seasoned C++ developer.

I'd have an easier time prototyping in Rust than C++, and I've been writing C++ for 10 years and Rust for a little over a year.

However, prototype in something like Ruby (or even TypeScript, and some people mentioned Elixir) is in a different universe compared to C++ or Rust.

Like, it's not even a "fair fight" to compare it meaningfully.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#115

Post by gizmo »

crabbone wrote: post #61
threeseed wrote: post #47

Earlier quoted context omitted.

> MongoDB is the only new database that managed to break through Snowflake, Cassandra, DuckDB, Clickhouse, Redis, InfluxDB, DynamoDB, BigQuery etc. There are many databases that have broken through they just are far more niche focused. > but it would still regularly eat your data No it didn't which is why it became so popular. The whole fsync saga was always overblown because (a) every client that shipped set it a sa…

I would love to hear more about "fsync saga" if you have any references. I know that PostgreSQL had "problems" with fsync(), but they never ended. They just accepted defeat, but so would everyone else who relies on filesystems to store their data. It's even more ingrained than that. The way hardware works, and, especially, the communication protocols around it (eg. SCSI or NVMe) are structured, fsync() is always goin…

See this chain of articles: https://aphyr.com/posts/284-jepsen-mongodb

The mongodb organization repeatedly poo-pooed the issues with data loss, and only after repeated public exposure of the issues did they do anything about it. I don't know why people aggressively deny this when anybody can just google it and judge for themselves.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#116

Post by flohofwoe »

gpderetta wrote: post #101
flohofwoe wrote: post #97

Earlier quoted context omitted.

Yes, but people need to be aware that "auto obj = make_unique..." or "auto obj = make_shared..." should be a very rare thing and not the norm (and I've seen plenty code bases like that). There needs to be a proper memory management strategy with the goal of minimizing heap allocation in random places in the code. E.g. automatic memory management doesn't resolve you from thinking just as much about memory management t…

I would hope they are, otherwise why are you even writing C++. I might have too high expectations though...

The idea that ref-counting is automatically "better" than a garbage collector is still pretty popular unfortunately (and that's just the tip of the iceberg).

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#117

Post by craftit »

hamilyon2 wrote: post #88
fooker wrote: post #4

Earlier quoted context omitted.

I disagree. Rust is not difficult because of lifetimes, it just gets in the way of freely prototyping what you want. This situation is slowly improving with the compiler getting better and better. Then there is some annoying macro usage. Some Rust code looks truly alien.

Is rust code heavily seasoned with unsafe keyword really that hard to prototype in? Is it meaningfully harder than c++ in this regard?

From what I've experimented with so far the biggest barrier to using rust was having a mature ecosystem of libraries to use. This is changing though!

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#118

Post by Tamschi »

crabbone wrote: post #57
foxbee wrote: post #9

"The STL library lacks support for some modern programming tools, for example, native co-routine support. As a result, developers must rely on many community projects, and most lack long-term support." I couldn't agree more with this comment. As great as the community is, there's a lot of projects that are just not maintained. I wish this was different.

Rust doesn't have support for native co-routines. You rely on community project for that (Tokyo). At first I even thought it was an argument against Rust, not in favor...

It's "Tokio".

Coroutines in Rust are a native language feature and their API is part of `std`, but the executors indeed come from community libraries.

Tokio is one option, but as long as you don't contextually fork (for parallelism) you can in theory run the same coroutines on any other executor, like the ones from async-std. Spawning a sibling task (as opposed to mixed child task polling, which can be macroed inline) unfortunately isn't available through the common API so far, which is where the bulk of the mentioned incompatibilities stems from.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#119

Post by ArtWomb »

qwertox wrote: post #92
mzimbres wrote: post #11

> But as more and more engineers joined us, some shortcomings > of C++ came to bite us: unreadable coding style, memory leak, > segmentation fault, and more. * unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers. The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safet…

I used to code primarily in C++ around 12 years ago, then moved over to Python. I've now started with Rust and I really love it. I find myself constantly fighting with the compiler, it telling me what it won't allow me to do. In C++, threading was my only way to execute parallel tasks, Python 3 showed me a about coroutines, which I've really started to love. So in Rust I'm experimenting a lot with Tokio right now, an…

Cloudflare Workers (serverless) + WebSockets is the lingua franca of the internet now. Using languages with built-in concurrent stream processing features is I feel key to moving at internet speeds ;)

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#120

Post by menaerus »

hdhrufjdi wrote: post #75
menaerus wrote: post #68

Earlier quoted context omitted.

Aside from you being completely uninformed because free(nullptr) or, for that matter, delete nullptr does absolutely nothing and is well defined operation so you don't need to check for that condition in the first place. But even if you had to, what implications would it have, if you care to explain?

The implications would be checking if it is null before performing the deallocation of the memory, which is a runtime overhead. Stack overflow says "delete" would check for null before deallocation: https://stackoverflow.com/questions/4190703/is-it-safe-to-de... Happy to be educated

Extra branch which is going to be taken 99.99999% of the time is not going to present any runtime overhead. CPU BP unit handles it for us.

That said, if this really had been an overhead, virtually every language out there would suffer from it, including Rust. Every language out there at some point needs to call into the libc.

Post reply on HN