Live data from Hacker News

Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

risingwave-labs.com

291–300 of 307 posts

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#291
post #85

> But as more and more engineers joined us, some shortcomings > of C++ came to bite us: unreadable coding style, memory leak, > segmentation fault, and more. * unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers. The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safet…

In Rust you can allocate small structs on the stack while being confident that the plain pointer passed down the stack will be valid throughout the function execution. In C++, you need to use a plain pointer or a unique pointer. The former makes the function leak when passed a pointer to heap. The latter requires a heap allocation and free for the struct contents.

You probably want to revisit your understanding because what you said doesn't make much sense. Allocating variables on stack and passing them over to other functions is perfectly fine. If you want them to live on a heap that's perfectly fine too.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#292
post #140

Earlier quoted context omitted.

I don't fall into any of the camps you mentioned specifically. I have been using C++ as my main programming language for a good 15 years and am a big fan of modern C++. I tried Rust for a month (every day) and I just feel like it gets in my way too much and it's just not worth the extra friction. A friend of mine (using Python as a physicist) wanted to try system programming recently and I told them to just try Rust…

> It's not as hard to use correctly as people pretend it is (though I admit that years of experience and studying are required) The original statement and the one in parens are directly at odds with each other. If every C++ dev requires years of experience and studying to use correctly, it follows that they have left years worth of code that is not done correctly in their wake. Also, no other (non-esoteric) language…

> violations of memory safety - the one class of bugs that we know how to eliminate completely,

Do we? Why exactly do we need sanitizers in Rust then?

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#293
post #45

Earlier quoted context omitted.

Any thoughts on Timescale?

Unfortunately, you cannot install timescale on an rds, or I believe , other managed Postgres services : https://stackoverflow.com/a/67712962 .

https://www.timescale.com/ - they offer this as part of Timescale Cloud

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#294
post #262

The more I read the "cons" in this article's C++ section the more I scratch my head. Code style is something that should be uniform and enforced. It's bikeshedding, but unless you're using a language with opinionated formatting (e.g. Go), it's absolutely a must have, and should be one of the first things done even at a startup. This really looks like a case of not seasoned developers throwing crap at the wall and see…

I totally agree with your first paragraph. There should have been code reviews and new developers should have been coached.

About Rust... I'm open-minded, let's see what happens.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#295
post #145
post #37

Every time I read of memory leaks in C++ codebase, I get no idea why RAII didn't work in those cases. Even if the codebase was huge and complex. Circular references? Some special cases why one can't use RAII? Forget unique pointer or smart pointers. People had been coding in C++ far before that without fearing memory leaks by following RAII principals.

RAII isn't a perfect safeguard against memory leaks. Rust doesn't provide 100% protection against them, either, especially when `unsafe` gets involved. It's better than C++, though, in this regard. There is no fool-proof way to avoid them, other than by never dynamically allocating memory.

If everything is stack allocated, then the never dynamically allocating memory part will be true.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#296

Earlier quoted context omitted.

> As a hobby software engineer who mostly writes ETL jobs in Python the biggest selling point of Rust is Cargo. I usually use a lot of .clone() in my code and most of my fields are Strings which would make a seasoned Rust/C++ laugh at the code. [...] Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability. It sounds like you'd be better off with higher-lev…

Absolutely not. The last time I have tried Haskell it was failing some pretty basic tasks. The JSON library required to be re-compiled for some reason and it used 20+G of RAM when the build crashed. The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used. I do not have time for these, Rust offers a much better experience…

> The community is flat out hostile towards Mac users and a basic request was closed with a comment that Mac is a broken platform and it should not be used.

Have you got a link to that request?

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#297

Earlier quoted context omitted.

Haskell does have both

Haskell is not a high-performance systems programming language, not sure how it's relevant in a discussion about Rust, C++ and Ada

It's relevant because many people using Rust are using it for building web backends where high-performance systems language is not needed, yet the safe concurrency is required.

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#298

Earlier quoted context omitted.

It's "Tokio". Coroutines in Rust are a native language feature and their API is part of `std`, but the executors indeed come from community libraries. Tokio is one option, but as long as you don't contextually fork (for parallelism) you can in theory run the same coroutines on any other executor, like the ones from async-std. Spawning a sibling task (as opposed to mixed child task polling, which can be macroed inline…

Sorry if I'm missing something important as I have experience with C++ but not rust, but is async/await the right abstraction in Rust vs. the underlying generators which I assume is lower level and closer to C++20 Co-routines in terms of performance ? [1] In other words, it seems Rust co-routines at the level of Tokio are not as performant as C++ native Co-routines [2] and further down the thread, it's mentioned that…

Hmm… I'd say for IO-bound tasks/loading in a DBMS yes, async/await coroutines are the correct abstraction, since you're waiting to reschedule on external notice and normally don't want to spin-wait (or you could block on the optimistic case and then `.await` the fallback).

If you'd like to do modular data processing instead then the current best Rust approach is using `Iterator`s[1] and possibly the `rayon`[2] crate for safe parallelisation.

There currently is no coroutine feature that maps nicely onto `Iterator`s, or rather that's indeed expected to be the unstable generators feature, as it should cover that use case without overhead. (Async-coroutines map onto `Future`[3] instead.)

You can think of Rust's `async` and generator use cases vaguely like the ones of JS's async and generator functions. The latter will function without any runtime or trampoline if you advance it in a loop, as far as I can tell, but they're unwieldy if what you want to yield on are only waits on external events. (`Waker`s[4] have similar consumer semantics to JS's or C#'s continuation passing, even if they are only indirect memory management handles.)

It also often makes sense to use both of these at the same time to produce something like an `AsyncIterator`[5], but I expect that feature to be quite some ways away in Rust, at least in a seamlessly mixing fashion.[6]

- - -

I'm actually pretty curious which will turn out faster in practice between C++'s coroutines and Rust's generators, since their memory management is so different.

The Rust coroutines (that is: including `Future`s) are generally stackless already, but also by default heapless, as their entire memory is abstracted as opaque instance with anonymous type that is returned to the caller by value. Boxing them is an explicit operation, and it's equally possible to pin them on the current stack/in the current instance instead. (That's how `.await` works; the child is embedded into the outer `Future` directly.)

There are some pitfalls related to this, since that instance's size is determined by the largest `.await`/yield site, but the tooling should (eventually) be able to warn on unexpectedly large `.await`/yield locations like it currently warns on unexpectedly large enum variants.[7]

(I'll borrow your footnote style.)

[1] https://doc.rust-lang.org/stable/core/iter/trait.Iterator.ht...

[2] https://crates.io/crates/rayon

[3] https://doc.rust-lang.org/stable/core/future/trait.Future.ht...

[4] https://doc.rust-lang.org/stable/core/task/struct.Waker.html

[5] https://doc.rust-lang.org/stable/core/async_iter/trait.Async...

[6] then comparable to JS, I'd assume, where `yield` and `await` aren't necessarily paired: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...>

[7] https://rust-lang.github.io/rust-clippy/stable/index.html#la...

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#299

Earlier quoted context omitted.

>Rust is not difficult because of lifetimes, it just gets in the way of freely prototyping what you want. I'm not a rust programmer, but I guess that's an issue if you come from a dynamic language, not from c++.

Yeah, sadly it is. With e.g. Elixir I can literally get into a REPL and prototype my solution in minutes, right there on the spot, and then just copy a few lines from it and have the solution be 90% done (minus tests, of course). With Go and Rust I have to make a dedicated function somewhere and then have it be called after starting the program. Ain't exactly rocket science but the difference in time to do it and the…

While not Elixir good, the evcxr python notebook plugin gets you 50% of the way there.

https://depth-first.com/articles/2020/09/21/interactive-rust...

https://github.com/google/evcxr

The other option is to use the playground, https://play.rust-lang.org/?version=stable&mode=debug&editio...

Re: Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)

#300

Earlier quoted context omitted.

Sorry if I'm missing something important as I have experience with C++ but not rust, but is async/await the right abstraction in Rust vs. the underlying generators which I assume is lower level and closer to C++20 Co-routines in terms of performance ? [1] In other words, it seems Rust co-routines at the level of Tokio are not as performant as C++ native Co-routines [2] and further down the thread, it's mentioned that…

Hmm… I'd say for IO-bound tasks/loading in a DBMS yes, async/await coroutines are the correct abstraction, since you're waiting to reschedule on external notice and normally don't want to spin-wait (or you could block on the optimistic case and then `.await` the fallback). If you'd like to do modular data processing instead then the current best Rust approach is using `Iterator`s[1] and possibly the `rayon`[2] crate…

Thank you for the very detailed response. I'll check the links out.
Post reply on HN