Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A year of Rust in ClickHouse

clickhouse.com

Post reply on HN
Print view View original

71–80 of 102 posts

Re: A year of Rust in ClickHouse

#71

Post by eptcyka »

arnsholt wrote: post #63
eptcyka wrote: post #32

Earlier quoted context omitted.

What complexity specifically does Rust add to that model?

I can think of three things off the top of my head: - Rust doesn’t let you pretend that memory is a flat array of bytes - Single ownership of data can be annoying in some cases - The borrow checker pointing out that you’re trying to do something stupid with pointers (again) can be annoying Of course, I’m of the opinion that the hassles are worth it, especially the borrow checker. Almost every time I have to fight the…

How does single ownership conflict with the idea that memory is a flat array of bytes?

Further, the borrow checker does not care about pointers, only references. With pointers, you are on your own. It is true that using pointers in Rust is more cumbersome than it could be. But it is much easier to compartmentalise the pointer parts into separate functions and expose references instead.

I agree that some paradigms and patterns are genuinely difficult to use, e.g. any intrusive data structure, but I do not see the contentious link between simple memory models and the borrow checker and the like.

Re: A year of Rust in ClickHouse

#72

Post by DeathArrow »

angra_mainyu wrote: post #61
DeathArrow wrote: post #58

Earlier quoted context omitted.

I like Zig, too. Not sure if it's a Rust alternative as in you still have to manage the memory yourself. But is much simpler, easier to read, easier to understand, easier to follow and easier to reason about. It's less verbose and more productive. It feels like what C would look like had it been invented today.

When I say "Rust alternative" it's precisely because it competes in the same space: very low-level, no GC, extremely high performance constraints, safety guarantees. Re: safety guarantees, much digital ink has been spilled on how Zig can give Rust a run for its money when it comes to safety. When people say: Rust C++, Zig C; they forget that C++ was precisely meant to be an enhanced C, which is what Zig is trying to…

If you present Zig like a C successor (which C++ was at the moment of its inception), I totally agree.

Zig is decent as a systems programming language. It's good they don't add lots of features and keep it simple.

The only downside I see is companies aren't investing in it much.

Re: A year of Rust in ClickHouse

#73

Post by sph »

alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

Since Rust is not a managed/high-level language, panics are unrecoverable crashes so they need to be dealt at a higher-level, i.e the OS, with appropriate supervisor systems like systemd, or having a master Rust process that spawns subprocesses, and react when one of them abnormally terminates with regular POSIX APIs.

On a platform like Elixir, for example, you can deal with process crashes because everything runs on top of a VM, which is at all effects and purposes your OS, and provides process supervision APIs.

Re: A year of Rust in ClickHouse

#74

Post by tuetuopay »

alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

You can install a global panic handler to avoid bringing the whole process down. Instead of aborting, take the stack trace, print it, perhaps raise to sentry, and kill the specific "work unit" that caused it. This "work unit" can be a thread of a task, depending on how the application is architected.

This is precisely what Tokio does: by default, a panic in async code will only bring down the task that panicked instead of the whole application. In the context of a server, where you'll spawn a task for each request, you have no way to bring down the whole application (), only your current scope.

(): there could be other issues, like mutex poisoning, which is why nobody uses the stdlib's mutexes. But the general point still stands.

Re: A year of Rust in ClickHouse

#75

Post by EE84M3i »

tuetuopay wrote: post #74
alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

You can install a global panic handler to avoid bringing the whole process down. Instead of aborting, take the stack trace, print it, perhaps raise to sentry, and kill the specific "work unit" that caused it. This "work unit" can be a thread of a task, depending on how the application is architected. This is precisely what Tokio does: by default, a panic in async code will only bring down the task that panicked inste…

> there could be other issues, like mutex poisoning, which is why nobody uses the stdlib's mutexes.

What does everyone use instead?

Re: A year of Rust in ClickHouse

#76

Post by tuetuopay »

sph wrote: post #73
alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

Since Rust is not a managed/high-level language, panics are unrecoverable crashes so they need to be dealt at a higher-level, i.e the OS, with appropriate supervisor systems like systemd, or having a master Rust process that spawns subprocesses, and react when one of them abnormally terminates with regular POSIX APIs. On a platform like Elixir, for example, you can deal with process crashes because everything runs on…

This is factually incorrect. The behavior you describe with Elixir (sic) is precisely what most Rust async runtimes do. (sic because it's Erlang that's to thank)

IMHO that is the sensible thing to do for pretty much any green thread or highly concurrent application. e.g. Golang does the same: panicking will only bring down the goroutine and not the whole process.

Re: A year of Rust in ClickHouse

#77

Post by pornel »

sph wrote: post #73
alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

Since Rust is not a managed/high-level language, panics are unrecoverable crashes so they need to be dealt at a higher-level, i.e the OS, with appropriate supervisor systems like systemd, or having a master Rust process that spawns subprocesses, and react when one of them abnormally terminates with regular POSIX APIs. On a platform like Elixir, for example, you can deal with process crashes because everything runs on…

Rust can be optionally compiled in a panic=abort mode, but by default panics are recoverable. From implementation perspective Rust panics are almost identical to C++ exceptions.

For servers that must not suddenly die, it's wise to use panic=unwind and catch_unwind at task/request boundaries (https://doc.rust-lang.org/stable/std/panic/fn.catch_unwind.h...)

In very early pre-1.0 prototypes Rust was meant to have isolated tasks that are killed on panic. As Rust became more low-level, it turned into terminating a whole OS thread on panic, and since Rust 1.9.0, it's basically just a try/catch with usage guidelines.

Re: A year of Rust in ClickHouse

#78

Post by alkonaut »

sph wrote: post #73
alkonaut wrote: post #69

Re: panics: If you have a single long lived process that must do multiple short-lived things (web requests, say) and a panic in one of them MUST NOT take down the whole process, is that extremely difficult to pull off in Rust? I thought you could set up panic boundaries much like you would use catch-all exception handlers around e.g. each web request or similar, in other languages?

Since Rust is not a managed/high-level language, panics are unrecoverable crashes so they need to be dealt at a higher-level, i.e the OS, with appropriate supervisor systems like systemd, or having a master Rust process that spawns subprocesses, and react when one of them abnormally terminates with regular POSIX APIs. On a platform like Elixir, for example, you can deal with process crashes because everything runs on…

But no few would write a process-per-request web server today for example. And if a single process web server handles 100 requests you would then accept that one bad request one tore down the handling of the 99 others. Even if you have a watchdog that restarts the service after the one request choked, you wouldn't save the 99 requests that were in-flight on the same process. Can't you catch_unwind for each request handler, if one chokes then you just ignore that request. If you worry about that messing anything up, then you can tear down and restart your process after that, so the 99 other requests get a chance to complete?

Re: A year of Rust in ClickHouse

#79

Post by tuetuopay »

EE84M3i wrote: post #75
tuetuopay wrote: post #74

Earlier quoted context omitted.

You can install a global panic handler to avoid bringing the whole process down. Instead of aborting, take the stack trace, print it, perhaps raise to sentry, and kill the specific "work unit" that caused it. This "work unit" can be a thread of a task, depending on how the application is architected. This is precisely what Tokio does: by default, a panic in async code will only bring down the task that panicked inste…

> there could be other issues, like mutex poisoning, which is why nobody uses the stdlib's mutexes. What does everyone use instead?

In the context of Tokio, the tokio's native mutexes / locking primitives. For sync code, parking_lot is the de facto replacement for the stdlib's ones.

I don't remember where I read it, but it has been admitted that having synchronization primitives with poisoning in the stdlib was a mistake, and "simpler" ones without it.

for context: a mutex is poisoned should a panic occur while the mutex is held. it is then assumed the guarded data to be broken or in an unknown state, thus poisoned.

Re: A year of Rust in ClickHouse

#80

Post by pornel »

alserio wrote: post #20
IshKebab wrote: post #4

This guy seems to be both very positive about Rust and unfairly cynical about it at the same time... Rust is a really fantastic language but having worked on a mixed C++/Rust codebase I can see why they had so many issues. Rust just wasn't really designed with C++ interop in mind so it's kind of painful to use them together. Impressive that they made it work.

Check the post date. It was published on April first

They link to actual issues in their bug tracker, so if it was a joke, it was an impressive long con.
Post reply on HN