Why did we choose Rust to develop TiKV? • Hacker News Board

Re: Why did we choose Rust to develop TiKV?

#141

Post by Tyr42 » Wed, Sep 27, 2017, 3:27 PM UTC

Koshkin wrote: post #105

pjc50 wrote: post #23

Earlier quoted context omitted.

And do all the C++ libraries take only smart pointers as arguments and return only smart pointers as return values?

They don't, and it is (or should be) considered bad practice for functions, in general, to take/return smart pointers. Normally these should only be used as variables or class members; functions, on the other hand, should take/return raw pointers (except some special cases) or, better yet, references (except when there is a need to check for null value).

I disagree? Functions which allocate things returning unique_ptr is much clearer than returning a raw pointer. Clear ownership being passed, as opposed to maybe just having a view into some internal buffer.

I mean, if you only view functions as being called for side effects, then yeah maybe. But if you're constructing a data pipeline, unique_ptr in and out makes a lot of sense.

Re: Why did we choose Rust to develop TiKV?

#142

Post by alexnewman » Wed, Sep 27, 2017, 3:29 PM UTC

Thaxll wrote: post #137

steveklabnik wrote: post #113

Earlier quoted context omitted.

What parts of Go tooling do you miss in Rust?

- Fast / cross compile - Benchmark / profiling are much better ( https://blog.golang.org/profiling-go-programs ) - go fmt / go vet - go doc All of that that is supported by default by the Go team on all platforms, meaning it works well and I don't have to worry of using a 3rd party Cargo package.

Go doc and go fmt have rust equivalents. What’s better at profiling?

Re: Why did we choose Rust to develop TiKV?

#143

Post by alexnewman » Wed, Sep 27, 2017, 3:30 PM UTC

steveklabnik wrote: post #113

Thaxll wrote: post #95

Earlier quoted context omitted.

What do you mean by not adequate? Go tooling is way better than Rust. The only thing that Rust does better in that field is package dependency and it's going to change soon for Go.

What parts of Go tooling do you miss in Rust?

The other way around closed source development is weird in go.

Re: Why did we choose Rust to develop TiKV?

#144

Post by alexnewman » Wed, Sep 27, 2017, 3:33 PM UTC

eikenberry wrote: post #5

TLDR; the author likes rust and wanted to use it. The article reads like some dev's rationalizing what they want to do to the management. These types of things are fine, but as a dev to a dev it is obvious that they just want to use this cool tech. Good for them.

Nonsense. I love go and rust but I would never develop a database in a garbage collected language again. Btw I have done it a few times. Go is better than java but it still

Can’t call into c as fast Has micro gc pauses which effect performance Doesn’t give me explicit control of the hardware

Go is great for middleware but he authors go it right

Re: Why did we choose Rust to develop TiKV?

#145

Post by dullgiulio » Wed, Sep 27, 2017, 3:36 PM UTC

alexnewman wrote: post #140

Kubuxu wrote: post #130

Earlier quoted context omitted.

go fmt

What’s wrong with cargo fmt

It has options ;)

Re: Why did we choose Rust to develop TiKV?

#146

Post by Santosh83 » Wed, Sep 27, 2017, 3:37 PM UTC

jerf wrote: post #112

sriram_malhar wrote: post #68

Considering that their team likes Go, it seems strange to me that they would consider Rust over Go for the storage layer. A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor. The big wins in that space are architectural, not language specific.

"A storage layer should be IO-bound, and should hardly trouble the CPU; the choice of language really should not be a determining factor." This used to be true, but it's out-of-date now. You can now get a network pipe in to a system that a rather beefy multi-core CPU using a user-space TCP stack can barely keep up with, let alone do any real work, and if you can scrape up the PCI express lanes, putting a few of the l…

And cache. Compiled languages tend to be more cache friendly, which contributes a lot to speed of execution.

And too many apps still do cause CPU spikes, often for quite a bit of time, and no doubt much potential for optimisation lies there. The popular perception that CPUs are fast enough to deal with nearly every workload is inaccurate. Even an innocuous bit of Javascript on a webpage, probably doing some trivial stuff, causes 100% usage for several seconds.

Re: Why did we choose Rust to develop TiKV?

#147

Post by imtringued » Wed, Sep 27, 2017, 3:39 PM UTC

cyber1 wrote: post #60

zerr wrote: post #58

Earlier quoted context omitted.

I don't think ARC is an issue here - it's just a deterministic reference counting - a la implicit using of smart pointers. Besides, Swift also has some unsafe pointer types as well.

Yes, you right according to "deterministic" but you forget about overhead. Rust gives you more safe control on memory management and in most cases, you can always write the fast and efficient safe program without the overhead. Rust give different smart pointers and even sometimes when you 100% understand what are you doing you can use unsafe raw ptr.

For certain programs a GC can basically make a programming language completely useless (no shared libraries with C abi, no realtime audio, videogames with stutter) regardless how fast the programming language is. This is not the case for swift. It's merely slower than rust.

Re: Why did we choose Rust to develop TiKV?

#148

Post by Thaxll » Wed, Sep 27, 2017, 3:40 PM UTC

alexnewman wrote: post #142

Thaxll wrote: post #137

Earlier quoted context omitted.

- Fast / cross compile - Benchmark / profiling are much better ( https://blog.golang.org/profiling-go-programs ) - go fmt / go vet - go doc All of that that is supported by default by the Go team on all platforms, meaning it works well and I don't have to worry of using a 3rd party Cargo package.

Go doc and go fmt have rust equivalents. What’s better at profiling?

There is nothing out of the box for profiling ( but it's bit different since no GC and use C/C++ tools ). What's the equivalent of doc godoc -http=:6060 ( it sounds silly by last time I was in a plane and was able to browse the Go doc Like if I had internet access )

Re: Why did we choose Rust to develop TiKV?

#149

Post by abiox » Wed, Sep 27, 2017, 3:40 PM UTC

cyber1 wrote: post #51

zerr wrote: post #40

Earlier quoted context omitted.

On the other hand I can't wait when Swift becomes general, non-Apple language, available on most platforms (including the most popular one) "with batteries" - that will be the end of Go and Rust I think :)

Swift can't replace Rust in system programming domain because he uses ARC for memory management but Rust gives you choices.

> system programming

worth mentioning that this term has a broader meaning than how some rust evangelists use it.

Re: Why did we choose Rust to develop TiKV?

#150

Post by steveklabnik » Wed, Sep 27, 2017, 3:43 PM UTC

Thaxll wrote: post #137

steveklabnik wrote: post #113

Earlier quoted context omitted.

What parts of Go tooling do you miss in Rust?

- Fast / cross compile - Benchmark / profiling are much better ( https://blog.golang.org/profiling-go-programs ) - go fmt / go vet - go doc All of that that is supported by default by the Go team on all platforms, meaning it works well and I don't have to worry of using a 3rd party Cargo package.

Thanks!

Compile speed is certainly a thing go is excellent at. Cross-compiling, Rust is almost as good, but not quite yet, working on it!

Benchmark stuff is not yet stable, but for profiling, you don't need a special tool for Rust; all the C and C++ tooling works.

fmt/vet is rustfmt and clippy, which work today, and soon will be distributed with Rust tool.

rustdoc already ships with Rust.

So, my takeaway is mostly, needs some polish and shipping with the compiler. Thanks!