Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust – A hard decision pays off

pinecone.io

Post reply on HN
Print view View original

41–50 of 386 posts

Re: Rust – A hard decision pays off

#41

Post by throwaway894345 »

> Dev velocity, which was supposed to be the claim to fame of Python, improved dramatically with Rust.

I don't doubt this in the least. I've been a professional Python developer for 15 years, and I can't believe Python ever had the reputation for "high dev velocity" beyond toy examples. In every real world code base I've worked in, Python has been a strict liability and the promise that you can "just rewrite the slow parts in C/C++/Numpy/etc" has never held (you very often spend at least as much time marshaling the data to the target language format than you gain by processing in the faster language and you have all of the maintainability problems of C/C++/Numpy). Python trades developer velocity for poor runtime performance.

I don't think Rust is the paragon of high dev velocity languages either, but it seems to be more predictable. You don't have something that works as a prototype, but then you run into a bunch of performance problems (which virtually cannot be worked around) when you go to productionize, nor do you get all of the emergent quality and maintainability issues that come about from a dynamic type system (and last I checked, Python's optional static type system was still pretty alpha-grade).

I strongly recommend avoiding Python for new projects. Use Go if you're looking for an easy GC language with max productivity and a decent performance ceiling. Use Rust if you're writing really high performance or correctness-is-paramount software. I'm sure there are other decent options as well (I've heard generally good things about TypeScript, but I'd be concerned about its performance even if it is better than Python).

Re: Rust – A hard decision pays off

#43

Post by benreesman »

ramoz wrote: post #12

> Simultaneous fetches of thousands (sometimes tens of thousands) of objects started becoming inefficient, especially when fetching from collections of tens of millions of objects Why not try leveldb. We're doing random reads of 170,000 vectors (/130M) a second. No startup needed.

RocksDB started as a fork of LevelDB (Dean/Ghemawat).

The performance imperatives have changed with the hardware, but NAND flash at that time had an asymmetry between reads and writes in terms of the amount of data: once you were writing even one byte you had effectively paid for writing a whole block and were therefore incentivized to get your money’s worth and write to a “log”, which then would be “merged”, in some “structured” way, hence LSM.

This is old news these days but it was quite the novelty at the time!

Re: Rust – A hard decision pays off

#44

Post by benreesman »

throwaway894345 wrote: post #41

> Dev velocity, which was supposed to be the claim to fame of Python, improved dramatically with Rust. I don't doubt this in the least. I've been a professional Python developer for 15 years, and I can't believe Python ever had the reputation for "high dev velocity" beyond toy examples. In every real world code base I've worked in, Python has been a strict liability and the promise that you can "just rewrite the slow…

I think Rust deserves a lot of credit for popularizing the notion of “constraints as power”, but excessive enthusiasm around it does seem to obscure the rich traditions of PLT that it has more than liberally borrowed from.

There are more rungs on the ladder.

Re: Rust – A hard decision pays off

#45

Post by FartyMcFarter »

throwaway894345 wrote: post #41

> Dev velocity, which was supposed to be the claim to fame of Python, improved dramatically with Rust. I don't doubt this in the least. I've been a professional Python developer for 15 years, and I can't believe Python ever had the reputation for "high dev velocity" beyond toy examples. In every real world code base I've worked in, Python has been a strict liability and the promise that you can "just rewrite the slow…

If people only used Python as originally intended (as a language to write small scripts in), this kind of problem never would have happened.

Re: Rust – A hard decision pays off

#46

Post by ncmncm »

diarrhea wrote: post #39
ncmncm wrote: post #30

Earlier quoted context omitted.

Yes. This comes across as "we are so smart, this project wasn't hard enough to demand our full attention". Coding in a manifestly immature language signals that you are not really serious about being used where long-term support across a variety of platforms may be important. The language is still more likely than not to fizzle. If it does, it makes your flagship project a niche player.

> The language is still more likely than not to fizzle. What makes you think that?

Fizzling is the normal fate of any new language, absent a miracle. Only a tiny handful of languages get a miracle.

In any given week, more people pick up C++ or Javascript to use professionally than the total number now employed to code Rust. Not to fizzle, it has to increase its adoption rate by orders of magnitude, but its fans are almost uniformly hostile toward any measure that could make it easier to adopt.

Re: Rust – A hard decision pays off

#47

Post by golergka »

throwaway894345 wrote: post #41

> Dev velocity, which was supposed to be the claim to fame of Python, improved dramatically with Rust. I don't doubt this in the least. I've been a professional Python developer for 15 years, and I can't believe Python ever had the reputation for "high dev velocity" beyond toy examples. In every real world code base I've worked in, Python has been a strict liability and the promise that you can "just rewrite the slow…

> I've heard generally good things about TypeScript, but I'd be concerned about its performance even if it is better than Python

I wouldn't recommend Typescript for CPU-bound tasks, but for I/O-bound tasks with complicated business logic it's type system is on the order of magnitude better than Go, and makes you able to get really close to the "make invalid states of the system a type error" ideal of functional languages like Haskell, while still being a lot more approachable.

Re: Rust – A hard decision pays off

#48

Post by klabb3 »

RcouF1uZ4gsC wrote: post #20

> Dev velocity, which was supposed to be the claim to fame of Python, improved dramatically with Rust. Built-in testing, CI/CD, benchmarking, and an overzealous compiler increased engineers’ confidence in pushing changes, and enabled them to work on the same code sections and contribute simultaneously without breaking the code base. Most impressively though, real time operational events dropped almost to zero overnig…

> Making copies and using Rc can bypass a lot of the trickier parts of the borrow checker and still allow you to access much of the benefits of Rust in terms of performance and correctness, especially compared to a dynamic language like Python.

> As you get more experience, you can then use references and other techniques to eliminated unnecessary copies, etc.

I've always found this advice to be infeasible in practice, despite sounding like a reasonable pitch.

Arc allows you to avoid thinking about ownership, which is largely what dictates the architecture when writing idiomatic Rust. Plastering on ownership later is going to change the code so much you might as well rewrite it .

Moreover, if you want to prototype (with Arcs, clones) a new part of the code, while integrating with a more mature part of the code base, you still need to adhere to the ownership structure.

In fact, this is one of my main issues with Rust, that it's a terrible language for prototyping. I don't have a solution, and I'm not blaming Rust – it's still a great language. But you have to have a very solid design/mental model before you start coding.

Re: Rust – A hard decision pays off

#49

Post by sidlls »

runevault wrote: post #23
throwaway642012 wrote: post #9

I don't want to be dismissive but how much of this success belongs to the team of rock solid senior engineers with enough experience under their belt rather than rewrite in Rust?

You can give some credit to the fact they were writing it a second time certainly, but as far as we know at least a lot of the people were the same ones who were central to writing V1. So certainly there was some lift from already doing it once, but if they went from a lot of subtle bugs they struggled to track down to having far fewer of them, and those bugs tended to be the types of issue the rust compiler and/or l…

It’s likely they could have chosen to stay with Python, C, and C++ and achieved similar results. I’d bet a pretty penny that the fact they were building on lessons learned with the rewrite accounts for almost 100% of the success here.
Post reply on HN