Making Rust as Fast as Go • Hacker News Board

Re: Making Rust as Fast as Go

#171

Post by dcow » Mon, May 04, 2020, 1:48 AM UTC

arcticbull wrote: post #102

chrfrasco wrote: post #52

Hey all, as some keen-eyed commenters have pointed out, it looks like the rust program is not actually equivalent to the go program. The go program parses the string once, while the rust program parses it repeatedly inside every loop. It's quite late in Sydney as I write this so I'm not up for a fix right now, but this post is probably Fake News. The perf gains from jemalloc are real, but it's probably not the alloca…

Thanks for following up. Just as an FYI, there's a few bugs in your implementation, the most obvious one is the use of ".len()" in a number of places interspersed with ".chars().count()". These two return different values. ".len()" returns then number of UTF-8 bytes in the input string, which for ASCII is the same as ".chars().count()" obviously, but if you do attempt any Unicode characters, your function won't work.…

"Fake News" entered the realm (not even recently, might I add) of popular misuse. Actually, is there a term for: language/grammar incorrectly used because society has developed a familiar "meme" use?

Common examples:

* Look at this dank "meme".

meme has come to mean "a picture shared on the internet that has words on it".

* Let's [have a] "cheers".

It's a toast. You say "cheers" when you toast.

* You missed Suzie and I's party last night.

It's Suzie and my party. This one is particularly annoying because it's made it way past editors and into writing, screenplay, etc.

Re: Making Rust as Fast as Go

#172

Post by entha_saava » Mon, May 04, 2020, 3:10 AM UTC

arcticbull wrote: post #117

blablabla123 wrote: post #94

Earlier quoted context omitted.

I'd be really surprised to hear that Go is supposed to be faster than Rust. Of course Rust is a bit newer but to me it always sounded like Go is fast because it's static but it doesn't have to be high-speed if that would sacrifice conciseness. Given that this is an artifical example, this here looks more realistic: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Rust and Go are contemporaries. Rust started in 2006 at Mozilla, and the first Go public release from Google was in 2009, meaning it probably started at the same time. Except of course all the Plan 9 garbage (like Go's hand-rolled assembler) brought in to underpin Go from the 80s ;)

> Except of course all the Plan 9 garbage (like Go's hand-rolled assembler) brought in to underpin Go from the 80s

This is unfair criticism. If Go had used LLVM, it would affect its selling point (fast compile times) and authors knew the plan 9 toolchain well.

Go the language feels like it is from 80s. But its toolchain is not at all bad. LLVM monoculture is the last thing one would want. Obligatory reminder that LLVM has its flaws too..

Re: Making Rust as Fast as Go

#173

Post by codeflo » Mon, May 04, 2020, 4:28 AM UTC

jfkebwjsbx wrote: post #137

codeflo wrote: post #30

Earlier quoted context omitted.

That’s partly true, but the stable interface on Windows is not the libc, but the kernel32/user32 functions like VirtualAlloc, CreateFileW etc. Those are stable since the NT days. The libc functions like malloc and fopen are a compatibility layer above that and unfortunately switch places every few years. Currently they are delivered by a combination of a Windows update and a redistributable package, which makes it a…

Do you really need to ship them? I thought libc (CRT?) in Windows was a given, and what used to be redistributed was only the C++ ones. Is not that the case?

Thanks for the correction, Microsoft calls it the C Runtime/CRT. It‘s unfortunately complicated, and I’ll completely ignore static linking, which is possible, but not supported in many scenarios involving DLLs.

It used to be the case that the CRT shipped with Windows (msvcrt.dll). That file is now considered legacy/deprecated and is no longer supported by current compilers. For several years after that, you always had to redistribute the CRT (msvcrtXXX.dll), even for pure C support.

The current state of affairs is that the CRT is split into several files, some of which come with Windows update (the so-called UCRT) and some of which are compiler-specific and have to be redistributed. C++ std support requires yet more files.

This document gives an overview: https://docs.microsoft.com/en-us/cpp/c-runtime-library/crt-l...

Re: Making Rust as Fast as Go

#174

Post by fluffything » Mon, May 04, 2020, 6:57 AM UTC

jfkebwjsbx wrote: post #168

fluffything wrote: post #139

Earlier quoted context omitted.

> It does not make sense either to expect someone to use bleeding edge libraries from cargo yet use an old rustc compiler. Of course it does. Many software users are stuck on multiple-year-old toolchains for various reasons, yet these systems still need to be able to handle unicode properly. > They can easily update it if needed. No, they cannot. Many users are stuck in older windows versions, linux versions, LTS lin…

> Of course it does. Many software users are stuck on multiple-year-old toolchains for various reasons, yet these systems still need to be able to handle unicode properly. So? Use the external library then. One thing does not preclude the other. > No, they cannot. Many users are stuck in older windows versions, linux versions, LTS linux versions, etc. because of their organization or their clients requirements. I wor…

> So? Use the external library then. One thing does not preclude the other.

That's what everybody already does? You are proposing to, instead of doing that, move that library into the standard library where it cannot ever change.

> You can provide standard Unicode handling that is good enough for 99% software out there.

That's already in std? 99% of the code doesn't need to handle unicode grapheme clusters, because it doesn't deal with unicode at all.

You are suggesting moving something into standard that would make unicode software harder to update, and would make the standard library huge (>20mb larger) for all programs (the unicode tables take a lot of binary size), even those that don't use unicode, to try to solve a problem that does not exist.

> I work in such an organization and no, we cannot use third-party packages

If a Rust user cannot write `cargo add unicode-segmentation`, they have bigger problems than not being able to handle grapheme clusters. You can't run async code because you don't have an executor, you can't do http because the standard library doesn't support that, you can't solve partial differential equations, or do machine learning, or pretty much anything interesting with Rust.

That's bad for you, but the solution isn't to make Rust bad for everybody else instead.

If your organization doesn't let you use third-party packages, then write your own: that's what your organization wants you to do.

Some organizations want all code in CamelCase, they can't use the standard library at all. But the solution isn't to make Rust case insensitive, or to prove a 2nd standard library API for those organizations.

Re: Making Rust as Fast as Go

#175

Post by fluffything » Mon, May 04, 2020, 7:04 AM UTC

afiori wrote: post #162

arcticbull wrote: post #151

Earlier quoted context omitted.

IMO it's not as clear-cut as you make it out to be. It's a pretty arbitrary line to exclude full Unicode support from the standard library. There's a ton of stuff in libstd that could be supported as third-party crates. I don't disagree with what the Rust team has done, and I think there could be a world in which the compiler team also releases first-party crates with "enhanced" functionality beyond just libstd. I co…

the point would be that the standard library should be forward compatible while crates should be backward compatible. so that current crates work with old version of compilers/toolchains. this applies here as each new Unicode standard requires an update of the Unicode crate. ideally the best case would be to make it so that in 20 years Rust 1.0 can still use the most updated version of Unicode fragmentation. similarl…

Today, if you write `cargo add unicode-segmentation` to a Rust 1.0 program, you can use the latest version of Unicode. You can also add an older version of the library and use an older version if you want.

Adding unicode segmentation to the standard library and making all Rust binaries, most of which don't do unicode segmentaiton, 20 Mb larger by unnecessarily bundling the unicode tables, makes no sense.

As you see in the other thread, the problem the parent poster has is that their organizaiton doesn't let them use crates from crates.io.

That's a stupid policy for a language like Rust, and the solution isn't "move all crates of crates.io into the standard library". The solution there is for them to write their own unicode-segmentation code (and async executor, and http stack, and linear algebra, and... the whole world), since that is what their organization wants them to do. That's a stupid policy, but it is their own stupid fault.

Most organizations either allow using crates from crates.io, or have a vetting policy, where you just submit a 1 liner email saying "I need to do unicode-segmentation and there is only one library for that that's used by Firefox here: ...". Somebody then checks licenses and stuff, and gives you approval in a couple of days. If their organization doesn't have such processes, then i'm sorry for them, but I don't see how this is in any way the standard library fault.

Whatever their reasons are "my org doesn't allow us to use cargo" isn't a good reason to move something into the standard library.

Re: Making Rust as Fast as Go

#176

Post by fluffything » Mon, May 04, 2020, 7:10 AM UTC

arcticbull wrote: post #151

fluffything wrote: post #139

Earlier quoted context omitted.

> It does not make sense either to expect someone to use bleeding edge libraries from cargo yet use an old rustc compiler. Of course it does. Many software users are stuck on multiple-year-old toolchains for various reasons, yet these systems still need to be able to handle unicode properly. > They can easily update it if needed. No, they cannot. Many users are stuck in older windows versions, linux versions, LTS lin…

IMO it's not as clear-cut as you make it out to be. It's a pretty arbitrary line to exclude full Unicode support from the standard library. There's a ton of stuff in libstd that could be supported as third-party crates. I don't disagree with what the Rust team has done, and I think there could be a world in which the compiler team also releases first-party crates with "enhanced" functionality beyond just libstd. I co…

> For the record, I also disagree with your assertion that "easily done in rust" should be extended to include "...by importing a third-party framework." In that sense anything is easy to do in any language where a third-party framework exists. I'm confident it's just as easy in go.

Have you tried doing that in C++? Doing that in a cross-platform way (or even in a single platform) is anything but easy, because you don't have a tool like cargo, you have to change your build system, do the dependency resolution manually, etc.

So no, such a library existing does not imply that using that is easy.

In Rust, you just need to write `cargo add unicode-segmentation` once in a project, and then you can directly use the library API. There is literally nothing else for you to do.

That's a pretty low barrier of entry, and something you will need to do 100s of times per project anyway, because the standard library is minimal by design.

If you prefer languages without a minimal standard library, then Rust isn't for you. Go try Python, where half of the standard library has a warning saying "deprecated: use this other better external dependency instead; adding this to the standard library for convenience was the worst idea ever and now we need to maintain all this code forever".

Re: Making Rust as Fast as Go

#177

Post by afiori » Mon, May 04, 2020, 7:17 AM UTC

fluffything wrote: post #175

afiori wrote: post #162

Earlier quoted context omitted.

the point would be that the standard library should be forward compatible while crates should be backward compatible. so that current crates work with old version of compilers/toolchains. this applies here as each new Unicode standard requires an update of the Unicode crate. ideally the best case would be to make it so that in 20 years Rust 1.0 can still use the most updated version of Unicode fragmentation. similarl…

Today, if you write `cargo add unicode-segmentation` to a Rust 1.0 program, you can use the latest version of Unicode. You can also add an older version of the library and use an older version if you want. Adding unicode segmentation to the standard library and making all Rust binaries, most of which don't do unicode segmentaiton, 20 Mb larger by unnecessarily bundling the unicode tables, makes no sense. As you see i…

(as a maybe unneeded clarification, my previous comment agrees with you)

I agree that something like unicode-segmentation should not be in the bundled standard library. specifically because the std should be forward compatible.

In my opinion it is why foundational libraries should strive to be on Rust 2015 edition rather than the latest.

Re: Making Rust as Fast as Go

#178

Post by Robin_Message » Mon, May 04, 2020, 7:31 AM UTC

dcow wrote: post #171

arcticbull wrote: post #102

Earlier quoted context omitted.

Thanks for following up. Just as an FYI, there's a few bugs in your implementation, the most obvious one is the use of ".len()" in a number of places interspersed with ".chars().count()". These two return different values. ".len()" returns then number of UTF-8 bytes in the input string, which for ASCII is the same as ".chars().count()" obviously, but if you do attempt any Unicode characters, your function won't work.…

"Fake News" entered the realm (not even recently, might I add) of popular misuse. Actually, is there a term for: language/grammar incorrectly used because society has developed a familiar "meme" use? Common examples: * Look at this dank "meme". meme has come to mean "a picture shared on the internet that has words on it". * Let's [have a] "cheers". It's a toast. You say "cheers" when you toast. * You missed Suzie and…

If it's the kind of things people say, you want it in screenplays, otherwise Brooklyn 99 would sound like Shakespeare.

Re: Making Rust as Fast as Go

#179

Post by [deleted] » Mon, May 04, 2020, 7:36 AM UTC

entha_saava wrote: post #172

arcticbull wrote: post #117

Earlier quoted context omitted.

Rust and Go are contemporaries. Rust started in 2006 at Mozilla, and the first Go public release from Google was in 2009, meaning it probably started at the same time. Except of course all the Plan 9 garbage (like Go's hand-rolled assembler) brought in to underpin Go from the 80s ;)

> Except of course all the Plan 9 garbage (like Go's hand-rolled assembler) brought in to underpin Go from the 80s This is unfair criticism. If Go had used LLVM, it would affect its selling point (fast compile times) and authors knew the plan 9 toolchain well. Go the language feels like it is from 80s. But its toolchain is not at all bad. LLVM monoculture is the last thing one would want. Obligatory reminder that LLV…

[deleted]

Re: Making Rust as Fast as Go

#180

Post by arcticbull » Mon, May 04, 2020, 7:48 AM UTC

fluffything wrote: post #175

afiori wrote: post #162

Earlier quoted context omitted.

the point would be that the standard library should be forward compatible while crates should be backward compatible. so that current crates work with old version of compilers/toolchains. this applies here as each new Unicode standard requires an update of the Unicode crate. ideally the best case would be to make it so that in 20 years Rust 1.0 can still use the most updated version of Unicode fragmentation. similarl…

Today, if you write `cargo add unicode-segmentation` to a Rust 1.0 program, you can use the latest version of Unicode. You can also add an older version of the library and use an older version if you want. Adding unicode segmentation to the standard library and making all Rust binaries, most of which don't do unicode segmentaiton, 20 Mb larger by unnecessarily bundling the unicode tables, makes no sense. As you see i…

> Adding unicode segmentation to the standard library and making all Rust binaries, most of which don't do unicode segmentaiton, 20 Mb larger by unnecessarily bundling the unicode tables, makes no sense.

Isn't that what dead code stripping is for?