Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speed of Rust vs. C

kornel.ski

Post reply on HN
Print view View original

511–520 of 546 posts

Re: Speed of Rust vs. C

#511

Post by simias »

simias wrote: post #267
totalperspectiv wrote: post #232

Earlier quoted context omitted.

I’ve been using smartstrings, which is both excellent and maintained. https://github.com/bodil/smartstring

Ah, nice, I was looking at the smallstring package that's appears abandoned. I'll be sure to check this one out. The good thing about having a decent type system is that I expect that transitioning to smartstrings should be painless! Thank you for that.

In case somebody stumbles upon this conversation in the future: I just migrated a project to use smartstrings and it works a bit differently from smallvec. Smallvec lets you decide how big you want to make the static buffer before it allocates, whereas smartstring's static buffer size is alsways `size_of::() - 1`, that is 23 bytes on 64 bit architectures and 11 on 32 bits. If I want, say, a static 128B string smartstring won't do any better than std::string::String.

It's still a very nice lib though, and a smart optimization, but it doesn't cover all of my use cases for small string buffers.

Re: Speed of Rust vs. C

#512

Post by fhunt »

kbenson wrote: post #485
fhunt wrote: post #475

Earlier quoted context omitted.

My question is probably off because I lack the knowledge but how do the commercial games/game engines do this then if this is such a rocket science? Something like Fortnite or an aged GTA do what you've described (downloading assets on demand without any fps-drop) for quite some time now.

The claim isn't that it's impossible, or "rocket science", it's that it's hard to do right and was made much easier. You're bringing up for comparison a game engine that has been in constant development by experts for over two decades (Unreal Engine) and a game engine in constant development for over a decade ago (RAGE). Just because someone makes a professional product using tens or hundreds of millions of dollars d…

Yeah, but just picking one of many requirements in game dev and advocating why lang x can do this better than y ignores all the other checkboxes. Yeah, C++ is nerve-wrecking but Rust can be even more. IIRC there was a thread about Zig vs Rust and why Rust is just the wrong tool (for OPs use case in that case). IDK but there is a reason why C++ dominates game dev and a reason why Rust still struggles with mainstream adoption compared to same agers like Go or TS.

Re: Speed of Rust vs. C

#513

Post by eeZah7Ux »

rstuart4133 wrote: post #459
gspr wrote: post #197

Earlier quoted context omitted.

Rust seems great to me, but aren't we losing a lot by giving up on C's dynamic linking and shared libraries?

To give some context to the parent comment: $ ls -lh $(which grep) $(which rg) -rwxr-xr-x 1 root root 199K Nov 10 06:37 /usr/bin/grep -rwxr-xr-x 1 root root 4.2M Jan 19 09:31 /usr/bin/rg My very unscientific measurement of the startup time of grep vs ripgrep is 10ms when the cache is cold (ie, never run before) and 3ms when the cache is hot (ie, was run seconds prior). For grep even in the cold case libc will already…

Thanks!

Re: Speed of Rust vs. C

#514

Post by pjmlp »

trunnell wrote: post #307
pjmlp wrote: post #146

Earlier quoted context omitted.

Indeed, but the fearless concurrency sales pitch tends to overlook that.

It’s hard to see how you could be arguing in good faith given that your point of “fearless concurrency” being an overstated “sales pitch” has now been answered with multiple substantive answers. Please stop moving the goal posts from “program concurrency” to “distributed or multi-process transactions.” It subtracts from the conversation. Rust is honest about what it does and doesn’t do.

Rust is honest, https://doc.rust-lang.org/nomicon/races.html

I am not moving goal posts, rather talking about issues that many apparently lack the knowledge to understand how many variants of data races exist in an application.

Quite understanable given the majority of developers that keep writing single threaded applications.

Re: Speed of Rust vs. C

#515

Post by pjmlp »

shakow wrote: post #504
pjmlp wrote: post #317

Earlier quoted context omitted.

Indeed, yet that is not how many Rust advocacy blog posts sell it.

Such as?

Any blog post that gives examples accessing shared variables and never goes beyond anything else, leaving to the reader that it works the same way regardless of what resources are being accessed.

https://blog.knoldus.com/how-we-can-do-fearless-concurrency-...

Now rewrite the same blog post using SQL queries to update the same counter variable on a table row.

The Rustonomicon is quite clear that it wouldn't work,

> However Rust does not prevent general race conditions.

https://doc.rust-lang.org/nomicon/races.html

Re: Speed of Rust vs. C

#516

Post by pjmlp »

simias wrote: post #169
pjmlp wrote: post #125

Earlier quoted context omitted.

Accessing files or database content from multiple threads without proper locking, or transactions, in place will compile just fine.

Rust has the expressive power to design an API that explicitly makes it impossible if you so desire though. For instance by using an API similar to mutexes where you have to lock to access the contents.

> However Rust does not prevent general race conditions.

https://doc.rust-lang.org/nomicon/races.html

Re: Speed of Rust vs. C

#517

Post by simias »

pjmlp wrote: post #516
simias wrote: post #169

Earlier quoted context omitted.

Rust has the expressive power to design an API that explicitly makes it impossible if you so desire though. For instance by using an API similar to mutexes where you have to lock to access the contents.

> However Rust does not prevent general race conditions. https://doc.rust-lang.org/nomicon/races.html

I never claimed otherwise. If you want to enforce invariants in your DB API you'll have to implement them yourselves, Rust won't do it for you because Rust doesn't know what a database is. Still, it's perfectly possible to design a DB API is such a way that it would prevent some issues, such as for instance doing changes without an active transaction.

But you're right to point out that Rust won't magically solve all your possible sources of crash and data corruption, it just does make it a lot easier to enforce arbitrary constraints. For instance if you decide that in order to avoid these issues you want all your DB stuff to only run in a single thread you could design your API in a way that would prevent DB handles from being shared between threads, all that enforced by the language. You couldn't accidentally leak a DB handle to a different thread.

Re: Speed of Rust vs. C

#518

Post by burntsushi »

eru wrote: post #491
burntsushi wrote: post #473

Earlier quoted context omitted.

No, it doesn't. It's only theoretically easy to implement. In practice, they explode the size of the underlying FSM. Moreover, in a command line tool, it's somewhat easy to work around that through the `-v` switch and shell pipelining. Paul's talk introduced redgrep is amazing by the way. Give it a watch if you haven't yet: https://www.youtube.com/watch?v=Ukqb6nMjFyk ripgrep's regex syntax is the same as Rust's regex…

> No, it doesn't. It's only theoretically easy to implement. Oh, I didn't say anything about easy! I am on and off working on a Haskell re-implementation (but with GADTs and in Oleg's tagless final interpreter style etc, so it's more about exploring the type system). > In practice, they explode the size of the underlying FSM. You may be right, but that's still better than the gymnastics you'd have to do by hand to ge…

Yeah sorry, I've gotten asked this question a lot. The issue is that building a production grade regex engine---even when it's restricted to regular languages---requires a lot more engineering than theory. And these particular features just don't really pull their weight IMO. They are performance footguns, and IMO, are also tricky to reason about inside of regex syntax.

If you get something working, I'd love to look at it though! Especially if you're building in a tagless final interpreter style. I find that approach extremely elegant.

Re: Speed of Rust vs. C

#519

Post by burntsushi »

rstuart4133 wrote: post #503
burntsushi wrote: post #319

Earlier quoted context omitted.

You're overcomplicating it. When it comes to finite state machines at least, it's very easy to use an ID index instead of the raw pointer itself. That's exactly what the regex crate does. For reference, I am also the author of the regex crate. The only unsafe it uses specific to finite automata is to do explicit elimination of bounds checks in the core hybrid NFA/DFA loop.

> When it comes to finite state machines at least, it's very easy to use an ID index instead of the raw pointer itself. As an old C programmer, the difference between an array index and a pointer caught me by surprise. In C a pointer is just an unchecked offset into memory. A real array index is just a unchecked offset into ... maybe a smaller chunk of raw memory. But in rust, an array index is something that comes w…

I guess so. But note that I didn't equate them. I just said that you can use an ID index instead. For the particular program of FSMs, they work very well.

If bounds checks prove to be a problem, you can explicitly elide them. Indeed, Rust's regex does just that. :-)

Re: Speed of Rust vs. C

#520

Post by burntsushi »

eru wrote: post #492
burntsushi wrote: post #474

Earlier quoted context omitted.

Has anyone built a production grade regex engine using derivatives? I don't think I've seen one. I personally always get stuck at how to handle things like captures or the very large Unicode character classes. Or hacking in look-around. (It's been a while since I've given this thought though, so I'm not sure I'll be able to elaborate much.)

I've made some attempts, but nothing production grade. About large character classes: how are those harder than in approaches? If you build any FSM you have to deal with those, don't you? One way to handle them that works well when the characters in your classes are mostly next to each other unicode, is to express your state transition function as an 'interval map' What I mean is that eg a hash table or an array lets…

> About large character classes: how are those harder than in approaches? If you build any FSM you have to deal with those, don't you?

I mean specifically in the context of derivatives. IIRC, the formulation used in Turon's paper wasn't amenable to large classes.

Yes, interval sets work great: https://github.com/rust-lang/regex/blob/master/regex-syntax/...

This is why I asked if a production grade regex engine based on derivatives exists. Because I want to see how the engineering is actually done.

> What do you want your capture groups to do? Do you eg just want to return pointers to where you captured them (if any)?

Look at any production grade regex engine. It will implement captures. It should do what they do.

> I have an inkling that something inspired by https://en.wikipedia.org/wiki/Viterbi_algorithm might work.

Nothing about Viterbi is fast, in my experience implementing it in the past. :-)

> https://github.com/google/redgrep/blob/main/parser.yy mentions something about capture, but not sure if that has anything to do with capture groups.

It looks like it does, and in particular see: https://github.com/google/redgrep/blob/6b9d5b02753c4ece17e2f...

But that's only for parsing the regex itself. I don't see any match APIs that utilize them. I wouldn't expect to either, because you can't implement capturing inside a DFA. (You need a tagged DFA, which is a strictly more powerful thing. But in that case, the DFA size explodes. See the re2c project and their associated papers.)

If I'm remembering correctly, I think the problem with derivatives is that they jump straight to a DFA. You can't do that in a production regex engine because a DFA's worst case size is exponential in the size of the regex.

Post reply on HN