Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rapidstring: Maybe the fastest string library ever

github.com

Post reply on HN
Print view View original

21–30 of 62 posts

Re: Rapidstring: Maybe the fastest string library ever

#21

Post by johnboyer »

sbjs wrote: post #20
johnboyer wrote: post #15

Hey there, creator of the library here. Strangely enough my initial post about this on HN received no attention, but better late than never! If any of you have some questions or feedback, I would be delighted to hear it.

Hey man don't worry about that, that's just the way of life. People notice things when they do, that part's not up to us. I've seen articles here posted years after they were written. Very nice and thoroughly documented code! Well done! What gave you the initial idea to write this maybe-fastest-ever string library? Did you just have an idea one day of how it could be done and you just went for it? Or did you have a p…

I just really needed a string library written in C, and it didn't seem as though there were many options. The first thing I found was the Simple Dynamic Strings library, but it wasn't maintained and depended on GCC extensions, so I decided to write my own. After getting a basic functional concatenation, I benchmarked it against std::string to see how slow it was, and to my great surprise, it was actually faster. From that point, I realized I can actually write some pretty fast code, and I decided to make the library centered around that.

Re: Rapidstring: Maybe the fastest string library ever

#22

Post by lifthrasiir »

johnboyer wrote: post #16
mamcx wrote: post #7

Earlier quoted context omitted.

I think instead is better to title this as "maybe the fastest ascii string library"

Unicode is supported with UTF-8, only different character types aren't supported because generics are messy in C. I thought of accepting void* to support wchar_t and others, but some performance penalties came with it so I decided against it.

No, having different character types (I believe you are referring to C11's `char16_t` and `char32_t`?) is not a requirement for Unicode support. At the very least you need to have a single function or two that...

* Receives a string expected to be encoded in UTF-8, and an offset to it expected to be a UTF-8 sequence boundary.

* Scans forward or backward for the next or previous UTF-8 sequence boundary.

* Optionally returns the code point for the scanned UTF-8 sequence.

* Has proper error handling for every imaginable cases: out of boundary, not a boundary, not a valid UTF-8 sequence. (OOB case needs to be handled because it will be the end condition of the iteration. Preferably should be distinct from other error conditions.)

Every other functionality can build upon this little function, in particular the iteration and UTF-8 validation will be trivial. The full Unicode support including case mapping, folding, normalization and property lookup will of course require a not-so-small table but is not strictly necessary anyway.

Björn Höhrmann's Flexible and Economical UTF-8 Decoder [1] will be handy for a concise implementation.

[1] https://bjoern.hoehrmann.de/utf-8/decoder/dfa/

Re: Rapidstring: Maybe the fastest string library ever

#24

Post by dwheeler »

MrEldritch wrote: post #9
abenedic wrote: post #6

Earlier quoted context omitted.

I don't really know why you are being downvoted. There are a lot of string libraries that do not work well with Unicode strings. If you natively speak any non-European language it can be hard to use those libraries to effectively solve your problems. Not to bad mouth this project, but we don't really need a fast incomplete library.

In the modern development environment, I think it's genuinely fair to say that something which doesn't support Unicode strings cannot be claimed to support strings. It's like saying you have the fastest integer math library ever written, with the minor caveat that it only supports numbers smaller than 256 because it achieves that speed by being simply a hardcoded lookup table, and crashes if you give it anything else…

It appears to support Unicode just fine, you just have to use UTF-8. I find that in most cases you should only use UTF-8 anyway, so in most cases that is not a problem.

Re: Rapidstring: Maybe the fastest string library ever

#25

Post by slavik81 »

mamcx wrote: post #7
rurban wrote: post #3

I like the stack trick. But it's still just a membuf library, without any string support. No encoding, no Unicode, no upper/lower/fc/norm support, which would be important to compare or find strings. And coreutils (e.g grep) still have no unicode support. It's 2018, not the seventies anymore. Unicode strings need to be normalized to be able to be found.

I think instead is better to title this as "maybe the fastest ascii string library"

I don't see anything encoding-specific in there. It may not be a Unicode string library, but it's also not an ASCII string library. Rurban was right. It's really just a nice membuf library—but there's nothing wrong with that! You could build a great Unicode or ASCII string library on top of this.

Re: Rapidstring: Maybe the fastest string library ever

#27

Post by vbezhenar »

dwheeler wrote: post #24
MrEldritch wrote: post #9

Earlier quoted context omitted.

In the modern development environment, I think it's genuinely fair to say that something which doesn't support Unicode strings cannot be claimed to support strings. It's like saying you have the fastest integer math library ever written, with the minor caveat that it only supports numbers smaller than 256 because it achieves that speed by being simply a hardcoded lookup table, and crashes if you give it anything else…

It appears to support Unicode just fine, you just have to use UTF-8. I find that in most cases you should only use UTF-8 anyway, so in most cases that is not a problem.

If you're using WinAPI, you have to use UCS-16. Converting strings for every call would be abysmal to performance.

Re: Rapidstring: Maybe the fastest string library ever

#28

Post by slavik81 »

johnboyer wrote: post #15

Hey there, creator of the library here. Strangely enough my initial post about this on HN received no attention, but better late than never! If any of you have some questions or feedback, I would be delighted to hear it.

I like it. That's a good set of principles and a very clean API.

I had a bit of trouble finding stuff in the documentation at first. You direct people towards Modules on the very first line of the Main Page but it's too subtle. When scanning the page looking for how to get to the list of all the classes and functions in the project, my eyes are drawn to the headings which are basically all irrelevant. It would help to add some headings beyond the autogenerated ones, and to provide a link from the struct documentation to set of functions that manipulate that struct (if possible). Additionally, the top of every page says "rapidstring 0.1.0" and I'm not sure what that number means. The version is listed on the main page as 1.0.0.

The documentation is well-written, which I think is a really good sign, and the problems I mentioned are only really an issue for the very first time somebody tries to use the documentation.

Re: Rapidstring: Maybe the fastest string library ever

#29

Post by hrydgard »

vbezhenar wrote: post #27
dwheeler wrote: post #24

Earlier quoted context omitted.

It appears to support Unicode just fine, you just have to use UTF-8. I find that in most cases you should only use UTF-8 anyway, so in most cases that is not a problem.

If you're using WinAPI, you have to use UCS-16. Converting strings for every call would be abysmal to performance.

For most software it's perfectly fine to convert. What Windows API call that takes string input do you call so often that it would be an issue?
Post reply on HN