Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unicorn: C++ Unicode string library

github.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Unicorn: C++ Unicode string library

#11

Post by weinzierl »

aurelian15 wrote: post #4

Looks like a nice project. I'm currently searching for a Unicode library and it appears to me that ICU is the de-facto standard here, which has the benefit of comming pre-installed on pretty much any Linux distribution. Any reason why I should use Unicorn instead? I couldn't find information on how it compares to ICU in the documentation (well, except for the most welcome usage of modern C++).

Comparison with ICU would be interesting but probably unfair given size and age of ICU. Personally I'd like to see it compared to utf8rewind (previously discussed on HN [1]).

[1] https://news.ycombinator.com/item?id=10029979

Re: Unicorn: C++ Unicode string library

#12

Post by fantasticfears »

rspeer wrote: post #6
aurelian15 wrote: post #4

Looks like a nice project. I'm currently searching for a Unicode library and it appears to me that ICU is the de-facto standard here, which has the benefit of comming pre-installed on pretty much any Linux distribution. Any reason why I should use Unicorn instead? I couldn't find information on how it compares to ICU in the documentation (well, except for the most welcome usage of modern C++).

It looks like Unicorn can apply operations (such as regexes) to text that is natively in UTF-8, giving it a distinct advantage over ICU, which was written back when UTF-16 seemed like a good idea and has to convert everything into UTF-16.

It's hard but needed to differentiate between UTF-16 and UChar byte array. UChar byte array are not essentially an well-formed UTF-16 string. Beyond, why bother use UnicodeString? It's fairly easy to use. It covers the detail from your sight.

It's indeed super cool to see a modern Unicode C++ library. But anyway, is it really useful for production usage? The answer could be no. In contrast, ICU was old, battle-tested, compact and well-tested.

Re: Unicorn: C++ Unicode string library

#13

Post by captaincrowbar »

geekone wrote: post #9
captaincrowbar wrote: post #7

Earlier quoted context omitted.

No idea what you mean, sorry. I'm just using Github's automatically generated web pages, so if there's a problem there it's probably a Github issue.

Probably referring to the Documentation link you provide on the GitHub page, and it breaks back button for me too.

I just tried that on several browsers; Safari and Chrome are fine, it seems to be only Firefox that has a problem with that. I have no idea whether that's a bug in Firefox or Github, and either way there's nothing I can do about it, sorry.

Re: Unicorn: C++ Unicode string library

#14

Post by funkaster »

captaincrowbar wrote: post #13
geekone wrote: post #9

Earlier quoted context omitted.

Probably referring to the Documentation link you provide on the GitHub page, and it breaks back button for me too.

I just tried that on several browsers; Safari and Chrome are fine, it seems to be only Firefox that has a problem with that. I have no idea whether that's a bug in Firefox or Github, and either way there's nothing I can do about it, sorry.

yes, you can: publish your docs as real web pages and not a link to the htmlpreview of a file inside your repo. That should fix the problem.

Re: Unicorn: C++ Unicode string library

#15

Post by nly »

rspeer wrote: post #6
aurelian15 wrote: post #4

Looks like a nice project. I'm currently searching for a Unicode library and it appears to me that ICU is the de-facto standard here, which has the benefit of comming pre-installed on pretty much any Linux distribution. Any reason why I should use Unicorn instead? I couldn't find information on how it compares to ICU in the documentation (well, except for the most welcome usage of modern C++).

It looks like Unicorn can apply operations (such as regexes) to text that is natively in UTF-8, giving it a distinct advantage over ICU, which was written back when UTF-16 seemed like a good idea and has to convert everything into UTF-16.

Looks like unicorn is just using PCRE for regex to me.

Re: Unicorn: C++ Unicode string library

#16

Post by nly »

cmrdporcupine wrote: post #5

The unicode portion looks reasonable, but why is it necessary for it to include its own flags, file io, file management, and environment classes? Why is it so many C++ libraries fall into this habit of trying to build one big framework. I'm perfectly happy with gflags -- a unicode library would be nice for my project, but now I won't consider this library.

That's what will happen until there's a defacto/standard library for this stuff. Languages like Python and Go have a wider base in the standard library. C++14 still only gives you platform dependent 'wide' strings, UTF-8 string literals, and UTF-8 conversion... which makes things awkward.

Re: Unicorn: C++ Unicode string library

#17

Post by sdegutis »

maaku wrote: post #2

Your github pages breaks the back button.

That's not fair. It's pretty well known that Github uses JS to hijack page navigation and make it "smoother" for people. And of course that's going to be faulty, and I emailed them years ago when they made the switch, and asked them to make it an optional behavior because I hate it. But that has nothing to do with OP or OP's link or content. It's like judging a book by the book store.

Re: Unicorn: C++ Unicode string library

#19

Post by rspeer »

fantasticfears wrote: post #12
rspeer wrote: post #6

Earlier quoted context omitted.

It looks like Unicorn can apply operations (such as regexes) to text that is natively in UTF-8, giving it a distinct advantage over ICU, which was written back when UTF-16 seemed like a good idea and has to convert everything into UTF-16.

It's hard but needed to differentiate between UTF-16 and UChar byte array. UChar byte array are not essentially an well-formed UTF-16 string. Beyond, why bother use UnicodeString? It's fairly easy to use. It covers the detail from your sight. It's indeed super cool to see a modern Unicode C++ library. But anyway, is it really useful for production usage? The answer could be no. In contrast, ICU was old, battle-tested…

I'm talking about using UTF-8 as the string representation, not UChars. UChars are an artifact of UTF-16, and thus require converting all text on input and output, unless you work in a Windows API world where I/O is UTF-16.

Modern programming languages such as Rust gain efficiency by working with unmodified UTF-8. All you lose is constant-time arbitrary indexing, which is a bad idea in most cases anyway.

Re: Unicorn: C++ Unicode string library

#20

Post by dpark »

funkaster wrote: post #14
captaincrowbar wrote: post #13

Earlier quoted context omitted.

I just tried that on several browsers; Safari and Chrome are fine, it seems to be only Firefox that has a problem with that. I have no idea whether that's a bug in Firefox or Github, and either way there's nothing I can do about it, sorry.

yes, you can: publish your docs as real web pages and not a link to the htmlpreview of a file inside your repo. That should fix the problem.

I guess he should have said that there's nothing reasonable he can do about it. Creating an entirely separate set of HTML pages would require a new publishing flow, add a new step every time docs update, and generally encourage the docs to fall out of sync with the repo. He could do all of this, or he could do the sensible thing and leave the docs exactly like they are.
Post reply on HN