Is pasting em-dashes everywhere some kind of inside joke?
I pity the fools who don't have compose keys for all their em—dash and “smart quote” needs.
Zig's new bitCast semantics and LLVM back end improvements
91–100 of 159 posts
Re: Zig's new bitCast semantics and LLVM back end improvements
#92Earlier quoted context omitted.
> Just don't allow casting to u24, as it makes no sense unless you define u24 to be u32 sized as I think c standard does. The reason u32->u24 casting must be well defined is because some hardware (e.g. many GPUs, microcontrollers) only have floating point multipliers. A 24 bit unsigned integer (stored in a 32 bit register) can be losslessly converted to a 32 bit float by the hardware, multiplied, then converted back.…
> many GPUs Citation please - every single GPU in the literal world supports integer arithmetic for operating on tid, gid, etc.
From page 175 of the AMD CDNA4 ISA:
https://www.amd.com/content/dam/amd/en/documents/instinct-te...
> V_MUL_U32_U24
>,Multiply two unsigned 24-bit integer inputs and store the result as an unsigned 32-bit integer into a vector register. D0.u32 = 32'U(S0.u24) * 32'U(S1.u24)
> Notes
> This opcode is expected to be as efficient as basic single-precision opcodes since it utilizes the single-precision floating point multiplier. See also V_MUL_HI_U32_U24.
Nvidia GPUs used to do the same thing and theres a umul24 intrinsic if you care to use it.
https://stackoverflow.com/questions/5544355/cuda-umul24-func...
This is super-super-niche since it basically only applies to 32-bit integer multiplication.
You likely won't run into it unless you're doing high performance embedded systems or GPU programming on non-NVDIA cards, and for some unknowable reason, your workload does a 32-bit integer multiplication in the hot path.
Re: Zig's new bitCast semantics and LLVM back end improvements
#93Earlier quoted context omitted.
zig does not allow arrays in packed structs/unions specifically for endianness reasons (there may be other reasons as well but endianness is what i know of)
Ah, that is useful to know. Is that documented somewhere? From what I can quickly find in the obvious place [0], the only requirement is that "all fields in a packed union must have the same @bitSizeOf" and [2]u8 does satisfy that requirement. [0] https://ziglang.org/documentation/0.16.0/#packed-union
no, but the documentation for packed structs gives the list of allowed field types. it's also not documented that packed union fields must be valid packed struct fields but people may be able to assume that
edit: also, this is a relevant issue: https://github.com/ziglang/zig/issues/12547
Re: Zig's new bitCast semantics and LLVM back end improvements
#94> Quite long devlog coming up, apologies—I got a little carried away with this one! mlugg, please don't apologize for creating something I actually want to read. I'm drowning in low effort garbage, the in depth technical explanation is a refreshing breath of fresh air. Might as well apologize for creating a language without a garbage collector, sure most people are unwilling to think, but some of us like nice things…
Why I've moved more to a couple of language/software dev discords and away from Hacker News. Way too much uninteresting AI nonsense on here for a while now.
Re: Zig's new bitCast semantics and LLVM back end improvements
#95Earlier quoted context omitted.
> many GPUs Citation please - every single GPU in the literal world supports integer arithmetic for operating on tid, gid, etc.
From page 175 of the AMD CDNA4 ISA: https://www.amd.com/content/dam/amd/en/documents/instinct-te... > V_MUL_U32_U24 >,Multiply two unsigned 24-bit integer inputs and store the result as an unsigned 32-bit integer into a vector register. D0.u32 = 32'U(S0.u24) * 32'U(S1.u24) > Notes > This opcode is expected to be as efficient as basic single-precision opcodes since it utilizes the single-precision floating point multi…
That's literally only for 32bx24b (I don't remember why we did that specifically for CDNA - I'll ask someone) but as you see from V_MUL_HI_I32, V_MUL_LO_U32 there is very much vector arithmetic hardware (nevermind that we're not talking about VALU but conventional scalar ALU).
Re: Zig's new bitCast semantics and LLVM back end improvements
#96Earlier quoted context omitted.
The ease of dealing with arbitrary bit-width integers and packed structs is actually one of the 'killer features' for me in zig. Zig natively supports arbitrary bit-width integers, the ABI is defined and you could simply think it as a slice of the next larger backing integer. The[3]u8 to u24 bitCast will simply be backed by a 32bit int, using the same ABI. As you have u1 - u65535, sometimes it can be multiple words.…
> The 24 Bits (3 Bytes) [3]u8 to u24 example is exactly related to utf-8 that covers all the languages but excludes the emojis. I'm not familiar with Zig, so maybe it's doing something weird here, but that doesn't really make sense with Unicode in general. First, the largest Unicode codepoint that will ever be allocated is U+10FFFF [0], which is less than 2^21, so all Unicode characters will fit in a 24-bit integer.…
Note the utf-8[0] in my response, the answers are on the pages you linked, but not in the sections you linked,
utf-8 encodes code points in one to four bytes, it is byte oriented vs utf-16 etc. In zig u8 is a byte, and is also (by convention) a char, although there isn't an explicit char type in zig. Technically there are chars in languages that need all 4 bytes in utf-8, but almost all of them are historical or emoji's in utf-8.
24bits (3 bytes) in utf-8 gets you Chinese, Japanese, Korean. 16 bits (2 bytes) gets you Latin letters with diacritics, Greek, and Arabic scripts. With 8 bits (1 byte) getting you Standard ASCII etc...
There is a point you could make that it may have been better to use utf-16 etc... and that we should have dropped ascii/latin-1 support, but once again go up to the 'Basic Multilingual Plane' in your [3] and notice that is covered by 24bits (3 bytes) in utf-8 encoding.
Re: Zig's new bitCast semantics and LLVM back end improvements
#97Earlier quoted context omitted.
I am criticizing the part where they allowed [3]u8 to u24 bitCast in the first place. It doesn't make sense logically as u24 is likely not 24 bits in any targets let alone portably on every target. Interpreting u24 like it is actually 24 bits sounds like programming in crazy land since it is not 24 bits in any relevant architecture afaik. They didn't allow []u24 with a similar rationale as far as I can remember. I ag…
The ease of dealing with arbitrary bit-width integers and packed structs is actually one of the 'killer features' for me in zig. Zig natively supports arbitrary bit-width integers, the ABI is defined and you could simply think it as a slice of the next larger backing integer. The[3]u8 to u24 bitCast will simply be backed by a 32bit int, using the same ABI. As you have u1 - u65535, sometimes it can be multiple words.…
> ... utf-8 that covers all the languages but excludes the emojis ...
Ah, but the U+0000 to U+FFFF plane does not cover all the languages. You might think that only historical and archaic languages are found in Unicode's astral planes (e.g., U+20000 to U+2A6DF is used for historical Chinese characters no longer used today), but in fact there are modern languages found in the U+10000 plane.
You might not care about Osage (the language of the Osage Nation of northern Oklahoma) since its last native speaker passed away in 2005, but there is a revival program trying to teach Osage to people. Osage's script was developed quite recently as part of the revival program, so it couldn't fit into the U+0000 to U+FFFF block and it was assigned U+104B0 to U+104FF.
The Toto language of Bengal, on the other hand, is still active: over 1000 speakers, all living in the village of Totopara. It also never had an alphabet until recently, so its Unicode block is U+1E290 to U+1E2BF.
Then there's Wancho, spoken by about 60,000 people in India. Its alphabet was created between 2001 and 2012, and added to Unicode in 2019. It was assigned the U+1E2C0 to U+1E2FF block (immmediately after the Toto language, you might notice).
Then there's the Ho language spoken by over a million people in India. Wikipedia cites a 2001 census as having 2.2 million speakers, and a 2011 census as having 1.4 million speakers. I very much doubt that both of those are accurate (you don't lose half a million people from an ethnic group in just ten years without some kind of war or genocide, and the Wikipedia article would have at least mentioned that if such a thing had happened), but to be safe, let's go with the lower estimate and say that at least one and a half million people speak Ho. It can be written with the Latin alphabet, but its own alphabet is Warang Chiti (sometimes spelled Warang Citi), which was added to Unicode in 2014 and assigned the U+118A0 to U+118FF block.
And then there's the Adlam script for writing Fulani, the language of the Fufulde people of western Africa. Fulani is spoken natively by 37 million people, and as a second language by another 2.7 million. Adlam's Unicode block is U+1E900 to 1+1E95F.
So if you restrict your program to only working with the basic multilingual plane, it's not just emoji you'll be leaving out. It's also modern languages, spoken by anywhere from 1000 people to 37 million. How many speakers of a language are enough to draw the line and say "No, I won't ever translate my software into your language"?
Now, if your software is only targeting one language and you never intend to translate it, then yes, you'll only lose out on emoji if you stick to the U+0000 to U+FFFF range of the basic multilingual plane.
But realize that the higher planes are not just for dead languages. Living languages have ended up there too, and there are likely to be more in the future. It's quite possible that right now, someone somewhere is saying "Hey, why doesn't my language have its own alphabet instead of using Latin characters to write it? The Latin characters don't express the sounds of my language very well." And when they do get that alphabet worked out and manage to get it accepted into Unicode, it'll certainly land in one of the higher planes. Most likely the U+10000 to U+1FFFF plane which isn't at all full yet, but who knows. If you want to be able to handle every language spoken (and written) in the world today, you must be able to accept the full range of Unicode, not just the 16-bit range.
Re: Zig's new bitCast semantics and LLVM back end improvements
#98Earlier quoted context omitted.
> The 24 Bits (3 Bytes) [3]u8 to u24 example is exactly related to utf-8 that covers all the languages but excludes the emojis. I'm not familiar with Zig, so maybe it's doing something weird here, but that doesn't really make sense with Unicode in general. First, the largest Unicode codepoint that will ever be allocated is U+10FFFF [0], which is less than 2^21, so all Unicode characters will fit in a 24-bit integer.…
Note the utf-8[0] in my response, the answers are on the pages you linked, but not in the sections you linked, utf-8 encodes code points in one to four bytes , it is byte oriented vs utf-16 etc. In zig u8 is a byte, and is also (by convention) a char, although there isn't an explicit char type in zig. Technically there are chars in languages that need all 4 bytes in utf-8, but almost all of them are historical or emo…
> ... but almost all of them are historical or emoji's in utf-8.
I just posted a comment, five minutes after you wrote that, which I won't repeat here since it was quite long. But one of the languages whose alphabet is found in the higher multilingual plane is Fulani, spoken natively by 37 million people (plus another two and a half million who have learned it as a second language). While it can be written in other alphabets (both Latin and Arabic have been used to write it in the past, for example), other alphabets don't usually represent all the sounds of the language properly, making it awkward. There's a reason why the Adlam script was invented to write Fulani with; and that invention was recent enough that it was assigned the U+1E900 to U+1E95F block, since the basic multilingual plane was full by then.
So although it's easy to think that the astral planes are only used for emoji and historical languages, that's not actually true. There are languages spoken by millions of people in those astral planes as well (yes, languages plural; Fulani isn't the only one, it's just the largest).
Re: Zig's new bitCast semantics and LLVM back end improvements
#99> Quite long devlog coming up, apologies—I got a little carried away with this one! mlugg, please don't apologize for creating something I actually want to read. I'm drowning in low effort garbage, the in depth technical explanation is a refreshing breath of fresh air. Might as well apologize for creating a language without a garbage collector, sure most people are unwilling to think, but some of us like nice things…
Why I've moved more to a couple of language/software dev discords and away from Hacker News. Way too much uninteresting AI nonsense on here for a while now.
Would like to join as well if you're willing to share
Re: Zig's new bitCast semantics and LLVM back end improvements
#100Earlier quoted context omitted.
Does that mean there are no file formats thatbuse big endian? And network byte order isn't a thing?
> there are no file formats thatbuse big endian if someone chooses to do that they own the problems. > network byte order isn't a thing if the network serializes/deserializes for you (kernel primitives) then you don't care what it does. if it doesn't and for some reason you choose to use big endian, again, you own the problem.
If someone chooses to load a TIFF or a PSD or an AIFF or…