511–520 of 524 posts
Earlier quoted context omitted.
And are you going to enlighten us? What is life like working in a Chinese mine?
Not good. You may look for news online. If you can’t find anything, you might reflect why this happens.
Earlier quoted context omitted.
Not good. You may look for news online. If you can’t find anything, you might reflect why this happens.
There is not much information about conditions in the the mining industry in most countries, because the situation is unremarkable and comfortable. I put it to you that you have no idea what mining conditions are like in China and the situation is overall good.
Earlier quoted context omitted.
Notoriously terrible? Posits do a little better if your numbers are biased enough toward 1, but not much better. A 16 bit posit in a near-ideal situation matches an 18 bit IEEE float, and in a pretty wide range of situations loses to either fp16 or bf16. Training anything at 8 bits is going to be tough, and it's hard to say if the flexible exponent is worth the precision tradeoffs.
> A 16 bit posit in a near-ideal situation matches an 18 bit IEEE float Unsure what you mean by this... A posit16 has up to 11 bits of precision. There's no such thing as an 18 bit IEEE float. > and in a pretty wide range of situations loses to either fp16 or bf16 Many papers have compared neural networks at 16 bits or 8 bits, and posits beat the hell out of floats and it's not even close. Which is very much expected…
Is this excluding the implied bit?
In that case a short float has 10, but if you're messing with formats you can staple on an extra bit of precision and an extra bit of exponent.
> There's no such thing as an 18 bit IEEE float.
There's a lot of custom sizes out there. But if you keep following IEEE rules then there's no special circuitry needed, just a small scaling factor.
nVidia also laid out a 19 bit format that's a superset of both fp16 and bf16.
> Many papers have compared neural networks at 16 bits or 8 bits, and posits beat the hell out of floats and it's not even close.
Can you link a paper that shows posits beating floats at different sizes?
I found a 2021 paper that compares various posits to 32 bit floats, and finds that the model quality is close for some of them. It does not compare any smaller floats.
> Which is very much expected. As they're particularly suited to this task.
Posits show their value when you need a huge exponent range and your numbers focus very closely around 1. How strongly do neural nets fit that pattern?
And how often is their advantage better than 1 or 2 bits?
If you can keep your weights within a range of 9 orders of magnitude, I expect fp16 to do just fine since it loses a bit on some numbers and gains a bit or two on other numbers.
> But also in other domains, like numerical weather simulations, where tests have shown 16-bit posits can replace 32-bit floats.
Can you link this too? I found a 2019 paper that shows them beating fp16 and falling short of fp64, but no fp32 comparison. They also noted that 16,0 posits and bf16 did badly.
They did conclude that 16 bit posits were probably good enough to beat out measurement error and be suitable for the bulk of simulation, but that same chart showed that fp16 was almost good enough. So again I wonder how many bits you'd actually need, since if you're considering rebuilding your FPUs it would be silly to exclude "float sizes that aren't powers of two".
Earlier quoted context omitted.
https://ourworldindata.org/grapher/co-emissions-per-capita USA 14.2 Canada 13.42 China 8.66
Per capita is not a useful metric in this measurement. Why is that such a theme?
Earlier quoted context omitted.
Looking at the history of the memory industry the biggest risk is that a firm would over produce and go bankrupt. Maybe this time is different but so far no memory chip maker has gone under because their competition increased capacity.
I might be wrong but your second point can't be true if the first one is true. Let me explain, imagine CXML grows massive and builds a lot of fabs, so much so that it becomes the leader in multiple segments, then the market demand cools off. Then CXML the company that invested massively has oversupply so it undercuts every other memory company. Aka, Samsung, SK Hynix are dead, and to protect Micron now US has 10000%…
Earlier quoted context omitted.
Intel struggled because they bet the company that Moore's law was over back in ~2014, and instead of upgrading their fabs to EUV they sent the money back to shareholders. They forgot Moore's main lesson: only the paranoid survive. They thought they could coast, and it nearly killed them.
> They forgot Moore's main lesson: only the paranoid survive. "Only the Paranoid Survive" is rather a quote and book title by Andrew S. Grove.
Earlier quoted context omitted.
Intel struggled because they bet the company that Moore's law was over back in ~2014, and instead of upgrading their fabs to EUV they sent the money back to shareholders. They forgot Moore's main lesson: only the paranoid survive. They thought they could coast, and it nearly killed them.
That is not even close to correct.
Earlier quoted context omitted.
I have fairly simplistic view of the economics involved here. Could you explain why the ability to sell more chips wouldn't be sufficient enough incentive to increase supply?
Let's imagine you're drilling oil instead. You have to spend billions of dollars over years finding and developing a new oilfield to make any profit back. And once you have it, you have to continuously spend enormous amounts of money to keep producing it, which means your effective price floor is higher than the current stable price. Now it's 2021 and someone gets a tanker stuck in the Suez, sending the price of oil…
The price has doubled but it takes 10-15 years on average just for the permitting of a new mine in the US.
Offshore drilling for oil is a 7-8 year project.
People in software I think can't but help to have a Laissez-faire bias because software is so Laissez-faire. Most other businesses are highly regulated.