Earlier quoted context omitted.
Because futher down the stack, more reliable a tech has to be. Otherwise good luck debugging. Also agree with heisenbit.
Hmmm... on that note, if the universe is a simulation, then a bug in that could have some interesting ramifications. "Don't use the bookshelf over there, physics is broken on that shiny spot." :D
Intel Skylake/Kaby Lake processors: broken hyper-threading
121–130 of 278 posts
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#122Earlier quoted context omitted.
CPU manufacturers do do huge amounts of testing, and Intel does formal verification of some functional units. The reliability is far better than most software, in part because making a new release costs billions.
That's absolutely true. When it comes to CPU/memory, skilled software engineers always think, "it must be my bug, it always is". So in that super rare case of actually running into a CPU defect, it's a mindfuck, it'll drive you crazy. You'll be looking for the flaw in your algorithm which makes it fail once a week under production load. But you just can't find it, it makes no sense ... (When it comes to drivers for n…
The symptom was that a board with a specific microcontroller on it would be working fine, then after a power cycle it might not keep working. A flash dump would show that the reset vector, the first byte of flash on that system, would be all zeroed out. Of course the system would not run anymore, but why did it happen? After months of intermittent debugging and trying to reproduce the cause was determined. At least under certain conditions the brownout detection level was lower than the voltage level that caused the CPU to make errors. If the board lost power slowly then the CPU would start executing corrupted / arbitrary instructions which generally included lots of zeros. It would occasionally write zeros to the zero address, bricking the board.
Since then we have external power monitoring and reset circuits on all the new boards, but existing ones needed a fix. Luckily the board had power failure interrupt connected, so when that triggers we reconfigure the CPU to execute on the slowest possible clock rate, which greatly reduced the occurrence.
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#123The problem description is short and scary: Problem: Under complex micro-architectural conditions, short loops of less than 64 instructions that use AH, BH, CH or DH registers as well as their corresponding wider register (e.g. RAX, EAX or AX for AH) may cause unpredictable system behavior. This can only happen when both logical processors on the same physical processor are active. I wonder how many users have experi…
> short loops of less than 64 instructions that use AH, BH, CH or DH registers as well as their corresponding wider register (e.g. RAX, EAX or AX for AH) This is yet another of the many places where the complexity of the x86 ISA shows up and makes its hardware implementations more complicated: the x86 ISA has instructions which can modify the second -lowest byte of a register, while keeping the rest of the rest of th…
---
For Skylake, they probably optimized partial register
writes to these four partial "high" registers (AH, BH,
CH, DH), but the optimization was buggy in some hard-to-
hit corner case.
They did not do this.The high registers (AH/BH/DH/CH) are nearly written out of existence with the REX Prefix in 64bit mode. Within the manual(s) it is called out effectively not to use them as they're now emulated and not support directly in hardware.
The 16bit registers (AX/BX/DX/CX) are in worse situation, but it ends up costs additional cycles to even decode these instructions as the main encoder can't handle these instructions and you have to swap to the legacy encoder, and you'll end up losing alignment. This costs ~4-6 cycles, also the perf registers to track were only added in Haswell (and require Ring0 to use [2]).
High Register and 16bit registers are huge wart that it seems Intel is trying desperately hard to get us to stop using.
That corner case probably can only be reached when some
part of the out-of-order pipeline is completely full,
which is why it needs a short loop (so the decoder is not
the bottleneck, AFAIK there's a small u-op cache after the decoder)
There is a 64uOP cache between the decoder and L1i cache that is called loop stream detector. Normally this exists to do batched writes to the L1i cache.But in _some_ scenarios when a loop can fit completely within this cache it'll be given extremely priority. This is a way to max out the 5uOP per cycle Intel gives you [1]. It'll flush its register file to L1 cache piece meal as it continues to predict further and further and further ahead speculatively executing EVERYPART OF IT in parallel. [3]
In short this scenario is extremely rare. uOPs have stupidly weird alignment rules. Which you can boil down to:
Intel x64 Processor are effectively 16byte VLIW RISC processors
that can pretend to be 1-15byte AMD64 CISC processors at a minor performance
cost.
---The real issue here is when Loop Stream mode ends it is properly reloading the register file, and OoO state.
This is likely just a small micro-code fix. The 8low/8high/16bit/32bit/64bit weirdness is likely somebody wasn't doing alignment checks when flushing the register file.
---
[1] On Skylake/KabyLake. IvyBridge, SandyBridge, Haswell, and Boardwell limited this to 4.
[2] Volume 3 performance counting registers I think we're up to 12 now on Boardwell.
[3] Volume 3 Chapter 3.4.1.7 (Page 107)
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#124The problem description is short and scary: Problem: Under complex micro-architectural conditions, short loops of less than 64 instructions that use AH, BH, CH or DH registers as well as their corresponding wider register (e.g. RAX, EAX or AX for AH) may cause unpredictable system behavior. This can only happen when both logical processors on the same physical processor are active. I wonder how many users have experi…
> short loops of less than 64 instructions that use AH, BH, CH or DH registers as well as their corresponding wider register (e.g. RAX, EAX or AX for AH) This is yet another of the many places where the complexity of the x86 ISA shows up and makes its hardware implementations more complicated: the x86 ISA has instructions which can modify the second -lowest byte of a register, while keeping the rest of the rest of th…
My guess is that is where the bug is; the behavior for partial register access stalls---insert one extraneous uop to combine, e.g., ah with rax---is unchanged since Sandy Bridge.
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#125Earlier quoted context omitted.
That's absolutely true. When it comes to CPU/memory, skilled software engineers always think, "it must be my bug, it always is". So in that super rare case of actually running into a CPU defect, it's a mindfuck, it'll drive you crazy. You'll be looking for the flaw in your algorithm which makes it fail once a week under production load. But you just can't find it, it makes no sense ... (When it comes to drivers for n…
You have some lovely stories online about these things. Like the guy tracking down a stuck bit in RAM. Or on the network level, a VPN that failed only when traversing one possible route between company offices.
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#126I installed debian 9, installed virtualbox, vagrant, setup a clean development machine for myself, everything took 4 hours to finish.
I reboot the virtual machine, and boom, there was a kernel panic which I sadly don't remember exactly / didn't take a picture of. After I rebooted the machine, and opened terminal, the system froze. The cursor wouldn't move. Reboot again, motherboard has a CPU fail/undetected light on. Couldn't get it to boot after that.
I am both sad and relieved that bad stuff exists, but it's being patched to prevent proliferating.
I sincerely hope I'll get a replacement from Intel.
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#127Well done Debian folks!
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#128So will this be affecting most Macbook Pros of the past few years? If so, there's a way to disable hyper-threading, but you need Xcode (Instruments). Open Instruments. Go to Preferences. Choose 'CPU'. Uncheck "Hardware Multi-Threading". Rebooting will reset it.
sysctl -n machdep.cpu.brand_string
this will return something like Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz
(Found on http://osxdaily.com/2011/07/15/get-cpu-info-via-command-line... )Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#129Earlier quoted context omitted.
They used Skylake in late 2016 MBPs and Kaby in the 2017 models.
I'm on a Kaby Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz I haven't seen any erratic crashes yet on code compiled by LLVM 8.1.0 on multiple projects (ROS, Qt5). I hope a microcode fix is pushed by Apple on behalf of Intel soon.
Re: Intel Skylake/Kaby Lake processors: broken hyper-threading
#130So what does this mean for the thousands of new MacBook Pro 2016/2017 owners out there?
Nothing, since they've been running their laptops without issues (it would have been all over the news if it was some widespread issue) for 2+ years. At some point in the near future Apple will package the microcode fix in an update, and that will be it.