Live data from Hacker News

About the Rosetta Translation Environment

developer.apple.com

161–170 of 249 posts

Re: About the Rosetta Translation Environment

#161

I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusi…

Tim Cook says new Intel Macs are coming out this year. If you can convince your company to buy you one, you should be good until 2030.

Not being able to do your job would be enough for even my craptastic company to buy me a new machine.

Re: About the Rosetta Translation Environment

#162
post #106

Earlier quoted context omitted.

That's the big piece I've been wondering about too. Three options as I see it (none of them great): 1) Pin all threads in an x86 process to a single core. You don't have memory model concerns on a single core. 2) Don't do anything? Just rely on apps to use the system provided mutex libraries, and they just break if they try to roll their own concurrency? Seems like exactly the applications you care about (games, pro…

> 3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place. I would assume that a more strict memory model would be enabled only for processes that needs it (ie, Rosetta translated ones). So a cpu-flag is set/cleared when entering/exiting user mode for those processes. Does this require a separate/special cache coherency protocol? A comp…

It can be enabled per instruction.

Atomic instructions (and ARMv8.1 added a bunch of new atomic read-modify-write instructions that line up nicely with x86) use the new cache coherency protocol, while the older non-atomic instructions keep the relaxed memory model.

Though, I'm not sure if it's worth it to keep two concurrency protocols around. I wouldn't be surprised if the non-atomic instructions get an undocumented improvement to their memory model.

Re: About the Rosetta Translation Environment

#163

I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusi…

My understanding is that that line only refers to VM applications written and compiled for x86, and doesn't stop a VirtualBox (or any other VM provider) from compiling an ARM binary that doesn't require Rosetta. I don't think this is a policy restriction so much as there not being support for Intel virtualisation technologies that that might rely on.

Think this is accurate. Any modern x86 virtualization software depends on hypervisor assistance from Intel/AMD to run at reasonable speeds, and that's just virtualizing x86 on x86--more like a context shift between sandboxes. Full virtualization through a CPU emulator to boot isn't very likely to be practical anytime soon.

I suspect they'll treat it as a policy restriction, though, so they wouldn't have to deal with emulator compatibility going forward. Any solution that would be even minimally able to support virtualizing x86 would probably be ugly to keep running. If VMW or Parallels gets something going, my guess is they'll have to work through Apple to get an exception.

Re: About the Rosetta Translation Environment

#164
post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

I'm not sure why they're making a big deal about this, couldn't the original Rosetta do this too? QEMU has been doing this since (I think) even before the original Rosetta, they call it user mode emulation. You run as if it was a normal emulator but also trap syscalls and forward them to the native kernel instead of emulating a kernel too. I'm more interested in how they're doing the AOT conversion and (presumably) p…

I'm not sure why they're making a big deal about this

Because they're trying to get the message across to regular people (not HN types) that their software will continue working with the new chips, and they don't have to flee the Apple ecosystem.

Re: About the Rosetta Translation Environment

#165
post #155

What's old is new again. This reminds me of Digital's VEST technology. VEST would convert VAX programs to run on Alpha. From 32-bit CISC to 64-bit RISC. Nearly 30 years ago. https://web.stanford.edu/class/cs343/resources/binary-transl...

Or the PET emulator from 38 years ago that Commodore released because it wasn't sure that the C-64 would have any software.

Ditto for the C-64 CP/M cartridge of the same vintage.

Re: About the Rosetta Translation Environment

#166
post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

I'm not sure why they're making a big deal about this, couldn't the original Rosetta do this too? QEMU has been doing this since (I think) even before the original Rosetta, they call it user mode emulation. You run as if it was a normal emulator but also trap syscalls and forward them to the native kernel instead of emulating a kernel too. I'm more interested in how they're doing the AOT conversion and (presumably) p…

The original Rosetta couldn’t do it, or couldn’t do some part of it. I remember because Sixtyforce couldn’t run. I believe that it could have to do with self-modifying code.

Re: About the Rosetta Translation Environment

#168

I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusi…

I have a very similar situation: I loved using MacBook Pros as my work computer for over 10 years now, but while my job consists of 80% of cross-platform Java development stuff, it also includes working on and compiling various custom binaries (from big fat Chromium builds down to little system-level libraries and Linux kernel modules). All three major OSes are targets: MacOS, but most importantly Windows and Linux - especially Linux, because that's the main production environment on which our end products usually run, while the others are mostly needed to enable our devs to develop on their preferred platforms.

Windows and Linux MUST be x86_64, because that stuff needs to run on that arch eventually - it's of no use if it "works on my (ARM) machine". The Mac platform has, until now, been the ideal platform for my work, because it comes with by far the best OS on which I can be as productive as possible, while at the same time being able to execute all three targets within performant VM environments, greatly simplifying all the low-level development tasks as well as testing and experimentation. Even USB hardware could transparently be routed into the VMs - something that's also very important for me, as I often need to deal with weird and rather unusual periphery, and which is highly problematic when working on remotely hosted VMs, which I did try for a while back then when MacBooks had that RAM cap at 16GB in order to be able to utilize more memory.

I'm still good with my current MacBook Pro for about two years or so, which gets me well into the transition period. I know that I'll definitely be tempted to privately buy a replacement for an old 2011 MacBook Air, which I still use for light browsing tasks and stuff at home and which will be totally fine if it's ARM-based. If anyone happens to come around with some kind of VM emulation solution for actually virtualizing x86_64 OSes on the ARM-based Macs - there at least seems to be a demand, so I haven't lost hope that one of the likes of Parallels or VMware might take up that task - I'm going to give that a try and see whether it's a solution to my remaining 20%.

If that doesn't work out, I'll have to consider either switching to ARM Mac regardless and keeping the old machine (or another PC laptop or something) around as a second machine for the low-level Windows/Linux work, or ditching the Mac platform and moving to a Linux-based machine as my main computer, with Windows in a VM, and maybe getting a small ARM Mac as a "sidekick" for that bit of MacOS compilation and testing work. Either of those will be worse than my current, nearly-perfect setup, which was originally enabled only by the beauty of ISA compatibility between the Mac and PC platforms.

Re: About the Rosetta Translation Environment

#169
post #65
post #37

Having big flashbacks to the switch from PPC to x86 here. Rosetta worked relatively smoothly during that transition so fingers crossed it will be ok here too. Though with Docker support on the mac already being a second class citizen to running on Linux I wonder if a lot of devs will stop using macs for dev

Highly unlikely Apple would cede the software engineering market for a competitor to step in. Almost everyone in my company would prefer to have a MacBook vs a Thinkpad/Dell/etc if given the choice.

Speaking solely for myself, I do not intend to go with this experiment after using Macs for 16 years and change. I have found that with a few rough edges, Windows 10 and WSL fulfills my dev needs, and except for the Apple apps Logic and Final Cut Pro, virtually all my software is cross platform. Besides, I can use Ableton and Adobe Premiere which I also already "own."

Re: About the Rosetta Translation Environment

#170
post #160

Earlier quoted context omitted.

Expect you'd need to do that with every single store since the x86 instruction stream doesn't have those semantics embedded in it. That'd kill your memory perf by at least an order of magnitude, and kill perf for other cores as well. It'd be cheaper to just say "you only get one core in x86 mode". Essentially you'd be marking every store as an L1 and store buffer flush and only operating out of L2.

ARMv8.1 adds a bunch of improved atomic instructions that basically implement the same functionality as x86. Because x86 has atomic read-modify-write instructions by default; You need emulate those too. The ARMv8.1 extensions look like they have been explicitly designed to allow emulation of x86. The implication is that implementations of ARMv8.1 can (and perhaps should) implement cache/coherency subsystems with high…

So the ARMV8.1 extensions aren't for emulating x86, they're a reflection of how concurrency hardware has changed over the years.

It used to be that (and you can see this in the RISC ISAs from the 80s/90s, but AIUI this is what happend in x86 microcode as well)

* the CPU would read a line and lock it for modifications in cache.

* the CPU core would do the modification

* the CPU would write the new value down to L2 with an unlock, and L2 would now allow this cache line to be accessed

So you're looking at ~15 cycles of contention from L2 through the CPU core and back down to L2 of a lock for that line.

If this all looks really close to a subset of hardware transnational memory when you realize that the store can fail and the CPU has to do it again if some resource limit exceeded, you're not alone.

Then somebody figured out that you can just stick an ALU directly in L2, and send the ALU ops down to it in the memory requests instead of locking lines. That reduces the contention time to around a couple cycles. These ALU ops can also be easily included in the coherency protocol, allowing remote NUMA RMW atomics without thrashing cache lines like you'd normally need to.

This is why you see both in the RISC-V A extension as well. The underlying hardware has implementations in both models. I've heard rumors that earlier ARM tried to do macro op fusion to build atomic RMWs out of common sequences, but there were enough versions of that in the wild that it didn't give the benefits they were expecting.

However, all that being said, atomics are orthogonal to what I'm talking about. It's x86's TSO model, and how it _lacks_ barriers in places that ARM requires them, with no context just from the instruction stream about where they're necessary that's the problem here, not emulating the explicit atomic sequences.

Post reply on HN