Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About the Rosetta Translation Environment

developer.apple.com

Post reply on HN
Print view View original

181–190 of 249 posts

Re: About the Rosetta Translation Environment

#181

Post by morpheuskafka »

qubex wrote: post #84
Bigpet wrote: post #44

Earlier quoted context omitted.

I think I'm missing something. How is ARM closing things up? Does Apple have a bunch of processor extensions that make it incompatible with other ARM processors? Or do you think they'll take the chance to kill boot-camp or something?

The “ARM world” is almost inconceivably varied by x 86 standards. There’s not even an agreed-upon boot procedure (the Raspberrry Pi’s CPU famously gets booted by the GPU). Efforts to standardise this in the server space are only just getting off the ground with the Arm Server Base Boot Requirements version 1.2 being ratified last year. I doubt you’ll find Apple subscribing to anything that they won’t develop in-house…

The CPU booted by GPU is actually very similar to the way that the T2 boots the Intel CPU in existing Macs, providing signature assurance on the microcode, ME firmware, and boot image before the CPU even loads.

Re: About the Rosetta Translation Environment

#182

Post by ogre_codes »

comex wrote: post #175
Mister_Snuggles wrote: post #143

I've been using Macs at work for a long time, but sadly this will mark the end of that era. In particular, this limitation on Rosetta rules out an ARM-based Mac for work: > Virtual Machine apps that virtualize x86_64 computer platforms My job requires me to use a piece of proprietary Windows-only software for a large portion of my work. If I can't use this software I can't do my job. Currently I run it in VMware Fusi…

> There is a very remote possibility that Windows for ARM could be virtualized by some future version of VMware and the proprietary toolset could run under that, but I'm not holding my breath. How is that remote? We've already seen Parallels Desktop virtualize Linux for ARM.

As far as I can tell, you can't buy Windows for ARM, you can only get a license by buying specific hardware.

Aside from that, most people who want Windows support on a Mac want support for x86 Windows, not the ARM version. So even if it does run, it's likely to be a hobbyist/ hacker thing, not a product.

Re: About the Rosetta Translation Environment

#183

Post by LeoNatan25 »

burlesona wrote: post #113
rezmason wrote: post #73

Earlier quoted context omitted.

I think the longer Rosetta exists and macOS thereby supports a wider variety of binary executables, the more people will rely on it to deliver them day-to-day functionality in increasingly complex configurations, and the more Apple will expose itself to those users' criticisms. If Apple's rationale for the transition is to gain further control over their product design and manufacture, the prospect of having to appea…

Your first two paragraphs track, but I think the second two are quite a leap. Sure it’s possible, but I think it’s exceptionally unlikely that Apple will try to lock down the Mac the way the iPhone is. The reason is they think of them as fundamentally different products with fundamentally different purposes, and for the most part the trend line has been moving to more opening of the iOS ecosystem rather than more loc…

It’s already happening. macOS Bug Sur (Apple’s wording) enforces a read only system partition, which cannot be easily disabled or mounted as rw.

Re: About the Rosetta Translation Environment

#184

Post by monocasa »

anyfoo wrote: post #157
monocasa wrote: post #57

Earlier quoted context omitted.

Since they don't enforce W^X you need to go a little deeper than that since some JITs reuse the same memory later for new traces. So what you'll do is silently enforce W^X to trap code modifications, but not propagate those traps to the emulated code.

Pages containing x86 code are never actually being marked executable in the (ARM) pagetables, because that would be nonsensical: The CPU itself does not know how to run x86 code. mmap/mprotect from x86 with the exec bit therefore does something else than native code.

Right, but the x86 code _thinks_ it does, and those semantics have to be maintained.

What that practically means is that x86 executable pages get mapped as RO by default as seen by the ARM side even if they would normally be RWX. Modifications trap, the handler mprotects the pages as writeable, flush the translation cache for that region, then returns from the trap. Then on an x86 jump to that region, the JIT cache sees no traces for that region, marks the page as RO again so that modifications will trap, and starts recompiling (or maybe just interpreting; they may be using a tiered approach in the JIT).

Re: About the Rosetta Translation Environment

#185

Post by AgentOrange1234 »

ryanianian wrote: post #94
rz2k wrote: post #71

Does anyone have any insight into the business logistics involved in a transition like this? I presume Apple has done maintenance on transitioning desktop OS X to ARM as an option for a very long time. How many years ago would they have had to decide that was the direction they were going to make it reality? 2015? How many people would have been working on it? How many billions of dollars? How does the cost of develo…

> How many billions of dollars? Rounding-error-sized marginal costs. Even if you don't plan on pivoting architectures, it's good practice to write as much code as possible without assembly/intrinsics. If you're writing standards-compliant C/C++/Objective-C/Swift/* it's not a huge burden to change architectures. They also have past-experience with such transitions, so hopefully they kept the "keep the code portable" m…

Probably a big opportunity cost though. It surely takes a lot of time from senior engineers to port the software, figure out this whole translation system, etc...

Re: About the Rosetta Translation Environment

#186

Post by jojobas »

hedora wrote: post #179
jojobas wrote: post #74

Earlier quoted context omitted.

My gut feeling is that it will be about the same as Itanium/HP Envy x2 emulation. Emulation of highly optimized hardware where code is generated by highly optimized compilers without an order of magnitude slowdown is just too good to be true.

Valgrind does something similar (x86->intermediate language->x86), and is only about 4x slower than native with all the analyses disabled. I’d guess they left some optimizations out to make it easier to implement the dynamic checks it supports.

Valgrind leaves most instructions as they were doesn't it? If you're not touching the dynamic memory it should be as fast. You wouldn't be able to do that with complex MMX or SSE2 instructions with Arm translation.

Re: About the Rosetta Translation Environment

#187

Post by pgwhalen »

api wrote: post #85
lsllc wrote: post #53

Earlier quoted context omitted.

It probably just hooks mmap() (or mprotect()) and looks for when the JIT sets PROT_EXEC on memory it hasn't translated yet. Here's a tutorial for a simple JIT that uses mmap() with PROT_EXEC to write machine code to before executing: https://github.com/spencertipping/jit-tutorial

"Just" This is some serious black magic, especially if it actually works. Even if it only works some of the time I give it an A for effort, since they could easily have just said "JITs will not work with Rosetta." Most applications do not contain JITs. My guess is there are enough important Mac apps that contain JITs of some form to merit this work, probably high performance math and graphics kernels that JIT process…

>This is some serious black magic

If there were ever a time and a place for black magic, it would be this use case: swapping out an enormous layer (the hardware) at the bottom of the stack, while keeping everything at the top of the stack working seamlessly.

Re: About the Rosetta Translation Environment

#188

Post by AndrewStephens »

andoma wrote: post #87

I wonder how it handles the stricter memory ordering from x86? For example, this code: void store_value_and_unlock(int *p, int *l, int v) { *p = v; __sync_lock_release(l, 0); } on x86_64 compiles to: mov dword ptr [rdi], edx mov dword ptr [rsi], 0 ret vs on arm64: str w2, [x0] stlr wzr, [x1] ret Notice how the second store on ARM is a store with release semantics to ensure correct memory ordering as it was intended i…

Elsewhere in the documentation[0] Apple explicitly calls out that code that relies on x86 memory ordering will need to be modified to contain explicit barriers. All sensible code will do this already.

[0] https://developer.apple.com/documentation/apple_silicon/addr...

Re: About the Rosetta Translation Environment

#189

Post by saagarjha »

derefr wrote: post #118
mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

Think of binary translation as just another kind of compiler. It parses machine code, generates an IR, does things to the IR, and then codegens machine code in a different ISA. (Heck, Rosetta 2 is probably built on LLVM. Why wouldn't it be? Apple already put so much work into it. They could even lean on similar work like https://github.com/avast/retdec .) During the "do things to IR" phase of compilation, you can do…

> Rosetta 2 is probably built on LLVM. Why wouldn't it be?

LLVM is kind of slow; JavaScriptCore abandoned it years ago for their FTL backend.

> The unique pattern of machine-code that must occur in any implementation of JIT, is a jump to a memory address that was computed entirely at runtime, i.e. with a https://en.wikipedia.org/wiki/Use-define_chain for that address value that leads back to a call to mmap(2) or malloc(2).

  void *region = mmap(0x100000000, 0x1000, PROT_READ | PROT_WRITE, MAP_FIXED, 0, 0);
  memcpy(region, code, 0x1000);
  mprotect(0x100000000, 0x1000, PROT_READ | PROT_EXEC);
  reinterpret_cast(region)();

Re: About the Rosetta Translation Environment

#190

Post by perardi »

monocasa wrote: post #122
perardi wrote: post #101

Earlier quoted context omitted.

Well color me surprised. I figured that’d be on the chopping block.

I was surprised too. I wonder which crazy peripheral (or maybe it was antivirus/intrusion detection?) made them not go all in on "only apple gets to be in kernel space".

Judging by this horrible Bitdefender Endpoint Security that IT has foisted upon me: device management and anti-malware stuff.
Post reply on HN