Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About the Rosetta Translation Environment

developer.apple.com

Post reply on HN
Print view View original

131–140 of 249 posts

Re: About the Rosetta Translation Environment

#131

Post by himinlomax »

mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

> If executable code is being generated at runtime, it's going to be x86_64 binary machine code still

JITs have to explicitly make the memory they write the generated code to executable. The OS "just" needs to fail to actually make the page executable, and then handle the subsequent page fault by transpiling or interpreting the x86 code therein.

Re: About the Rosetta Translation Environment

#133

Post by himinlomax »

leeter wrote: post #20
nickcw wrote: post #15

I thought this was the most interesting paragraph. > What Can't Be Translated? > Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. I guess translation of JIT compiled stuff implies this isn't a once of translation. I guess translating plugins implies that too. It sounds like very clever stuff to me! > However, Rosetta doesn’t translate the following executables: >…

RE AVX and above: My guess is that they are licensing the Transmeta IP which doesn't include those licenses IIRC. So unless they wanted to get sued they'd have to be careful. I suspect some SSE instructions may also not be supported as I don't recall if Transmeta had a license up to SSE4.

Transmeta died what, 20 years ago? All their patents have now expired.

Re: About the Rosetta Translation Environment

#134

Post by amaranth »

mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

I'm not sure why they're making a big deal about this, couldn't the original Rosetta do this too? QEMU has been doing this since (I think) even before the original Rosetta, they call it user mode emulation. You run as if it was a normal emulator but also trap syscalls and forward them to the native kernel instead of emulating a kernel too.

I'm more interested in how they're doing the AOT conversion and (presumably) patching the result to still be able to emulate for JITs. That'd be (comparatively) simple if it was just for things from the iOS and Mac App Stores since Apple has the IR for them still but they made it sound like it was more generic than that.

https://www.qemu.org/docs/master/user/main.html

Re: About the Rosetta Translation Environment

#135

Post by amaranth »

Someone wrote: post #78
DCKing wrote: post #40

Rosetta's goal is to support legacy Mac apps. It's quite finely scoped, but I hope Apple will go above this and make it available as part of their virtualization framework. This way e.g. Parallels and Docker could use this to provide some way to tap in probably the fastest way to run x86 on ARM Macs. This would mean Rosetta would go beyond its stated scope, and certainly go beyond what the relatively short-lived Rose…

AAA games spend lots of times inside system libraries. Porting those to native could be enough to get acceptable performance. So, chess could be more of a challenge for emulation than AAA games (but probably less of an issue, as it would be easily ported, and have fewer users, anyways)

The page we're discussing here actually says you can't mix arm64 and amd64 code in a single process, if your binary is amd64 all your libraries will be too. I'd be surprised if they put in the work to generate thunks for all their system libraries and then didn't make it possible to use any others. More likely macOS will just have two copies of all the (non-iOS) libraries and the only point they work together is the kernel since that's a nice clear barrier.

Re: About the Rosetta Translation Environment

#137

Post by my123 »

monocasa wrote: post #125
BeeOnRope wrote: post #121

Earlier quoted context omitted.

There's another option: 4) Translate x86 loads and stores to acquire load and release stores, to align with the x86 semantics. These already exist in the ARM ISA, so it's not much of a stretch at all. This is the one I'm betting on.

Expect you'd need to do that with every single store since the x86 instruction stream doesn't have those semantics embedded in it. That'd kill your memory perf by at least an order of magnitude, and kill perf for other cores as well. It'd be cheaper to just say "you only get one core in x86 mode". Essentially you'd be marking every store as an L1 and store buffer flush and only operating out of L2.

What I can say is that they aren't pinning to only a single core, so the answer is elsewhere.

Re: About the Rosetta Translation Environment

#138

Post by saagarjha »

underdeserver wrote: post #58
jeffbee wrote: post #43

Earlier quoted context omitted.

But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset?

> But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? Yup. > I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset? Well, not…

> In modern OSes, memory is generally either writable or executable, but not both; so if you want to compile and write, and then execute, you need to call mprotect() between those steps to set permissions.

mprotect is slow. High-performance JIT compilers on macOS usually map memory as RWX or mirror-map the same page as RW- and R-X from two virtual addresses.

Re: About the Rosetta Translation Environment

#139

Post by vmception »

amaranth wrote: post #134
mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

I'm not sure why they're making a big deal about this, couldn't the original Rosetta do this too? QEMU has been doing this since (I think) even before the original Rosetta, they call it user mode emulation. You run as if it was a normal emulator but also trap syscalls and forward them to the native kernel instead of emulating a kernel too. I'm more interested in how they're doing the AOT conversion and (presumably) p…

> I'm not sure why they're making a big deal about this, couldn't the

That's par for course with Apple. They never acknowledge competitors including when its themselves. Everything they do or describe is awesome and magical, right now! It could be an incremental improvement, it could be a half decade old established technology, it could be something completely unexpected and science fiction turned into reality. Thats just how Apple works.

Re: About the Rosetta Translation Environment

#140

Post by dtech »

monocasa wrote: post #120
api wrote: post #93

Earlier quoted context omitted.

True, but those are trivial to port as there are already mature JDKs and JavaScript VMs for ARM64. Porting an electron app should be a matter of checking it out and building it with ARM support. Applications with internal JITs custom-built for X64 are going to be a lot tougher.

Electron doesn't support aarch64 builds compiled from aarch64 hosts yet (so you have to cross compile), and doesn't support cross compiling if you have any native code addons. Some apps out there are between a rock and a hard place if it weren't for rosetta.

I suspect this will change fairly soon now that there will be a significant non-mobile aarch64 market in the future
Post reply on HN