Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About the Rosetta Translation Environment

developer.apple.com

Post reply on HN
Print view View original

51–60 of 249 posts

Re: About the Rosetta Translation Environment

#51

Post by haxxorfreak »

godzillabrennus wrote: post #38

The Apple transition to ARM for me is sad. There was so much positivity around the Intel transition. It opened up the Mac platform. Now it’s going back into a closed black box.

I'm optimistic and while the constant references to ARM as "Apple Silicon" make it sounds proprietary I see it as an example of Apple following Jobs' goal of "skating to where the puck will be".

Microsoft is toying with the same transition with Windows 10 ARM builds and the Surface Pro X, and the news yesterday about an ARM-based supercomputer taking the TOP500 crown are all signs that ARM is the future. It will take years but I foresee most "general computing" devices making the transition in the future.

Re: About the Rosetta Translation Environment

#52

Post by monocasa »

leeter wrote: post #46
monocasa wrote: post #32

Earlier quoted context omitted.

Or they're just giving Intel the finger, banking on having a warchest of defensive patents from acquiring nearly every promising fabless CPU startup of the past 15 years. Edit: Also, I don't think Transmeta's license would include SSE4. They were defunct at that point.

Possible, I kinda doubt they would as Intel could just cut them off at the knees as they threatened MS with for the Surface Pro X. There is the remote possibility they are licensing from Intel too. At the end of the day I doubt they are going to tell us. Once people start getting the dev machines they can find the edges of the implementation. I suspect it has some quirks we'll discover over time. Regardless this is s…

> I kinda doubt they would as Intel could just cut them off at the knees

And then Apple would cut Intel off at the knees in return. Microsoft doesn't really have any CPU design patents, Apple has thousands, so they'd be treated differently.

Re: About the Rosetta Translation Environment

#53

Post by lsllc »

mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

It probably just hooks mmap() (or mprotect()) and looks for when the JIT sets PROT_EXEC on memory it hasn't translated yet.

Here's a tutorial for a simple JIT that uses mmap() with PROT_EXEC to write machine code to before executing:

https://github.com/spencertipping/jit-tutorial

Re: About the Rosetta Translation Environment

#54

Post by mkl »

monocasa wrote: post #50
mkl wrote: post #47

Earlier quoted context omitted.

I think it's neither of those things. The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation and the API calls don't need to change.

> The x86_64 machine code instructions are translated into ARM instructions, so there's no emulation That's what emulation is.

I think emulation would be interpreting the instructions one by one and doing what they say in software (a bit like Python's byte code interpreter). Rosetta translates the binary to another (now native) binary, and then runs it like normal, and much of the time this will be an ahead-of-time once off translation.

Re: About the Rosetta Translation Environment

#55

Post by lsllc »

cpuguy83 wrote: post #19
marcinzm wrote: post #4

>However, Rosetta doesn’t translate the following executables: Virtual Machine apps that virtualize x86_64 computer platforms How will this impact docker? Does this mean you can't run x86 docker containers on new Apple laptops?

Apple mentioned they are specifically "working with Docker to support 'these things' in coming months". Confirmed with some Docker folks they are working on "something". All very nondescript, but they said they can't talk about it yet.

Possibly they'll do something like WSL/WSL2 to provide an ARM based Linux kernel interface and then use Rosetta2 to translate the x64 code in user space to ARM?

Having ARM only containers would be mostly useless (it's technically possible, but I bet there are very few ARM container images around).

Re: About the Rosetta Translation Environment

#56

Post by bluedino »

mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

There might be insights from the way Dolphins PPC JIT works:

https://www.reddit.com/r/emulation/comments/2xq5ar/how_close...

Re: About the Rosetta Translation Environment

#57

Post by monocasa »

lsllc wrote: post #53
mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

It probably just hooks mmap() (or mprotect()) and looks for when the JIT sets PROT_EXEC on memory it hasn't translated yet. Here's a tutorial for a simple JIT that uses mmap() with PROT_EXEC to write machine code to before executing: https://github.com/spencertipping/jit-tutorial

Since they don't enforce W^X you need to go a little deeper than that since some JITs reuse the same memory later for new traces.

So what you'll do is silently enforce W^X to trap code modifications, but not propagate those traps to the emulated code.

Re: About the Rosetta Translation Environment

#58

Post by underdeserver »

jeffbee wrote: post #43
saagarjha wrote: post #31

Earlier quoted context omitted.

At runtime it doesn't really matter where you code is coming from, as long as you do flushes at the right spots.

But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset?

> But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated?

Yup.

> I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset?

Well, not exactly. Let's take JS as an example. First of all, you're only sampling calls that are still in the native language, so your instruction pointer is in the emulated virtual machine itself anyway, which is agnostic to the CPU architecture. See [1] for where SpiderMonkey samples.

What happens then is that the engine sees a function being called many times, so the JIT compiler decides to compile a function; it compiles the function and writes the compiled function to memory, then overwrites that function's address with a native function pointer.

The easiest hook the OS has to this process is mprotect(2). In modern OSes, memory is generally either writable or executable, but not both; so if you want to compile and write, and then execute, you need to call mprotect() between those steps to set permissions.

Knowing that your binary is Rosetta'd, MacOS can assume your JIT code is also AMD64 and translate that page on the fly. All they have to do is make sure the virtual memory addresses are correct - to not mess up those function pointers.

They can either do it lazily, through a pagefault handler, or greedily when mprotect is called to add executable permissions to the JIT-allocated memory.

[1] https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Sp...

Re: About the Rosetta Translation Environment

#59

Post by Angostura »

DCKing wrote: post #40

Rosetta's goal is to support legacy Mac apps. It's quite finely scoped, but I hope Apple will go above this and make it available as part of their virtualization framework. This way e.g. Parallels and Docker could use this to provide some way to tap in probably the fastest way to run x86 on ARM Macs. This would mean Rosetta would go beyond its stated scope, and certainly go beyond what the relatively short-lived Rose…

FWIW they showed the latest Tombraider game running on Rosetta on a dev Mac said to be using the ARM chip from the high-end iPad. It ran pretty well. Not sure if I would quite count this as AAA

Re: About the Rosetta Translation Environment

#60

Post by proninyaroslav »

Bigpet wrote: post #44
godzillabrennus wrote: post #38

The Apple transition to ARM for me is sad. There was so much positivity around the Intel transition. It opened up the Mac platform. Now it’s going back into a closed black box.

I think I'm missing something. How is ARM closing things up? Does Apple have a bunch of processor extensions that make it incompatible with other ARM processors? Or do you think they'll take the chance to kill boot-camp or something?

I think Apple has more maneuvers on ARM, since in general the PC architecture is open for modifications, unlike ARM. They will do it the same way as with iOS devices - a closed bootloader and hardware protection.
Post reply on HN