Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

About the Rosetta Translation Environment

developer.apple.com

Post reply on HN
Print view View original

91–100 of 249 posts

Re: About the Rosetta Translation Environment

#91

Post by monocasa »

lsllc wrote: post #89
monocasa wrote: post #57

Earlier quoted context omitted.

Since they don't enforce W^X you need to go a little deeper than that since some JITs reuse the same memory later for new traces. So what you'll do is silently enforce W^X to trap code modifications, but not propagate those traps to the emulated code.

Right, if you look at the example JIT project I posted, they start by creating the memory with mmap() and PROT_EXEC before generating any code. So yes, you'd need to trap subsequent writes to allow for retranslation [I assume they have hooks in the Darwin kernel for this].

Or they just enforce W^X in the rosetta runtime by intercepting client mmap calls and fix it up as far as the client code is concerned by catching SIGBUS first.

I don't see anything here that requires extra kernel hooks.

Re: About the Rosetta Translation Environment

#92

Post by toyg »

Angostura wrote: post #59
DCKing wrote: post #40

Rosetta's goal is to support legacy Mac apps. It's quite finely scoped, but I hope Apple will go above this and make it available as part of their virtualization framework. This way e.g. Parallels and Docker could use this to provide some way to tap in probably the fastest way to run x86 on ARM Macs. This would mean Rosetta would go beyond its stated scope, and certainly go beyond what the relatively short-lived Rose…

FWIW they showed the latest Tombraider game running on Rosetta on a dev Mac said to be using the ARM chip from the high-end iPad. It ran pretty well. Not sure if I would quite count this as AAA

> It ran pretty well

It was a postproduced, recorded promotional video, not a live demo. They could have shown laptops reaching Mars and you’d have thought “that landing gear looks pretty solid”.

Re: About the Rosetta Translation Environment

#93

Post by api »

needusername wrote: post #90
api wrote: post #85

Earlier quoted context omitted.

"Just" This is some serious black magic, especially if it actually works. Even if it only works some of the time I give it an A for effort, since they could easily have just said "JITs will not work with Rosetta." Most applications do not contain JITs. My guess is there are enough important Mac apps that contain JITs of some form to merit this work, probably high performance math and graphics kernels that JIT process…

> Most applications do not contain JITs. Java apps and JavaScript apps (Electron) come to mind.

True, but those are trivial to port as there are already mature JDKs and JavaScript VMs for ARM64. Porting an electron app should be a matter of checking it out and building it with ARM support. Applications with internal JITs custom-built for X64 are going to be a lot tougher.

Re: About the Rosetta Translation Environment

#94

Post by ryanianian »

rz2k wrote: post #71

Does anyone have any insight into the business logistics involved in a transition like this? I presume Apple has done maintenance on transitioning desktop OS X to ARM as an option for a very long time. How many years ago would they have had to decide that was the direction they were going to make it reality? 2015? How many people would have been working on it? How many billions of dollars? How does the cost of develo…

> How many billions of dollars?

Rounding-error-sized marginal costs.

Even if you don't plan on pivoting architectures, it's good practice to write as much code as possible without assembly/intrinsics. If you're writing standards-compliant C/C++/Objective-C/Swift/* it's not a huge burden to change architectures. They also have past-experience with such transitions, so hopefully they kept the "keep the code portable" mantra close over the past 10+ years.

The choice to do this seems relatively obvious for Apple. They have spent years building up their silicon game with portables. Their frustration with Intel is also palpable, so it was probably "just" a waiting game (not "how" or "if") in deciding when to pull the trigger and start using those chips outside of iPhones/iPads.

Re: About the Rosetta Translation Environment

#95

Post by monocasa »

andoma wrote: post #87

I wonder how it handles the stricter memory ordering from x86? For example, this code: void store_value_and_unlock(int *p, int *l, int v) { *p = v; __sync_lock_release(l, 0); } on x86_64 compiles to: mov dword ptr [rdi], edx mov dword ptr [rsi], 0 ret vs on arm64: str w2, [x0] stlr wzr, [x1] ret Notice how the second store on ARM is a store with release semantics to ensure correct memory ordering as it was intended i…

That's the big piece I've been wondering about too.

Three options as I see it (none of them great):

1) Pin all threads in an x86 process to a single core. You don't have memory model concerns on a single core.

2) Don't do anything? Just rely on apps to use the system provided mutex libraries, and they just break if they try to roll their own concurrency? Seems like exactly the applications you care about (games, pro apps), would be the ones most likely to break.

3) Some stricter memory model in hardware? Seems like that'd go against most of the stated reason for switching to ARM in the first place.

Re: About the Rosetta Translation Environment

#96

Post by ezconnect »

Bigpet wrote: post #44
godzillabrennus wrote: post #38

The Apple transition to ARM for me is sad. There was so much positivity around the Intel transition. It opened up the Mac platform. Now it’s going back into a closed black box.

I think I'm missing something. How is ARM closing things up? Does Apple have a bunch of processor extensions that make it incompatible with other ARM processors? Or do you think they'll take the chance to kill boot-camp or something?

Everything will now be digitally signed, look at the iphone jailbreak scene on how hard is it to customize your own hardware to run the software you want.

Re: About the Rosetta Translation Environment

#97

Post by djrogers »

rz2k wrote: post #71

Does anyone have any insight into the business logistics involved in a transition like this? I presume Apple has done maintenance on transitioning desktop OS X to ARM as an option for a very long time. How many years ago would they have had to decide that was the direction they were going to make it reality? 2015? How many people would have been working on it? How many billions of dollars? How does the cost of develo…

> * I am also really curious about the accounting and financial disclosures required in such decisions. *

There are none, beyond the dollar figures set aside for R&D.

> How much are institutional investors informed of long range plans?

In the case of Apple, very little to not at all. Other companies have different policies - most share roadmaps and long range plans rather freely compared to Apple.

Re: About the Rosetta Translation Environment

#98

Post by underdeserver »

jeffbee wrote: post #80
underdeserver wrote: post #58

Earlier quoted context omitted.

> But doesn't this mean that they are translating and running the JIT and the JIT is creating new executable x86 code which then also needs to be translated? Yup. > I'm a bit baffled as to how any of this can work properly, for example if my JIT is sampling instructions to determine when to optimize, does Rosetta need to reverse-translate the actual instruction pointer back to the original x86 code offset? Well, not…

The hard part seems to be metadata, or information about the program rather than the program itself. Rosetta will translate the program but it doesn't know about the metadata, so you have a referential integrity problem, seemingly. For example what is going to happen if a program running under Rosetta uses setitimer to deliver SIGPROF? What's in si_addr?

You'll still have the address of the fault in si_addr. Addresses are addresses, both in AMD64 and ARM64. What's the difference?

Re: About the Rosetta Translation Environment

#99

Post by jeffbee »

underdeserver wrote: post #98
jeffbee wrote: post #80

Earlier quoted context omitted.

The hard part seems to be metadata, or information about the program rather than the program itself. Rosetta will translate the program but it doesn't know about the metadata, so you have a referential integrity problem, seemingly. For example what is going to happen if a program running under Rosetta uses setitimer to deliver SIGPROF? What's in si_addr?

You'll still have the address of the fault in si_addr. Addresses are addresses, both in AMD64 and ARM64. What's the difference?

Maybe I'm just imagining things, but if I have a map from PC to function or line and Rosetta rewrites all my functions then I have a problem unless the Rosetta output is by some miracle exactly the same length as the input.

Re: About the Rosetta Translation Environment

#100

Post by eholk »

mkl wrote: post #41

> Rosetta can translate most Intel-based apps, including apps that contain just-in-time (JIT) compilers. How on Earth does it do that? If executable code is being generated at runtime, it's going to be x86_64 binary machine code still (there are too many ways to generate valid machine code, and it won't know right away whether you're JITting, or cross compiling and actually want x86_64), so Rosetta would need to dete…

Typically the way systems do this is by translating small sections of straightline code, and patching the exits as they are translated. So you start by saying translate the block at address 0x1234. That code may go until a jump to address 0x4567. When translating that jump, they instead make a call to the runtime system which says "where is the translated code starting at address 0x4567?" If the code doesn't exist, it goes ahead and translates that block and patches the originally jump to skip the runtime system next time around.

This means early on in the program's run you spend a lot of time translating code, but it pretty quickly stabilizes and you spend most of your time in already translated code.

Of course, if your program is self modifying then the system needs to do some more work to invalidate the translation cache when the underlying code is modified.

Post reply on HN