Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Rosetta 2 fast?

dougallj.wordpress.com

Post reply on HN
Print view View original

161–170 of 367 posts

Re: Why is Rosetta 2 fast?

#161

Post by Symmetry »

my123 wrote: post #153
Symmetry wrote: post #109

Earlier quoted context omitted.

If you've got some management core somewhere in your silicon you can, with RISC-V, give it a MMU but no FPU and save area. You're going to be writing custom embedded code anyways so you get to save silicon by only incorporating the features that you need instead of having to meet the full ARM spec. And you can add your own custom instructions for the job at hand pretty easily. That would all be a terrible idea if you…

Arm64 allows FPU-less designs. There are some around…

Sure. The FPU is optional on a Cortex M2, for instance. But those don't have MMUs. You'd certainly need an expensive architectural license to make something with an MMU but no FPU if you wanted to and given all the requirements ARM normally imposes for software compatibility[1] between cores I'd tend to doubt that they'd let you make something like that.

[1] Explicitly testing that you don't implement total store ordering by default is one requirement I've heard people talk about to get a custom core licensed.

Re: Why is Rosetta 2 fast?

#162

Post by skrrtww »

cwzwarich wrote: post #111
darzu wrote: post #76

Does anyone know the names of the key people behind Rosetta 2? In my experience, exceptionally well executed tech like this tends to have 1-2 very talented people leading. I'd like to follow their blog or Twitter.

I am the creator / main author of Rosetta 2. I don't have a blog or a Twitter (beyond lurking).

Are you able to speak at all to the known performance struggles with x87 translation? Curious to know if we're likely to see any updates or improvements there into the future.

Re: Why is Rosetta 2 fast?

#163

Post by Vt71fcAqt7 »

masklinn wrote: post #95
preisschild wrote: post #68

Earlier quoted context omitted.

> They've got 15 years experience and essentially full control on ARM. Do they? ARM made it very clear that they consider all ARM cores their own[1] [1]: https://www.theregister.com/2022/11/07/opinion_qualcomm_vs_a...

> Do they? They do, yes. They were one of the founding 3 members of ARM itself, and the primary monetary contributor. Through this they acquired privileges which remain extant: they can literally add custom instructions to the ISA ( https://news.ycombinator.com/item?id=29798744 ), something there is no available license for. > ARM made it very clear that they consider all ARM cores their own[1] The Qualcomm situation…

Is there a real source for this claim? It gets parroted a lot on HN and elsewhere, but I've also heard it's greatly exagerated. I don't think Apple engineers get to read the licences, and even if they did, how do we know they understood it corretly and that it got repeated correctlty? I've never seen a valid source for this claim.

Re: Why is Rosetta 2 fast?

#164

Post by duped »

com2kid wrote: post #121
kmeisthax wrote: post #103

Earlier quoted context omitted.

It's more the case that the ahead-of-time compilation is suboptimal. Modern compilers have a thing called PGO (Profile Guided Optimization) that lets you take a compiled application, run it and generate an execution profile for it, and then compile the application again using information from the profiling step. The reason why this works is that lots of optimization involves time-space tradeoffs that only make sense…

> Theoretically, a JIT could produce binary code hyper-tailored to a particular user's habits and their computer's specific hardware. However, I'm not sure if that has that much of a benefit versus PGO AOT. In theory JIT can be a lot more efficient, optimizing for not only the exact instruction set, and do per CPU architecture optimizations, such as instruction length, pipeline depth, cache sizes, etc. In reality I d…

Like another commented, JIT compilers do this today.

The thing that makes this mostly theoretical is that the underlying assumption is only true when you neglect that an AOT has zero run-time cost while a JIT compiler has to execute the code it's optimizing and the code to decide if it's worth optimizing and generate new code.

So JIT compiler optimizations are a bit different than AOT optimizations since they have to both generate faster/smaller code and the execute code that performs the optimization. The problem is that most optimizations beyond peephole are quite expensive.

There's another thing that AOT compilers don't need to deal with, which is being wrong.Production JITs have to implement dynamic de-optimization in the case that an optimization was built on a bad assumption.

That's why JITs are only faster in theory (today), since there are performance pitfalls in the JIT itself.

Re: Why is Rosetta 2 fast?

#165

Post by masklinn »

Symmetry wrote: post #161
my123 wrote: post #153

Earlier quoted context omitted.

Arm64 allows FPU-less designs. There are some around…

Sure. The FPU is optional on a Cortex M2, for instance. But those don't have MMUs. You'd certainly need an expensive architectural license to make something with an MMU but no FPU if you wanted to and given all the requirements ARM normally imposes for software compatibility[1] between cores I'd tend to doubt that they'd let you make something like that. [1] Explicitly testing that you don't implement total store ord…

Apple has an architecture license (otherwise they could not design their own cores, which they’ve been doing for close to a decade), and already had the ability to take liberties beyond what the average architecture licensee can, owing to being one of ARM’s founders.

Re: Why is Rosetta 2 fast?

#166

Post by duped »

mockery wrote: post #106
freedomben wrote: post #83

Earlier quoted context omitted.

If JIT-ing a statically compiled input makes it faster, does that mean that JIT-ing itself is superior or does it mean that the static compiler isn't outputting optimal code? (real question. asked another way, does JIT have optimizations it can make that a static compiler can't?)

In addition to the sibling comments, one simple opportunity available to a JIT and not AOT is 100% confidence about the target hardware and its capabilities. For example AOT compilation often has to account for the possibility that the target machine might not have certain instructions - like SSE/AVX vector ops, and emit both SSE and non-SSE versions of a codepath with, say, a branch to pick the appropriate one dynam…

AOT compilers support this through a technique called function multi-versioning. It's not free and only goes so far, but it isn't reserved to JITs.

The classical reason to use FMV is for SIMD optimizations, fwiw

Re: Why is Rosetta 2 fast?

#167

Post by saagarjha »

peatmoss wrote: post #9

Not having any particular domain experience here, I've idly wondered whether or not there's any role for neural net models in translating code for other architectures. We have giant corpuses of source code, compiled x86_64 binaries, and compiled arm64 binaries. I assume the compiled binaries represent approximately our best compiler technology. It seems predicting an arm binary from an x86_64 binary would not be insa…

People have tried doing this, but not typically at the instruction level. Two ways to go about this that I’m aware of are trying to use machine learning to derive high-level semantics about code, then lowering it to the new architecture.

Re: Why is Rosetta 2 fast?

#168

Post by masklinn »

Vt71fcAqt7 wrote: post #163
masklinn wrote: post #95

Earlier quoted context omitted.

> Do they? They do, yes. They were one of the founding 3 members of ARM itself, and the primary monetary contributor. Through this they acquired privileges which remain extant: they can literally add custom instructions to the ISA ( https://news.ycombinator.com/item?id=29798744 ), something there is no available license for. > ARM made it very clear that they consider all ARM cores their own[1] The Qualcomm situation…

Is there a real source for this claim? It gets parroted a lot on HN and elsewhere, but I've also heard it's greatly exagerated. I don't think Apple engineers get to read the licences, and even if they did, how do we know they understood it corretly and that it got repeated correctlty? I've never seen a valid source for this claim.

For what claim? They they co-founded ARM? That’s historical record. That they extended the ISA? That’s literally observed from decompilations. That they can do so? They’ve been doing it for at least 2 years and ARM has yet to sue.

> I've never seen a valid source for this claim.

What is “a valid source”? The linked comment is from Hector Martin, the founder and lead of Asahi, who worked on and assisted with reversing various facets of Apple silicon, including the capabilities and extensions of the ISA.

Re: Why is Rosetta 2 fast?

#169

Post by varenc »

dylan604 wrote: post #135
nicoburns wrote: post #30

Earlier quoted context omitted.

> If so, does it affect the application startup times? It does, but only the very first time you run the application. The result of the transpilation is cached so it doesn't have to be computed again until the app is updated.

Does that essentially mean each non-native app is doubled in disk use? Maybe not doubled but requires more space to be sure.

Yes... you can see the cache in /var/db/oah/

Though only the actual binary size that gets doubled. For large apps it’s usually not the binary that’s taking up most of the space.

Re: Why is Rosetta 2 fast?

#170

Post by saagarjha »

One thing that’s interesting to note is that the amount of effort expended here is not actually all that large. Yes, there are smart people working on this, but the performance of Rosetta 2 for the most part is probably the work of a handful of clever people. I wouldn’t be surprised if some of them have an interest in compilers but the actual implementation is fairly straightforward and there isn’t much of the stuff you’d typically see in an optimizing JIT: no complicated type theory or analysis passes. Aside from a handful of hardware bits and some convenient (perhaps intentionally selected) choices in where to make tradeoffs there’s nothing really specifically amazing here. What really makes it special is that anyone (well, any company with a bit of resources) could’ve done it but nobody really did. (But, again, Apple owning the stack and having past experience probably did help them get over the hurdle of actually putting effort into this.)
Post reply on HN