Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is Rosetta 2 fast?

dougallj.wordpress.com

Post reply on HN
Print view View original

351–360 of 367 posts

Re: Why is Rosetta 2 fast?

#351

Post by olliej »

phire wrote: post #332
olliej wrote: post #279

Earlier quoted context omitted.

There are two ways to approach x87: either saying to heck with it and just using doubles for everything (this is essentially what Qemu does) or creating a software fp80 implementation. Both approaches get burned by the giant amount of state, and state weirdness, that x87 brings to the table. It's also not possible to "fix" things by optimizing for the cases where the x87 unit's precision is set to the same as fp32 or…

> It's also not possible to "fix" things by optimizing for the cases where the x87 unit's precision is set to the same as fp32 or fp64, as the precision flags don't impact the exponent range. I've been meaning to look into this. Certainly you can't blindly optimise all x87 code sequences to fp32 or fp64. But some sequences are safe. For example, adding two numbers and saving back to memory is safe to optimise (at lea…

> For example, adding two numbers and saving back to memory is safe to optimise (at least for the infinity case, I haven't double-checked the subnormal behaviour). It's only when you need to add three or more numbers that you run into issues (though you can go further, if all N numbers have the same sign, you will get the correct result, you just might have saturated at infinity a few operations earlier than native x87)

No, you cannot. Operations you can optimise are negation, nan, and infinity checks (ignoring pseudo nans and pseudo infinity checks of course).

fp80 has a 15bit exponent, and functionally a 63 significand, vs fp64's 11bit exponent and 52bit significand. When setting x87 to a reduced precision mode you aren't switching to fp64, you're getting a mix with a 15 bit exponent and a 53 significand. The effect is that you retain 53bits of precision for values where fp64 has entered subnormals, and conversely you maintain 53 bits of precision after fp64 has overflowed. There are perf benefits to reducing precision in x87 (at least in the 90s), but the main advantage is consistent rounding with fp64 while in the range of normalized fp64 values.

Re: Why is Rosetta 2 fast?

#352

Post by phire »

olliej wrote: post #351
phire wrote: post #332

Earlier quoted context omitted.

> It's also not possible to "fix" things by optimizing for the cases where the x87 unit's precision is set to the same as fp32 or fp64, as the precision flags don't impact the exponent range. I've been meaning to look into this. Certainly you can't blindly optimise all x87 code sequences to fp32 or fp64. But some sequences are safe. For example, adding two numbers and saving back to memory is safe to optimise (at lea…

> For example, adding two numbers and saving back to memory is safe to optimise (at least for the infinity case, I haven't double-checked the subnormal behaviour). It's only when you need to add three or more numbers that you run into issues (though you can go further, if all N numbers have the same sign, you will get the correct result, you just might have saturated at infinity a few operations earlier than native x…

The key to this optimisation idea is the exponent gets truncated back to 8 bits when being written back to memory.

For the example of two fp32s adding to infinity.

With 15bit exponent: The add results in a non-infinite with exponent outside the -125 to 127 range. Then when writing back to memory, it the FPU notices the exponent is outside of the valid range, clamps it and writes infinity to memory.

With 8 bit exponent: The add immediately clamps to infinity in the register. and then it writes to memory.

In both cases you get the same result in memory, so the result is valid as long as the in-register version is killed. And the same should apply to the subnormal case (I have not double checked the x87 spec). If you start with two subnormals that are valid f32s, add them, get a subnormal result and then write back to memory as a f32, it should be guaranteed to produce the same result with both a 15bit exponent and a 8bit exponent. It doesn't matter if the subnormal mantissa was truncated before writing back to the register, or writing back to memory. It was still truncated.

You only start getting accuracy issues start doing multiple additions in a row without truncating the exponent. If you add 3 floats, the result of A+B might be infinite, but A+B+C could result in a normal f32 if you had a 15bit exponent (when A+B is positive infinity, and C is negative)

This line of thought could potentially be pushed further. If you can prove (or guard) at compile time that all N floats in a sequence of N adds will be positive (and not subnormal), then you can't have a case where one of intermediary exponent exceeds 127 but then the final exponent is less than 128. If there is an infinite anywhere along the chain, it will saturate to infinity. With 15bit exponent, the saturation might not be applied until the f32 value is written to memory at the end, but because of the preconditions the optimiser can guarantee the same result in memory (either infinity or a normal) at the end while only using 8 bit exponents operations.

Most of the above should also apply to other operations like multiply. I've only done some preliminary thinking about this idea, enough to be sure that some operations could be optimised. I'm only fully confident about clamping to infinity case, and I'm going to be really bummed if when I get around to double-checking, there is something about how the x87 deals with subnormals that I'm not aware of. Or some other x87 weirdity.

Re: Why is Rosetta 2 fast?

#353

Post by saagarjha »

menaerus wrote: post #334
dougall wrote: post #265

Earlier quoted context omitted.

Yeah, these features exist, and they help, but I don't think they should be given all the credit. Both "Apple's Secret Extension" and "Total Store Ordering" are features that other emulators can choose to disable to get exactly the same performance. "Apple's Secret Extension" isn't even used by Rosetta 2 on Linux (opting for, at least, explicit parity flag calculations rather than reduced fidelity). It's still fast.…

> TSO is only required for accuracy on multithreaded applications If by accuracy you mean not segfaulting then yes. Every moderately complex x86-64 application will have memory fences in the generated machine code. x86-64 design of store-buffers and load-buffers are making the memory fences a necessity. In reality it's enough just to use the mutex or atomics in your code to end up with the memory fence in your genera…

You can approximate it fairly well without much impact.

Re: Why is Rosetta 2 fast?

#354

Post by bogomipz »

GeekyBear wrote: post #192
kijiki wrote: post #146

Earlier quoted context omitted.

> Anyone know how they implemented PPC-to-x86 translation? They licensed Transitive's retargettable binary translator, and renamed it Rosetta; very Apple. It was originally a startup, but had been bought by IBM by the time Apple was interested.

> It was originally a startup, but had been bought by IBM by the time Apple was interested. Rosetta shipped in 2005. IBM bought Transitive in 2008. The last version of OS X that supported Rosetta shipped in 2009. I always wondered if the issue was that IBM tried to alter the terms of deal too much for Steve's taste.

>"The last version of OS X that supported Rosetta shipped in 2009."

Interesting so was Rosetta 2 written from the ground up then? Did Apple manage to hire any of the former Transitive engineers after IBM acquired them? It seems like this would be a niche group of engineers that worked in this area no?

Re: Why is Rosetta 2 fast?

#355

Post by olliej »

phire wrote: post #352
olliej wrote: post #351

Earlier quoted context omitted.

> For example, adding two numbers and saving back to memory is safe to optimise (at least for the infinity case, I haven't double-checked the subnormal behaviour). It's only when you need to add three or more numbers that you run into issues (though you can go further, if all N numbers have the same sign, you will get the correct result, you just might have saturated at infinity a few operations earlier than native x…

The key to this optimisation idea is the exponent gets truncated back to 8 bits when being written back to memory. For the example of two fp32s adding to infinity. With 15bit exponent: The add results in a non-infinite with exponent outside the -125 to 127 range. Then when writing back to memory, it the FPU notices the exponent is outside of the valid range, clamps it and writes infinity to memory. With 8 bit exponen…

> The key to this optimisation idea is the exponent gets truncated back to 8 bits when being written back to memory.

That's incorrect - the clamping of the exponent only occurs if you were to use FST/m32 or FST/m64, but if you're using x87 you're presumably doing FSTP/m80fp so there is no truncation or rounding on store, regardless of the prevision flag in the control word.

It sounds like what you're trying to arrange is an optimization such that a rosetta like translator/emulator can optimize this highly awesome function to be performed entirely using hardware fp32 or fp64 support:

    fp32 f(fp32 *fs, size_t count) {
      // pseudo code obviously :D
      ControlWord cw = fstcw();
      cw.precision = Precision32;
      fldcw(cw);
      fp80 result = 0;
      for (unsigned j = 0; j
The problem you run into though, is that an optimization pass can make decisions based on anything other than the code it is presented with. So your optimizer can't assume sign or magnitude here, so that += has to be able to under or overflow the range that fp32 offers.

Things get really miserable once you go beyond +/-, because you can end up in a position where an optimization to do everything in the 32/64 bit units means that you won't get observable double rounding.

This is kind of moot in the rosetta case as I don't believe we ever implemented support for the precision control bits

More fun are the transcendentals - x87 specifies them as using range reduction and so they are incredibly inaccurate (in the context of maths functions), especially around multiples of pi/4, and if you go test it you'll find rosetta will produce the same degree of terrible output :D

Re: Why is Rosetta 2 fast?

#356

Post by phire »

olliej wrote: post #355
phire wrote: post #352

Earlier quoted context omitted.

The key to this optimisation idea is the exponent gets truncated back to 8 bits when being written back to memory. For the example of two fp32s adding to infinity. With 15bit exponent: The add results in a non-infinite with exponent outside the -125 to 127 range. Then when writing back to memory, it the FPU notices the exponent is outside of the valid range, clamps it and writes infinity to memory. With 8 bit exponen…

> The key to this optimisation idea is the exponent gets truncated back to 8 bits when being written back to memory. That's incorrect - the clamping of the exponent only occurs if you were to use FST/m32 or FST/m64, but if you're using x87 you're presumably doing FSTP/m80fp so there is no truncation or rounding on store, regardless of the prevision flag in the control word. It sounds like what you're trying to arrang…

I was thinking more about functions along the lines of this vertex transform function that you might theoretically find as hot code in a late 90s or early 2000s windows game (before hardware transform and lighting).

    void transform_verts(fp32 *m, fp32 *verts, size_t vert_count) {
      // it's a game, decent chance it applies percision32 across the whole process
      // Especially since directx
Would be nice if we could optimise it all to pure hardware fp32 without any issues. But not really possible with those six operation long chains. And you are right, we can't really assume anything about the data.

But we can go for guards and fallbacks instead. Implement that loop body as something like

    loop: 
        // Attempt calculation with hardware fp32
        $1 = hwmul(verts[0], m[0])
        $2 = hwmul(verts[0], m[1])
        $3 = hwmul(verts[0], m[2])
        $4 = hwadd($1, $2)
        $5 = hwadd($3, $4)
        $6 = hwadd($5, m[3]) // any infs from above sub-equations will saturate though to here
        if any(is_subnormal_or_zero([$1, $2, $3, $4, $5]) || $6 is inf: // guard
           // one of the above subcalulcations became either inf or subnormal, so our
           // hwresult might not be accurate. recalculate with safe softfloat
           $8 = swadd(swmul(verts[0], m[0]), swmul(verts[0], m[1]))
           $6 = swadd(swadd($8, swmul(vert[0], m[2])), m[3])
    
        verts[0] = $6
    
        // repeat above pattern for verts[1] and verts[2]
    
        goto loop
I think that produces bit-accurate results?

Sure, it might seem complicated to calculate twice. But the resulting code might end up faster than just pure softfloat code across average data. Maybe this is the type of optimisation that you only attempt at the highest level on a multi-tier JIT for really hot code. You could perhaps even instrument the function first to get an idea what the common shape of the data is.

> This is kind of moot in the rosetta case as I don't believe we ever implemented support for the precision control bits

So it's already producing inaccurate results for code that sets precision control? Might as well just switch over to hardware fp32 and fp64 /s

I guess for the rosetta usecase, Intel macs didn't until 2006 and so most of the install base of x86 programs will be compiled with SSE2 support, and commonly 64bit.

Probably the most common usecase for x87 support in rosetta will be 64bit code used long doubles and compilers/ABIs annoying implemented them as x87.

Re: Why is Rosetta 2 fast?

#357

Post by masklinn »

adgjlsfhk1 wrote: post #252
masklinn wrote: post #223

Earlier quoted context omitted.

> Consider that the wide availability of crappy low end hardware gave Windows laptops a terrible reputation. Standard DPI displays are not "crappy low-end hardware"? I don't think there's a single widescreen display which qualifies as hiDPI out there, that more or less doesn't exist: a 5K 34" is around 160 DPI (to say nothing of the downright pedestrian 5K 49" like the G9 or the AOC Agon).

dpi isn't the right metric. you don't keep a phone the same distance as a 49" monitor.

DPI is absolutely the right metric, the target goal just depends on the usage.

A hidpi computer display is north of 200 (iMacs are a bit below 220, MacBooks a bit above).

A phone is north of 300 (iPhones have ranged from 324 to 476).

Re: Why is Rosetta 2 fast?

#358

Post by scarface74 »

xattt wrote: post #331
scarface74 wrote: post #295

Earlier quoted context omitted.

> In comparison, the emulator used during the 68k to PPC transition was never removed from Classic MacOS, so the change stood out. It was never removed because Classic MacOS itself was never fully native.

> It was never removed because Classic MacOS itself was never fully native. Are there any current OSes that have the same level of historical cruft that Mac OS Classic had?

Windows.

Depending on which API you are calling, you have to represent a “string” differently. This is just one example

https://learn.microsoft.com/en-us/cpp/text/how-to-convert-be...

Re: Why is Rosetta 2 fast?

#359

Post by GeekyBear »

scarface74 wrote: post #358
xattt wrote: post #331

Earlier quoted context omitted.

> It was never removed because Classic MacOS itself was never fully native. Are there any current OSes that have the same level of historical cruft that Mac OS Classic had?

Windows. Depending on which API you are calling, you have to represent a “string” differently. This is just one example https://learn.microsoft.com/en-us/cpp/text/how-to-convert-be...

There are still dialog boxes that date back to Windows 3.1 that show up in Windows 10 and 11.

Re: Why is Rosetta 2 fast?

#360

Post by savoytruffle »

GeekyBear wrote: post #226
savoytruffle wrote: post #206

Earlier quoted context omitted.

I agree it was a bit worryingly short-lived. However the first version of Mac OS X that shipped without Rosetta 1 support was 10.7 Lion in summer 2011 (and many people avoided it since it was problematic). So nearly-modern Mac OS X with Rosetta support was realistic for a while longer.

> However the first version of Mac OS X that shipped without Rosetta 1 support was 10.7 Lion Yes, but I was pointing out when the last version of OS X that did support Rosetta shipped. I have no concrete evidence that Apple dropped Rosetta because IBM wanted to alter the terms of the deal after they bought Transitive, but I've always found that timing interesting. In comparison, the emulator used during the 68k to PP…

I agree. And I suppose since it was so intrinsic to the operating system, if a 68k app worked in Mac OS 9 (some would some might not), you could continue to run it in the Classic Environment (on a PPC Mac, not Intel Mac) Mac OS 10.4 Tiger in the mid 20 00's!
Post reply on HN