Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel publishes misleading benchmarks against AMD

servethehome.com

Post reply on HN
Print view View original

41–50 of 209 posts

Re: Intel publishes misleading benchmarks against AMD

#41

Post by gatherhunterer »

pinewurst wrote: post #30

There's often a long lead time between writing up benchmarks/white papers and publication. At my last employer it could take well over a month to let everyone have their 2 cents and get through Legal for approval. This is a long enough interval to explain the old/new GROMACS usage. As the saying goes, don't attribute to malice that which can be adequately explained by stupidity (including corporate bureaucracy).

The author pointed out that they published benchmarks using the same test that had not been optimized for AMD yet. But they included that information in the publication, including on the graphs that would otherwise appear to show the outright superiority of Intel in that regard.

If the publication was delayed to allow time for due diligence then this disclosure should have made the cut. I like to think that Intel had some top-notch benchmarking nerds who were superseded by unscrupulous executives and would not have published a dishonest benchmark otherwise.

Re: Intel publishes misleading benchmarks against AMD

#42

Post by Vindicis »

pinewurst wrote: post #37
Vindicis wrote: post #34

Earlier quoted context omitted.

Unless that company is well-noted for their malice.

That's not to deny the possibility of both. :)

Haha true enough. I'm sure the truth in this case in somewhere in the middle. Reading any marketing material should always be done with a healthy dose of skepticism.

Re: Intel publishes misleading benchmarks against AMD

#43

Post by agumonkey »

xenadu02 wrote: post #5
bryant wrote: post #3

tl;dr: using outdated benchmarking tools optimized for the latest Intel chips and not the latest AMD chips. GROMACS 2019.3 v. 2019.4, the latter of which is the latest but still came out over a month ago and addressed AMD Zen 2. Source is still a good read for explanations on why this is impactful. --- Disclosure: I'm long AMD

They used the Intel compiler. We know for a fact Intel’s compilers deliberately don’t optimize as well for non-Intel chips. That’s damning in and of itself.

It's been 30 years since benchmarks used icc .. it was already shocking when we realized that in the 2Ks.

Re: Intel publishes misleading benchmarks against AMD

#44

Post by stagger87 »

Tepix wrote: post #25
sp332 wrote: post #16

Earlier quoted context omitted.

Many applications don't need Spectre or Meltdown mitigations, but they should at least put the test parameters up front and not in the fine print.

Many users won't know how to disable Spectre and Meltdown mitigations. They will be active by default if the user installs all the latest updates.

For some reason, I doubt people in the server/workstation space don't know how to do this, especially if they are leaving double digit percentage gains on the table if they don't.

Re: Intel publishes misleading benchmarks against AMD

#45

Post by paulmd »

pinewurst wrote: post #30

There's often a long lead time between writing up benchmarks/white papers and publication. At my last employer it could take well over a month to let everyone have their 2 cents and get through Legal for approval. This is a long enough interval to explain the old/new GROMACS usage. As the saying goes, don't attribute to malice that which can be adequately explained by stupidity (including corporate bureaucracy).

Also, the difference in sub-NUMA configuration actually favors AMD as well. NPS=4 is optimal for AMD, as shown by ServeTheHome's own work earlier. Both systems are tested in their proper (fastest) configuration.

Despite ServeTheHome's own previous work showing this, they're whining about it being different. But if Intel hadn't tested AMD in the proper configuration they would have complained about that too.

Yeah, you take first-party benchmarks with a big grain of salt like always, but STH is just looking to stir some shit here.

Re: Intel publishes misleading benchmarks against AMD

#46

Post by gpm »

sp332 wrote: post #16
Jonnax wrote: post #4

I recall a few months ago they released a marketing pack benchmarking themselves against AMD with a small print disclaimer that they hadn't applied the spectre/meltdown fixes.

Many applications don't need Spectre or Meltdown mitigations, but they should at least put the test parameters up front and not in the fine print.

Many?

Anything running a browser needs it because of js.

Anything running connected to the internet at all would be well advised to have them because you can take advantage of the exploits via the network stack.

Anything running on a vm has to have them since microcode is set by the hypervisor.

So, we're talking about desktops not used to browse the internet, and bare metal servers whose network stacks aren't exposed to hostile actors?

Re: Intel publishes misleading benchmarks against AMD

#47

Post by monocasa »

dbcurtis wrote: post #22
xenadu02 wrote: post #5

Earlier quoted context omitted.

They used the Intel compiler. We know for a fact Intel’s compilers deliberately don’t optimize as well for non-Intel chips. That’s damning in and of itself.

That is a little disingenuous. The whole point of Intel’s compiler is to support Intel chips well. Results on other chips are what they are. No effort to make other chips perform eiher better or worse is expended. Keeping up with Intel chips is more than a full time job without other distractions. The question to ask is whether or not they used the best complier/swithes available for the other chips.

Intel is the one who defined feature presence flags in the CPU, but then doesn't use them for their compiler.

Re: Intel publishes misleading benchmarks against AMD

#48

Post by computerex »

sp332 wrote: post #16
Jonnax wrote: post #4

I recall a few months ago they released a marketing pack benchmarking themselves against AMD with a small print disclaimer that they hadn't applied the spectre/meltdown fixes.

Many applications don't need Spectre or Meltdown mitigations, but they should at least put the test parameters up front and not in the fine print.

You absolutely need Spectre/Meltdown mitigation otherwise someone could attack you with JavaScript running in your web browser.

https://linuxreviews.org/HOWTO_make_Linux_run_blazing_fast_(...

   noibrs - We don't need no restricted indirect branch speculation
   noibpb - We don't need no indirect branch prediction barrier either
   nospectre_v1 and nospectre_v2: Don't care if some program can get data from some other program when it shouldn't
   l1tf=off - Why would we be flushing the L1 cache, we might need that data. So what if anyone can get at it.
   nospec_store_bypass_disable - Of course we want to use, not bypass, the stored data
   no_stf_barrier - We don't need no barriers between software, they could be friends
   mds=off - Zombieload attacks are fine
   mitigations=off - Of course we don't want no mitigations

Re: Intel publishes misleading benchmarks against AMD

#50

Post by blurbleblurble »

NicoJuicy wrote: post #39

They wouldn't have an answer to AMD untill 2021: https://newsroom.intel.com/news/2019-intel-investor-meeting/... Ps. Long AMD, perfect execution the last two years. It's crazy, even their late firmware fixes improve performance as it had bad press for a moment for false benchmarks advertisement ( it was not false so it seems, the firmware was just not ok yet. Not everything is fully ok now, it's the only minor nitpic…

I agree. Also, good luck to AMD and Intel both as the next few iterations of ARM stuff comes out and people start porting software over to that! It's looking like ARM will be taking quite a jump.
Post reply on HN