Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Huawei caught cheating performance test for new phones

techcrunch.com

Post reply on HN
Print view View original

21–30 of 126 posts

Re: Huawei caught cheating performance test for new phones

#21

Post by Someone1234 »

EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

> Comparing processors, ram, test scores means nothing compared to the actual experience

But Apple brags endlessly about their A11 "Bionic" chip and similar[0].

Arguing that Apple values experience over specs might have been a valid argument back with Steven Jobs, but Apple's presentations spend significant amounts of time talking about hardware specs these days.

I don't specifically mind that Apple does, but people need to stop pretending like Apple hasn't changed as a company with the shift in leadership.

[0] https://youtu.be/M4Pc1eQtWmQ?t=11m43s

Re: Huawei caught cheating performance test for new phones

#22

Post by ali_af »

EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

  People who are really serious about software should make their own hardware
I never get tired of this quote, and I will always point to it when people compare iOS/Android or macOS/Windows.

Re: Huawei caught cheating performance test for new phones

#23

Post by vvanders »

Jerry2 wrote: post #15
vvanders wrote: post #7

Earlier quoted context omitted.

As someone who spent a ton of time in the smartphone GPU/CPU space from '09-'13 we saw every single vendor do this. They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps. We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also l…

> we saw every single vendor do this Except Apple.

So PowerVR is the only GPU design I don't have direct experience with in the mobile space, never worked with an SoC that fielded it.

However given that Apple isn't the only one who licenses their designs I'd put a large sum of money on them doing the same thing.

Re: Huawei caught cheating performance test for new phones

#24

Post by StudentStuff »

Jerry2 wrote: post #15
vvanders wrote: post #7

Earlier quoted context omitted.

As someone who spent a ton of time in the smartphone GPU/CPU space from '09-'13 we saw every single vendor do this. They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps. We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also l…

> we saw every single vendor do this Except Apple.

Apple plays this game, but in a different way. They know that apps like Geekbench test for only a short duration which won't necessarily hit their thermal throttling if the phone is cool enough. Detecting 100% load for a few dozen seconds and giving it full performance isn't hard, that is what Huawei and other vendors should have been doing.

Issues like the Intel i9 performance going significantly downward with consistent usage are often missed by benchmarking apps: https://arstechnica.com/gadgets/2018/07/we-tested-throttling...

Re: Huawei caught cheating performance test for new phones

#26

Post by jakevn »

StudentStuff wrote: post #24
Jerry2 wrote: post #15

Earlier quoted context omitted.

> we saw every single vendor do this Except Apple.

Apple plays this game, but in a different way. They know that apps like Geekbench test for only a short duration which won't necessarily hit their thermal throttling if the phone is cool enough. Detecting 100% load for a few dozen seconds and giving it full performance isn't hard, that is what Huawei and other vendors should have been doing. Issues like the Intel i9 performance going significantly downward with consi…

Are you implying that short burst performance is for gaming benchmarks?

Re: Huawei caught cheating performance test for new phones

#27

Post by ekianjo »

MBCook wrote: post #12
vvanders wrote: post #7

Earlier quoted context omitted.

As someone who spent a ton of time in the smartphone GPU/CPU space from '09-'13 we saw every single vendor do this. They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps. We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also l…

Desktops have gone through it too: quack.exe and the HardOCP exposé. It’s been a long time since I thought of that.

for benchmarks to work against such practices they need to be changed every single time, in a non predictable way.

Re: Huawei caught cheating performance test for new phones

#28

Post by ekianjo »

jackconnor wrote: post #13
EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

Yea I kind of agree. I think most people, myself included, just don't really recognize minor performance differences, only relatively large ones. And I've always thought that these things improve so fast year-to-year, that it's kind of pointless to try and keep up. So, i shoot for a phone I like that performs pretty well, and just leave it at that (hello iPhone SE).

Apple does the same thing when they introduce new models. They pretend it runs x times faster than the previous flagship one. Same thing really.

Re: Huawei caught cheating performance test for new phones

#29

Post by StudentStuff »

jakevn wrote: post #26
StudentStuff wrote: post #24

Earlier quoted context omitted.

Apple plays this game, but in a different way. They know that apps like Geekbench test for only a short duration which won't necessarily hit their thermal throttling if the phone is cool enough. Detecting 100% load for a few dozen seconds and giving it full performance isn't hard, that is what Huawei and other vendors should have been doing. Issues like the Intel i9 performance going significantly downward with consi…

Are you implying that short burst performance is for gaming benchmarks?

Nope, not implying that at all. For some games that may make sense, but I think that'd fall under the sustained heavy CPU/GPU usage category for most games. Same for compiling large apps, interactive HTML5 PWAs and such.

Re: Huawei caught cheating performance test for new phones

#30

Post by ekianjo »

ali_af wrote: post #22
EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

People who are really serious about software should make their own hardware I never get tired of this quote, and I will always point to it when people compare iOS/Android or macOS/Windows.

By your logic then Apple does not seem to take much about macOS and their latest pro offering then.
Post reply on HN