Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Huawei caught cheating performance test for new phones

techcrunch.com

Post reply on HN
Print view View original

41–50 of 126 posts

Re: Huawei caught cheating performance test for new phones

#41

Post by bri3d »

MBCook wrote: post #19
remlov wrote: post #11

Earlier quoted context omitted.

Reminds me of the time way back where Futuremark confirmed Nvidia was cheating on synthetic benchmarks by doing something similar except in reverse where they would degrade texture quality and use other tricks to artificially inflate scores in 3dmark. I'm certain ATI was caught doing so as well at some point.

Yep. I believe ATI was the one where if you renamed quake.exe to quack.exe it suddenly performed significantly worse.

Shader replacement and fingerprinting is par for the course in the GPU industry. The ethics are arguable when it comes to applications that aren't benchmarks - is it really wrong for the vendor to apply post-facto optimization to give end-users a better experience? Certainly, though, in the case of benchmarks it's a slimy practice.

Re: Huawei caught cheating performance test for new phones

#42

Post by barrkel »

anyfoo wrote: post #39
userbinator wrote: post #8

Earlier quoted context omitted.

What exactly does "adjust performance" even mean? If it's something like frequency throttling, there should be a user setting to force maximums, even at the expense of battery life, and likewise another option to force minimums.

If you enjoy a phone that runs hot, but only for a few minutes, depending on whether an empty battery or thermal protection kicks in first, sure. Power management and thermal budgets are a very real thing. The times where all device’s components would run at full speed are (thankfully) over for quite some time. 20 years ago PCs had multiple incredibly loud constantly running fans, that I think hardly anyone would put…

20 years ago, PCs had much smaller fans than they do today, and the cases usually had no separate fans of their own - the only fans you'd find would be in the power supply and on top of the CPU. And passive cooling for the CPU wasn't difficult to achieve with a big heatsink.

The industry didn't focus much on quietness, that's true.

(I used to assemble and sell custom-built PCs around 1996 to 2000.)

Re: Huawei caught cheating performance test for new phones

#44

Post by anyfoo »

barrkel wrote: post #42
anyfoo wrote: post #39

Earlier quoted context omitted.

If you enjoy a phone that runs hot, but only for a few minutes, depending on whether an empty battery or thermal protection kicks in first, sure. Power management and thermal budgets are a very real thing. The times where all device’s components would run at full speed are (thankfully) over for quite some time. 20 years ago PCs had multiple incredibly loud constantly running fans, that I think hardly anyone would put…

20 years ago, PCs had much smaller fans than they do today, and the cases usually had no separate fans of their own - the only fans you'd find would be in the power supply and on top of the CPU. And passive cooling for the CPU wasn't difficult to achieve with a big heatsink. The industry didn't focus much on quietness, that's true. (I used to assemble and sell custom-built PCs around 1996 to 2000.)

1.4GHz Athlons were a thing around 2000, almost 20 years ago, and had anything but "small fans" to be cooled. I used to build PCs too, and once I pretty much instantly burned up the die of one of those Athlons because I forgot to apply the heat paste.

Even with computers that could be run all passively at that time, there is just no comparison to today's smartphones, which sit in our pockets and are orders of magnitude more capable along almost every axis.

Re: Huawei caught cheating performance test for new phones

#45

Post by briandear »

ekianjo wrote: post #28
jackconnor wrote: post #13

Earlier quoted context omitted.

Yea I kind of agree. I think most people, myself included, just don't really recognize minor performance differences, only relatively large ones. And I've always thought that these things improve so fast year-to-year, that it's kind of pointless to try and keep up. So, i shoot for a phone I like that performs pretty well, and just leave it at that (hello iPhone SE).

Apple does the same thing when they introduce new models. They pretend it runs x times faster than the previous flagship one. Same thing really.

They actually do run x times faster. They almost always have faster chips than the previous model. So they’ll pretty much always be faster. Who would release a slower phone as a new model? It doesn’t make any sense.

Re: Huawei caught cheating performance test for new phones

#47

Post by jpatokal »

ekianjo wrote: post #30
ali_af wrote: post #22

Earlier quoted context omitted.

People who are really serious about software should make their own hardware I never get tired of this quote, and I will always point to it when people compare iOS/Android or macOS/Windows.

By your logic then Apple does not seem to take much about macOS and their latest pro offering then.

Many would argue that the logic is correct then.

Re: Huawei caught cheating performance test for new phones

#48

Post by on_and_off »

Huawei caught cheating at a meaningless 'test' nobody should care about.

I spoke a fair bit about performances with a Googler in charger of it. Apparently he is appalled that OEMs have been optimizing their phones in order to score high on these benchmarks.

They don't reflect real use at all and are not what should be optimized against.

In the end, he documented the work he did on the various terminals he worked on so OEMs could do what they should have been doing for years..

Re: Huawei caught cheating performance test for new phones

#49

Post by 013a »

Jerry2 wrote: post #15
vvanders wrote: post #7

Earlier quoted context omitted.

As someone who spent a ton of time in the smartphone GPU/CPU space from '09-'13 we saw every single vendor do this. They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps. We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also l…

> we saw every single vendor do this Except Apple.

Probably because they don't need to. Their mobile chips are so ludicrously far ahead of everyone else in the world, it seems possible they discovered a crashed UFO and integrated alien technology into their designs.

Re: Huawei caught cheating performance test for new phones

#50

Post by aslkdjaslkdj1 »

Someone1234 wrote: post #21
EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

> Comparing processors, ram, test scores means nothing compared to the actual experience But Apple brags endlessly about their A11 "Bionic" chip and similar[0]. Arguing that Apple values experience over specs might have been a valid argument back with Steven Jobs, but Apple's presentations spend significant amounts of time talking about hardware specs these days. I don't specifically mind that Apple does, but people…

The A11 chip's CV module is the reason FaceID works as well as it does. And FaceID was their one of their major marketing points of the X and likely the phones being announced this month.
Post reply on HN