Huawei caught cheating performance test for new phones • Hacker News Board

Re: Huawei caught cheating performance test for new phones

#51

Post by mtgx » Fri, Sep 07, 2018, 4:46 AM UTC

StudentStuff wrote: post #24

Jerry2 wrote: post #15

Earlier quoted context omitted.

> we saw every single vendor do this Except Apple.

Apple plays this game, but in a different way. They know that apps like Geekbench test for only a short duration which won't necessarily hit their thermal throttling if the phone is cool enough. Detecting 100% load for a few dozen seconds and giving it full performance isn't hard, that is what Huawei and other vendors should have been doing. Issues like the Intel i9 performance going significantly downward with consi…

Intel has been doing this for ages. It may even be the primary reason why they came up with Turbo Boost. It's gotten a lot worse these days, as Turbo Boost can add another Ghz on its own to the base clock, which allows Intel to keep base clocks very low, yet still claim its chips are as fast as if they always run at the peak Turbo Boost speed all the time (and that turbo boost running time has been shrinking every new generation, too).

Re: Huawei caught cheating performance test for new phones

#52

Post by make3 » Fri, Sep 07, 2018, 4:58 AM UTC

013a wrote: post #49

Jerry2 wrote: post #15

Earlier quoted context omitted.

> we saw every single vendor do this Except Apple.

Probably because they don't need to. Their mobile chips are so ludicrously far ahead of everyone else in the world, it seems possible they discovered a crashed UFO and integrated alien technology into their designs.

source?

Re: Huawei caught cheating performance test for new phones

#53

Post by samsonradu » Fri, Sep 07, 2018, 5:04 AM UTC

lysp wrote: post #3

> For the Huawei case, the rules are actually a little fuzzy. Phones are permitted to adjust performance based on workload, which results in peaks or dips in performance for different apps, but they are not permitted to hard-code peaks in performance specifically for the benchmark app. Huawei reportedly claimed that the peak in performance seen during the run of the benchmark app was an intuitive jump determined by A…

Looks a lot like the Volkswagen’s emission software /s

Re: Huawei caught cheating performance test for new phones

#54

Post by [deleted] » Fri, Sep 07, 2018, 5:05 AM UTC

Jerry2 wrote: post #15

vvanders wrote: post #7

Earlier quoted context omitted.

As someone who spent a ton of time in the smartphone GPU/CPU space from '09-'13 we saw every single vendor do this. They'll happily ignore thermals, run custom shader microcode and any other tricks that'll boost their benchmark scores on common apps. We always kept a set of internal benchmarks that we didn't share with vendors and used that as a part of SoC evaluation. It had the benefit of not being gamed and also l…

> we saw every single vendor do this Except Apple.

[deleted]

Re: Huawei caught cheating performance test for new phones

#55

Post by yjftsjthsd-h » Fri, Sep 07, 2018, 5:05 AM UTC

briandear wrote: post #45

ekianjo wrote: post #28

Earlier quoted context omitted.

Apple does the same thing when they introduce new models. They pretend it runs x times faster than the previous flagship one. Same thing really.

They actually do run x times faster. They almost always have faster chips than the previous model. So they’ll pretty much always be faster. Who would release a slower phone as a new model? It doesn’t make any sense.

Maybe if it tripled battery or something

Re: Huawei caught cheating performance test for new phones

#56

Post by yjftsjthsd-h » Fri, Sep 07, 2018, 5:07 AM UTC

chrisper wrote: post #25

That's a shame. I got a P20 Pro and I really like it (even the software). But I don't think I'll get another Huawei next time, because the company itself is really weird.

Using "weird" as a euphemism for unethical, or some other problem?

Re: Huawei caught cheating performance test for new phones

#57

Post by fapjacks » Fri, Sep 07, 2018, 5:17 AM UTC

Yeah, Huawei also sells shitty phones with locked bootloaders. Not interested in any case, cheating or no.

Re: Huawei caught cheating performance test for new phones

#58

Post by drampelt » Fri, Sep 07, 2018, 5:23 AM UTC

EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

And that's why I buy Google phones. The processor being slightly slower doesn't matter to me if I prefer the overall experience

Re: Huawei caught cheating performance test for new phones

#59

Post by lostlogin » Fri, Sep 07, 2018, 6:09 AM UTC

yjftsjthsd-h wrote: post #55

briandear wrote: post #45

Earlier quoted context omitted.

They actually do run x times faster. They almost always have faster chips than the previous model. So they’ll pretty much always be faster. Who would release a slower phone as a new model? It doesn’t make any sense.

Maybe if it tripled battery or something

Can I have this please. It can be thicker and heavier if that helps.

Re: Huawei caught cheating performance test for new phones

#60

Post by foobaw » Fri, Sep 07, 2018, 6:19 AM UTC

I used to work at an OEM that did this. When the VW Emissions Scandal was publicized, we removed almost every piece of code that did things like these from all our phones.

Not sure if it was ever reverted eventually but performance test might not matter to HN users but it is something that typical consumers do look at when making purchasing decisions, based on our research.