Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Huawei caught cheating performance test for new phones

techcrunch.com

Post reply on HN
Print view View original

111–120 of 126 posts

Re: Huawei caught cheating performance test for new phones

#111

Post by foobaw »

maxerickson wrote: post #92
foobaw wrote: post #60

I used to work at an OEM that did this. When the VW Emissions Scandal was publicized, we removed almost every piece of code that did things like these from all our phones. Not sure if it was ever reverted eventually but performance test might not matter to HN users but it is something that typical consumers do look at when making purchasing decisions, based on our research.

Your second paragraph is completely ethically bankrupt. That a lie increases sales is at the top of the list of bad reasons to lie to customers (and they are all bad reasons).

I agree that it was unethical - practices like these are one of the major reasons I left.

It's nearly impossible to be "ethical" in large OEM companies because of the extreme competition and complexity of the organization. Most of these large OEMs are so political that even the smallest changes would require way too much effort.

Re: Huawei caught cheating performance test for new phones

#112

Post by kllrnohj »

MBCook wrote: post #105
kllrnohj wrote: post #33

Earlier quoted context omitted.

That's not an entirely accurate TLDR here as what ATI did back then is now considered standard practice for GPU drivers and is welcome. It is a bit different than the mobile space as GPU drivers are doing game-specific optimizations rather than benchmark-specific ones. The end user experience is a net win for the major titles, and as long as there's no visual cost it's considered fair game. It did go too far, even to…

> what ATI did back then is now considered standard practice for GPU drivers and is welcome. Yep. That was the problem at he time. There were no shaders/etc so you’d just taking a demanding game and see how good it ran. No one optimized specific games in their drivers. So what ATI did made it appear the GPU was stronger on average that it was. If it was disclosed it would have been a feature and might have been welco…

> So what ATI did made it appear the GPU was stronger on average that it was.

Well, not really. It really did run Quake 3 at the speed the benchmarks were reporting. The problem was it got that speed at the expense of a drop in image quality, and a somewhat substantial one at that.

But if you launched the game and just played it you did get the exact same performance that the benchmarks were reporting.

> I remember seeing discussions about whether the kind of optimizations we see today are cheating. They sort of are. But why wouldn’t I want the faster frame rates?

Desktop GPU optimizations or mobile ones? Desktop GPU optimizations these days are not considered cheating because of 2 key points: 1) it improves the actual game experience (this is why reviews focus on game performance now) and 2) it doesn't impact image quality (periodically you'll see image quality comparisons to ensure both sides are still being honest)

The mobile things that Huawei are doing though are cheating because it does not translate into improvements in actual user experience or gameplay. It's tuning the thermal & power consumption to hit the throttling point at the exact duration of the benchmark. This doesn't translate in any way to any real-world improvements. Thus, cheating.

Re: Huawei caught cheating performance test for new phones

#113

Post by kllrnohj »

drampelt wrote: post #58
EZ-E wrote: post #9

I'm gonna get down voted but that's why I buy Apple phones. Comparing processors, ram, test scores means nothing compared to the actual experience

And that's why I buy Google phones. The processor being slightly slower doesn't matter to me if I prefer the overall experience

Google phones tend to use whatever the current high end SoC is. So when you see the same SoC scoring lower in a Google phone than on everyone else's phone is it really that the processor is slower, or is it just that Google/AOSP doesn't engage in benchmark detection and you're seeing the true performance of the SoC as a result?

Re: Huawei caught cheating performance test for new phones

#114

Post by bri3d »

hjfa0j wrote: post #65
bri3d wrote: post #38

Earlier quoted context omitted.

Shader replacement was extremely popular for years - see AMD's infamous Doom3 shader replacement: https://www.anandtech.com/show/1488/6 . NVidia drivers did the same thing although they were (generally) a bit smarter about it, relying on shader fingerprinting rather than application fingerprinting to prevent embarrassing "rename the EXE" expose articles. They were notoriously caught using a benchmark's camera path as…

There's absolutely nothing wrong with shader replacement. Quite the opposite, as it shows the OEM cares about performance. They have two options: convince the game developer to use a better optimised shader or just replace it themselves. The latter is obviously better.

The latter is obviously better? Shader replacement is fairly fragile and circumvents whatever QA process the game developers had in place - if the shader could be incorporated into the actual product I would think it would be better than shipping thousands of patches.

However, I agree that shader replacement is great for the consumer especially in situations where developers have agreed to vendor lock-in/specialization agreements. It's only questionable to me in the sense of benchmarking applications.

Re: Huawei caught cheating performance test for new phones

#115

Post by jackconnor »

briandear wrote: post #45
ekianjo wrote: post #28

Earlier quoted context omitted.

Apple does the same thing when they introduce new models. They pretend it runs x times faster than the previous flagship one. Same thing really.

They actually do run x times faster. They almost always have faster chips than the previous model. So they’ll pretty much always be faster. Who would release a slower phone as a new model? It doesn’t make any sense.

The point is that it may be such a minor difference that most consumers would never know the difference, not that it's worse.

Re: Huawei caught cheating performance test for new phones

#116

Post by maxerickson »

orf wrote: post #108
maxerickson wrote: post #92

Earlier quoted context omitted.

Your second paragraph is completely ethically bankrupt. That a lie increases sales is at the top of the list of bad reasons to lie to customers (and they are all bad reasons).

> That a lie increases sales is at the top of the list of bad reasons to lie to customers I think that in the society we've built it's actually one of the best reasons to lie to customers. Advertising is pretty much built on that.

Replying to that piece of text out of context is really boring.

Re: Huawei caught cheating performance test for new phones

#117

Post by jjeaff »

dingaling wrote: post #77
PinkMilkshake wrote: post #72

Earlier quoted context omitted.

> Have you seen the rig Apple uses? I cant find this, do you have a link? I'm quite interested in what they do.

They mount DSLR lenses onto stripped-down iPhone chassis via custom adapters. So it demonstrates what the Sony sensor + Apple software can do, but isn't reproducible in the real World.

Actually doesn't even showcase the apple software. The disclaimer in their own and says additional software is used. So I assume they are postprocessing with color correction and who knows what else.

Re: Huawei caught cheating performance test for new phones

#118

Post by culturestate »

mcphage wrote: post #110
culturestate wrote: post #31

Earlier quoted context omitted.

Apple has always tailored their marketing to whatever their advantage is at that moment. If they have the best tech, they'll talk about speeds and feeds - if they don't, they'll talk about the experience or some other aspect. This is almost as reliable a tick-tock cycle as Intel's.

> Apple has always tailored their marketing to whatever their advantage is at that moment. What's Option B? Advertise based on what they're not good at? "Snapple: Made from the Fifth or Sixth Best Stuff on Earth"

I'm just noting for GP that this has been SOP at Apple for a long time, not something new.

Re: Huawei caught cheating performance test for new phones

#119

Post by mcphage »

culturestate wrote: post #118
mcphage wrote: post #110

Earlier quoted context omitted.

> Apple has always tailored their marketing to whatever their advantage is at that moment. What's Option B? Advertise based on what they're not good at? "Snapple: Made from the Fifth or Sixth Best Stuff on Earth"

I'm just noting for GP that this has been SOP at Apple for a long time, not something new.

Of course it is. It’s the only useful thing you can do when advertising.

Re: Huawei caught cheating performance test for new phones

#120

Post by orf »

maxerickson wrote: post #116
orf wrote: post #108

Earlier quoted context omitted.

> That a lie increases sales is at the top of the list of bad reasons to lie to customers I think that in the society we've built it's actually one of the best reasons to lie to customers. Advertising is pretty much built on that.

Replying to that piece of text out of context is really boring.

I'm not sure how it's out of context, your entire comment was "lying to customers to increase sales is bad!".
Post reply on HN