Live data from Hacker News

Huawei caught cheating performance test for new phones

techcrunch.com

71–80 of 126 posts

Re: Huawei caught cheating performance test for new phones

#71
post #62

It reminds me of the old GPU vendors when we were still using AGP bus(or pre-AGP era I forgot). Good evil days.

Are those days gone? Benchmark-specific hacks in graphics drivers, I mean, not AGP. :P

Also all the outrage when people discovered that a Radeon 9600 was just a Radeon 9800 running in crippled mode, and that you could bridge the tracks and re-enable it in 9800 mode. (Or something similar, anyway, that was a long time ago... :P )

Re: Huawei caught cheating performance test for new phones

#72
post #36

This comes as no surprise. This is a company who has also been caught claiming their phone took a photo - but left in DSLR exif data, claiming a phone had UFS but shipped EMMC instead, boasting to work with dev community - only to lock them out months later, among other shady practices. They make nice phones at a good price, but is not a company to be trusted.

I am not surprised as others have been cought doing the same. Anyone remember the DSLR in the reflection in a Nokia ad that was supposedly shot a Nokia? Have you seen the rig Apple uses? doesn't look much like a phone anymore but technically is still an iPhone. Sadly the P20 pro has an incredible camera and would stand on its own without forgery. All I really care about is that they unlock their bootloader. The marke…

> Have you seen the rig Apple uses?

I cant find this, do you have a link? I'm quite interested in what they do.

Re: Huawei caught cheating performance test for new phones

#73
post #28

Earlier quoted context omitted.

Apple does the same thing when they introduce new models. They pretend it runs x times faster than the previous flagship one. Same thing really.

They actually do run x times faster. They almost always have faster chips than the previous model. So they’ll pretty much always be faster. Who would release a slower phone as a new model? It doesn’t make any sense.

point is its not as fast as they claim it to be, because they take edge cases as always. I never said the new models are slower, no need to strawman my argument.

Re: Huawei caught cheating performance test for new phones

#74

Earlier quoted context omitted.

I am not surprised as others have been cought doing the same. Anyone remember the DSLR in the reflection in a Nokia ad that was supposedly shot a Nokia? Have you seen the rig Apple uses? doesn't look much like a phone anymore but technically is still an iPhone. Sadly the P20 pro has an incredible camera and would stand on its own without forgery. All I really care about is that they unlock their bootloader. The marke…

> Have you seen the rig Apple uses? I cant find this, do you have a link? I'm quite interested in what they do.

Apparently at least specialized lenses and stabilizers. https://petapixel.com/2017/06/30/truth-shot-iphone-style-ads...

Re: Huawei caught cheating performance test for new phones

#75

Earlier quoted context omitted.

I am not surprised as others have been cought doing the same. Anyone remember the DSLR in the reflection in a Nokia ad that was supposedly shot a Nokia? Have you seen the rig Apple uses? doesn't look much like a phone anymore but technically is still an iPhone. Sadly the P20 pro has an incredible camera and would stand on its own without forgery. All I really care about is that they unlock their bootloader. The marke…

> Have you seen the rig Apple uses? I cant find this, do you have a link? I'm quite interested in what they do.

Well there is this:

https://youtu.be/OkPter7MC1I

Re: Huawei caught cheating performance test for new phones

#77

Earlier quoted context omitted.

I am not surprised as others have been cought doing the same. Anyone remember the DSLR in the reflection in a Nokia ad that was supposedly shot a Nokia? Have you seen the rig Apple uses? doesn't look much like a phone anymore but technically is still an iPhone. Sadly the P20 pro has an incredible camera and would stand on its own without forgery. All I really care about is that they unlock their bootloader. The marke…

> Have you seen the rig Apple uses? I cant find this, do you have a link? I'm quite interested in what they do.

They mount DSLR lenses onto stripped-down iPhone chassis via custom adapters. So it demonstrates what the Sony sensor + Apple software can do, but isn't reproducible in the real World.

Re: Huawei caught cheating performance test for new phones

#78
post #38
post #12

Earlier quoted context omitted.

Desktops have gone through it too: quack.exe and the HardOCP exposé. It’s been a long time since I thought of that.

Shader replacement was extremely popular for years - see AMD's infamous Doom3 shader replacement: https://www.anandtech.com/show/1488/6 . NVidia drivers did the same thing although they were (generally) a bit smarter about it, relying on shader fingerprinting rather than application fingerprinting to prevent embarrassing "rename the EXE" expose articles. They were notoriously caught using a benchmark's camera path as…

Doom 3 was selling title for Nvidia back in the days. Carmack and Co. didnt wanted to do any optimizations for ATI cards because it wont favor Nvidia cards and they had a contract with them back then.

Re: Huawei caught cheating performance test for new phones

#79
post #71
post #62

It reminds me of the old GPU vendors when we were still using AGP bus(or pre-AGP era I forgot). Good evil days.

Are those days gone? Benchmark-specific hacks in graphics drivers, I mean, not AGP. :P Also all the outrage when people discovered that a Radeon 9600 was just a Radeon 9800 running in crippled mode, and that you could bridge the tracks and re-enable it in 9800 mode. (Or something similar, anyway, that was a long time ago... :P )

It was the 9500 that could be modded into a 9700. Early versions could be softmodded, this was disabled in later revisions which could only be modded by soldering the additional rendering pipes back on. No idea why I remember this useless piece of information.

Also on a completely different note, the 9700 Pro which was the flagship card of that generation, cost about $300. Times have changed.

Re: Huawei caught cheating performance test for new phones

#80
post #4

I don't see why an earnest benchmarking firm would let manufacturers know details like the name of the testing app.

Have you ever thought about what the business model is for benchmark companies?

It probably won't come as a surprise to many of you, but there's a major conflict of interest when it comes to benchmark companies. They have "benchmark development programs" which companies pay tens or hundreds of thousands of dollars to join, and in turn the member companies get early access to beta versions of the benchmarks, and get to propose optimizations to the benchmarks.

Now the benchmark companies insist that they make impartial decisions when it comes to what to include or improve in the benchmarks, but you have to wonder just how much influence these member companies have over the benchmark companies, especially when it comes to new features that are in one but not all devices, or features that are not often used by developers but have different performance on different devices, what the decision process is to include tests for those features.

Post reply on HN