Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we challenge Microsoft’s battery test

opera.com

Post reply on HN
Print view View original

91–100 of 220 posts

Re: Why we challenge Microsoft’s battery test

#91

Post by kylealden »

Edge platform team employee here. A couple thoughts on methodology, focusing on our scripted test (under "stay productive longer" at https://blogs.windows.com/windowsexperience/2016/06/20/more-...), which is most comparable to Opera's test; nobody seems to challenge our Netflix results:

* We did not enable ad blockers, because we were testing the efficiency of the network stack, browser rendering pipeline, etc.; if you cut off ads, you're effectively skipping half the assignment. This is a valid way to improve the user experience, but not a valid way to measure browser efficiency, especially because it will disproportionately impact some sites (like the news sites that Opera's test focused on) and have no impact on other sites (like Netflix, where Edge demolishes the other browsers). It's basically skipping a lap of the race and then bragging that you finished faster.

* It's also worth noting that the ad blockers are often detected by many news sites and will actually prevent the main page content from loading at all. Not sure if Opera's test accounted for this.

* Our test was designed to mimic real-world behavior: Watching YouTube (foreground and background), shopping on Amazon, browsing the Facebook news feed, searching on Google, opening email in Gmail, and reading Wikipedia. To reduce variability, we used WebDriver to instrument the tests (supported by all four browsers tested) and made sure each task was timed rather than just a loop of consecutive tasks (which could disadvantage or advantage browsers depending on other factors like pageload performance or network conditions, in a way that doesn't reflect user experience, which is likely to linger on a page once it's loaded). We then used the Maxim 34407 power instrumentation built into the Surface Book (which is why we chose the Surface Book for this test) to measure actual instantaneous power consumption at the hardware, sampled once per second and then averaged across the duration of the test. We feel strongly that this is a highly scientific and defensible test setup which mimics typical user behavior and, significantly, measures the same markup and the same duration, on the same hardware, in every browser.

Re: Why we challenge Microsoft’s battery test

#92

Post by wernerb »

kylealden wrote: post #91

Edge platform team employee here. A couple thoughts on methodology, focusing on our scripted test (under "stay productive longer" at https://blogs.windows.com/windowsexperience/2016/06/20/more-... ), which is most comparable to Opera's test; nobody seems to challenge our Netflix results: * We did not enable ad blockers, because we were testing the efficiency of the network stack, browser rendering pipeline, etc.; if…

The way I read the blog is that the adblocking is enabled to show that even _with_ an expensive extension enabled it still beats Edge, but that was in my opinion not the gist of the post.

The post was about that microsoft is not transparent about the methodology/setup/scripts/target websites used. A third party should be able to support Microsoft's claims.

Is it possible for the setup to be published to your github.com/microsoft so that it may be executed automatically? Heck, go the extra mile put it in a CI and publish the data on regular basis :)

Re: Why we challenge Microsoft’s battery test

#93

Post by nachtigall »

Sylos wrote: post #86
nachtigall wrote: post #28

Where's firefox in the perf?

In the battery-benchmarks that I've seen so far, Firefox was always the leading browser, so granted those benchmarks did not test Opera's battery saving mode, Firefox should still be at least in second place. And Firefox does also have a native ad-blocker, so if you enable that, too, I could imagine that it maybe even still outperforms Opera's battery saving mode, but yeah, a proper benchmark would be good...

> Firefox does also have a native ad-blocker

I don't think it has one natively. Which one do you mean? Or are you referring to tracking protection (private mode only, but available in Fx 49 also in "normal" mode iirc)

Re: Why we challenge Microsoft’s battery test

#94

Post by outworlder »

Theodores wrote: post #12

Opera need to market themselves as the browser that extends your battery life so you can surf without the range anxiety. Testimonials are better than tests plus word of mouth.

This is the first time I've heard the term "range anxiety" applied to laptops, as opposed to electric vehicles.

Re: Why we challenge Microsoft’s battery test

#95

Post by IshKebab »

wernerb wrote: post #92
kylealden wrote: post #91

Edge platform team employee here. A couple thoughts on methodology, focusing on our scripted test (under "stay productive longer" at https://blogs.windows.com/windowsexperience/2016/06/20/more-... ), which is most comparable to Opera's test; nobody seems to challenge our Netflix results: * We did not enable ad blockers, because we were testing the efficiency of the network stack, browser rendering pipeline, etc.; if…

The way I read the blog is that the adblocking is enabled to show that even _with_ an expensive extension enabled it still beats Edge, but that was in my opinion not the gist of the post. The post was about that microsoft is not transparent about the methodology/setup/scripts/target websites used. A third party should be able to support Microsoft's claims. Is it possible for the setup to be published to your github.c…

Adblock isn't an "expensive extension". Ads are the expensive thing, so blocking them saves power.

Re: Why we challenge Microsoft’s battery test

#96

Post by eddieh »

Jerry2 wrote: post #90
callesgg wrote: post #67

I have found that safari gives best battery time on OSX (like 9 hours of use vs 6-7 when on firefox). But i generally dont use safari as battery time is not the most important factor in my common use cases.

>But i generally dont use safari as battery time is not the most important factor in my common use cases. I use Safari for almost everything. It's by far the best browser for browsing and just reading & enjoying the web. Reader interface and the speed of it is still unbeatable. I use Chrome for web development, however.

I agree Safari is the best. Every other browser is a clunker IMO. The Safari web inspector keeps getting better. You might want to take another look at it.

Re: Why we challenge Microsoft’s battery test

#97

Post by bialpio »

zeta0134 wrote: post #74
cptskippy wrote: post #49

Earlier quoted context omitted.

His point is that only one of the browsers was using an ad blocker so they were rendering different content because one rendered ads and the other did not.

Oh, I totally missed the part where Opera has the ad blocker turned on by default. Ignore my rambling then; my sleepy brain thought he was talking about two browsers that both had different ad blockers installed.

Not sure if Opera has it on by default - all they say is that it is build-in so that somehow makes it okay to use it and compare apples to oranges :)

If I understand both tests correctly, Opera ran with adblock and battery saver enabled (those are apparently not default settings), versus Edge without adblock, on sites which supposedly are ad-heavy - this will bias the results towards browser with adblock enabled.

Re: Why we challenge Microsoft’s battery test

#98

Post by wernerb »

IshKebab wrote: post #95
wernerb wrote: post #92

Earlier quoted context omitted.

The way I read the blog is that the adblocking is enabled to show that even _with_ an expensive extension enabled it still beats Edge, but that was in my opinion not the gist of the post. The post was about that microsoft is not transparent about the methodology/setup/scripts/target websites used. A third party should be able to support Microsoft's claims. Is it possible for the setup to be published to your github.c…

Adblock isn't an "expensive extension". Ads are the expensive thing, so blocking them saves power.

Until we know exactly what websites/videos etc are loaded (also on the Opera benchmark!); you, me and all consumers do not know if this is true :)

Re: Why we challenge Microsoft’s battery test

#99

Post by andromeduck »

z3t4 wrote: post #77

I wonder how much battery would be saved if not every site you visited was reported to bing api or opera sitecheck. But most power is is probably used to light the screen, so an extension to make every page use a black theme would probably beat most optimizations.

I don't think it would take much of a difference on an LCD display because your backlight is going to be on regardless.

Re: Why we challenge Microsoft’s battery test

#100

Post by blacksmith_tb »

adrianN wrote: post #11
blacksmith_tb wrote: post #7

Earlier quoted context omitted.

It wouldn't be too hard to dual-boot Win10 on Apple hardware and compare them, but it might be a little awkward for Microsoft to do so officially.

The OS is a pretty obvious confounding factor, don't you think?

It is, but with the Boot Camp drivers, it seems like a fair fight. Obviously both OSes do some things behind the scenes that impact battery life, but we hope both are tuned to conserve battery when not charging. Though to turn the argument around, given that ChromeOS isn't known for wasting a lot of battery while browsing with Chrome, having less OS is probably a boon.
Post reply on HN