Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Speedometer 3.0: A shared browser benchmark for web application responsiveness

browserbench.org

Post reply on HN
Print view View original

41–50 of 183 posts

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#41

Post by capitainenemo »

wswope wrote: post #36
tdudhhu wrote: post #25

On Firefox mobile I got a score of 3. So while Firefox is now super fast the performance of webapps might still be very bad on mobile. And I think this applies to all modern browsers: they are fast at rendering very slow webapps and websites.

If you’re on iOS, Apple gates Firefox from using JIT JS compilation which massively hinders performance. E: I was wrong/extremely out-of-date - it does have JIT but relies on the Safari/Webkit implementation. In ancient versions of iOS, the WebView widget that third-party browsers were forced to use had JIT disabled, but that’s long since changed.

It's more than that right? They have to use all of webkit. So it's pretty much a reskinned Safari in terms of layout/rendering/JS.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#42

Post by Timon3 »

dylan604 wrote: post #32

[flagged]

When that metric is "performance in real workloads", I can't imagine it ever becoming irrelevant. Just look at their new tests:

> In particular, we added new tests that simulate rendering canvas and SVG charts (React Stockcharts, Chart.js, Perf Dashboard, and Observable Plot), code editing (CodeMirror), WYSIWYG editing (TipTap), and reading news sites (Next.js and Nuxt.js).

> We’ve also improved the TodoMVC tests: updating the code to adapt to the most common versions of the most popular frameworks based on data from the the HTTP Archive. The following frameworks and libraries are included: Angular, Backbone, jQuery, Lit, Preact, React, React+Redux, Svelte, and Vue; along with vanilla JavaScript implementations targeting ES5 and ES6, and a Web Components version. We also introduced more complex versions of these tests which are embedded into a bigger DOM tree with many complex CSS rules that more closely emulate the page weight and structure from popular webapps today.

Improving these benchmark results will at least partially make those libraries faster in the real world, and most likely also many additional libraries and workloads.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#43

Post by om2 »

wswope wrote: post #36
tdudhhu wrote: post #25

On Firefox mobile I got a score of 3. So while Firefox is now super fast the performance of webapps might still be very bad on mobile. And I think this applies to all modern browsers: they are fast at rendering very slow webapps and websites.

If you’re on iOS, Apple gates Firefox from using JIT JS compilation which massively hinders performance. E: I was wrong/extremely out-of-date - it does have JIT but relies on the Safari/Webkit implementation. In ancient versions of iOS, the WebView widget that third-party browsers were forced to use had JIT disabled, but that’s long since changed.

That’s not accurate. Firefox and all third party WebKit apps get the same JOT as Safari.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#44

Post by Vinnl »

lapcat wrote: post #31
treyd wrote: post #29

Earlier quoted context omitted.

> On the other hand, premature optimization is the root of all evil. The web ecosystem is extremely mature and widely used. The workloads are fairly well understood. It is a magic unit, but the factors that go into it have a lot of thought from real-world scenarios. Bringing up "premature optimization" is completely irrelevant because that's not what this is, it's about as far as you can get from that.

> that's not what this is, it's about as far as you can get from that. I don't know what it is. How exactly does the score relate to the experience of the web browser user? I'm a browser extension developer, and I've occasionally had people ask me about Speedometer scores, but I have no idea what they're supposed to mean or what to tell these people.

They say something about the speed of the browser, so it doesn't really make sense to ask extension developers about it, I don't think. Possibly that your extension might make the browser slower, so you could compare scores with and without the extension and see whether it negatively affects performance? (Although I'm not sure it can necessarily tell you anything about to what extent it affects performance, only that it does.)

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#45

Post by om2 »

CharlesW wrote: post #13

Try it here: https://browserbench.org/Speedometer3.0/ Very unscientific results using a Mac Studio - Chrome: 20.4, Safari: 17.9, Firefox: 20.1. Safari on an iPhone 13 Pro Max - 16.5.

What Safari version are you using? For me, with 17.4, Safari is ahead of Chrome and Firefox, though it is close if you use dev channel.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#46

Post by capitainenemo »

om2 wrote: post #43
wswope wrote: post #36

Earlier quoted context omitted.

If you’re on iOS, Apple gates Firefox from using JIT JS compilation which massively hinders performance. E: I was wrong/extremely out-of-date - it does have JIT but relies on the Safari/Webkit implementation. In ancient versions of iOS, the WebView widget that third-party browsers were forced to use had JIT disabled, but that’s long since changed.

That’s not accurate. Firefox and all third party WebKit apps get the same JOT as Safari.

Right, but expecting the same behaviour from "Firefox" on iOS as on desktop is just not going to happen, since they have no control over the core engine. It's why, in general using iOS devices for cross-browser testing is pretty useless.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#47

Post by psychoslave »

dylan604 wrote: post #32

[flagged]

Probably Goodhart's law: when a measure becomes a target, it ceases to be a good measure.

To my mind, it means that the metric becoming the main focus, it makes easy to forget the original relevant goal and even works against it. That is not the case here.

https://en.m.wikipedia.org/wiki/Goodhart's_law

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#48

Post by om2 »

lapcat wrote: post #31
treyd wrote: post #29

Earlier quoted context omitted.

> On the other hand, premature optimization is the root of all evil. The web ecosystem is extremely mature and widely used. The workloads are fairly well understood. It is a magic unit, but the factors that go into it have a lot of thought from real-world scenarios. Bringing up "premature optimization" is completely irrelevant because that's not what this is, it's about as far as you can get from that.

> that's not what this is, it's about as far as you can get from that. I don't know what it is. How exactly does the score relate to the experience of the web browser user? I'm a browser extension developer, and I've occasionally had people ask me about Speedometer scores, but I have no idea what they're supposed to mean or what to tell these people.

Speedometer measures web app responsiveness. Roughly, it simulates a series of user operations on web apps built with various frameworks (as well as vanilla JS), and measures the time it takes to complete them and paint the results to the screen.

The score is a rescaled version of inverse time - if it goes up, that implies the browser can handle more user operations per second, or alternately, it takes fewer milliseconds to complete a user operation in a complex web app.

Re: Speedometer 3.0: A shared browser benchmark for web application responsiveness

#50

Post by coldpie »

Timon3 wrote: post #42
dylan604 wrote: post #32

[flagged]

When that metric is "performance in real workloads", I can't imagine it ever becoming irrelevant. Just look at their new tests: > In particular, we added new tests that simulate rendering canvas and SVG charts (React Stockcharts, Chart.js, Perf Dashboard, and Observable Plot), code editing (CodeMirror), WYSIWYG editing (TipTap), and reading news sites (Next.js and Nuxt.js). > We’ve also improved the TodoMVC tests: up…

The article even links to real-world performance measurements that are completely separate from the benchmark. I have no idea what the purpose of the OP's comment was. I guess they just wanted to blurt out the first cynical thing that came to mind. Awesome contribution.
Post reply on HN