Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Calibre Test Profiles – Test your sites under real world conditions

calibreapp.com

Post reply on HN
Print view View original

1–10 of 19 posts

Show HN: Calibre Test Profiles – Test your sites under real world conditions

#1

Post by benschwarz »

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com. Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time.

I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.) I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!

Show HN: Calibre Test Profiles – Test your sites under real world conditions
calibreapp.com

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#2

Post by benschwarz »

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com. Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.).

Last year I knuckled down to get into the position where I could work on it full time. I've spent the last 8 weeks working on "Test profiles" (The launch post is here: https://medium.com/get-calibre/announcing-test-profiles-and-...) — It allows you to emulate hardware devices (cpu speed, user-agents and screen dimensions), as well as limit the bandwidth (latency, download and upload speeds), and set cookies (you could use this to log into your app, test a staff-shipped feature, or maybe turn off advertising and see the performance impact.)

I think I've finally got the basis of a really powerful tool, and I'm keen to share it with… everyone! I do everything myself, so if you've any questions, I'm here to answer them!

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#3

Post by pspeter3 »

benschwarz wrote: post #2

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com . Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time. I've sp…

Is it only clean room metrics or can you also collect metrics from actual users?

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#4

Post by [deleted] »

pspeter3 wrote: post #3
benschwarz wrote: post #2

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com . Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time. I've sp…

Is it only clean room metrics or can you also collect metrics from actual users?

[deleted]

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#5

Post by benschwarz »

pspeter3 wrote: post #3
benschwarz wrote: post #2

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com . Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time. I've sp…

Is it only clean room metrics or can you also collect metrics from actual users?

Atm, the focus is clean room, but real time/user metrics are on the horizon—especially when the company is more than just me!

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#6

Post by mikerg87 »

benschwarz wrote: post #2

Hi HN! I'm Ben, the solo founder of Calibre Analytics: https://calibreapp.com . Calibre monitors websites and applications in a clean-room testing environment from all over the world. Up until January Calibre has been an "inbetween other things" / evenings / holidays kind of project. (Only, oops—I did that for four years.). Last year I knuckled down to get into the position where I could work on it full time. I've sp…

I have been using http://www.webpagetest.org/ for similar investigations. csn you tell me what you see as advantages of your tool over webpagetest.org ?

Btw congrats. This looks very well done.

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#8

Post by techwizrd »

I totally mistook this for Calibre[0], the open-source e-book app.

The site looks really good. This is a very impressive product for an "in-between other things"/evenings/holidays kind of project. My only issue is that the animation of the application window coming up can sometimes be laggy.

0: https://calibre-ebook.com/

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#9

Post by benschwarz »

techwizrd wrote: post #8

I totally mistook this for Calibre[0], the open-source e-book app. The site looks really good. This is a very impressive product for an "in-between other things"/evenings/holidays kind of project. My only issue is that the animation of the application window coming up can sometimes be laggy. 0: https://calibre-ebook.com/

Yeah… that animation isn't what it should be for a performance product—luckily, after this (and the lighthouse integration I'm working on right now), I'll have all-new marketing too!

Edit: Thank you! :-)

Re: Show HN: Calibre Test Profiles – Test your sites under real world conditions

#10

Post by benschwarz »

packetized wrote: post #7

Test profiles is an especially interesting idea, something that I don't believe other tools in this space (Gomez/Compuware APN, ThousandEyes) have quite yet. Other than test profiles, what sets Calibre apart?

Without being super familiar with the tech that any of these companies are running —

The biggest thing that sets Calibre apart is that its entirely built using the Chrome developer tools APIs. I like to think about it as someone on your team who opened devtools after every deploy, and kept a neat spreadsheet of all the changes that had occurred. In short, all the data is ready and available for the taking.

This also has the added benefit that I have the power of the Chrome browser to play with. Projects like Google's Lighthouse (https://developers.google.com/web/tools/lighthouse/ — Which I'm currently integrating into Calibre!) have meant that a lot more powerful tools/metrics/resources/etc are coming along strongly, and this will only help make Calibre more powerful too.

Calibre is firmly focused on what your users actually experience, and helping you get to the bottom of 'why' quickly and efficiently. So while technically focused tools are imperative for any team, many of them aren't focused on the most important people in the equation.

Also, thanks — Good question.

Post reply on HN