Your Startup Needs a Dashboard • Hacker News Board

Re: Your Startup Needs a Dashboard

#21

Post by viscanti » Fri, Jun 01, 2012, 2:50 PM UTC

andraz wrote: post #9

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

I can't agree more. First you need a product that works at least for some users, then when you are expanding the user base it's time for dashboards. Dashboard answers the question: Is my product right now working for my users? Eventually it can also show trends in terms of resource usage, strange peaks, etc. But the primary function of dashboard is answering the question: "are my users getting the service?" Additiona…

This seems pedantic. Of course the product needs to work for some users before you can measure anything about it. Otherwise your dashboard is filled with zeros, which should be obvious because the product doesn't work for anyone (and isn't a product).

Once you've gotten to the MVP stage, you should be tracking the key metrics, and those should be easily viewable in some sort of dashboard. Tracking up-time (or if the service is working) is a small part of what should be measured. When you're just starting off, you need to quickly figure out if your market hypothesis is correct. You can't do that without data, and a dashboard is both easy to build and helps make more informed decisions for everyone involved.

No one is arguing that you should build a dashboard before you have a MVP. Once that happens, you need to be measuring things.

Re: Your Startup Needs a Dashboard

#22

Post by plant42 » Fri, Jun 01, 2012, 2:57 PM UTC

It's not only start-ups that need dashboard or visual metrics, established companies can benefit from more information about how their services/applications/etc work.

Its one thing to give the numbers to the management group, but its more readily accepted when combined in a visual manner. i.e. dashboard

My current employer is only now beginning to see the advantage of collecting good metrics and how it shows the behaviour of both the users and the application.

Their decision making and planning becomes more accurate over time due to having good information at their fingertips rather than an educated guess because originally the system was a black box.

Re: Your Startup Needs a Dashboard

#23

Post by sardonicbryan » Fri, Jun 01, 2012, 2:57 PM UTC

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

Dashboards don't have to be expensive to build. My company uses Graphite now, but before we were at that level, I just had engineering give me credentials for our DBs, and used ODBC + Excel.

Next thing you know, you have a dashboard tracking: - Real time revenue/revenue projections - Real time payments funnel metrics (impressions on payments page, conversions, average transaction size) - Real time tutorial completion rate by hour (aka application health monitoring) - Other KPIs tracked by the day (revenue, install, etc.)

In fact, there was a period of time when my dashboard was the most effective health monitoring in the company, and detected an issue before engineering or ops.

Re: Your Startup Needs a Dashboard

#24

Post by lifeisstillgood » Fri, Jun 01, 2012, 3:00 PM UTC

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

maybe -

I have taken the time to build a graphite/stasd setup. It is over engineered for now, but it grows and is really easy to add in new metrics. (there is even a fab file to install on ubuntu buried in github/lifeisstillgood/frozone)

but, you are right - a dashboard that measures how the site is going is useful. One that measures cashflow, email conversion rates and more is more useful. Linking those into graphite is not a good idea. A more traditional data collection is needed.

Re: Your Startup Needs a Dashboard

#25

Post by nextmat » Fri, Jun 01, 2012, 3:00 PM UTC

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

My sense is that people continue to think that monitoring is hard mostly because it used to be. There are great open source tools and services available now to make both collecting the data and watching it pretty simple to set up.

Yes you need some customers, otherwise you have no data to look at. However, as soon as you have _any_ customers if you aren't looking at real data to decide what is important to do next you are essentially just guessing.

For a long time that's what we did because tracking what was relevant was just too hard. Now what I see in a lot of startups that are leading the pack is that they are monitoring everything whether they think they'll need it or not - then when they have a critical question the data is already there.

Re: Your Startup Needs a Dashboard

#26

Post by njx » Fri, Jun 01, 2012, 3:05 PM UTC

andraz wrote: post #9

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

I can't agree more. First you need a product that works at least for some users, then when you are expanding the user base it's time for dashboards. Dashboard answers the question: Is my product right now working for my users? Eventually it can also show trends in terms of resource usage, strange peaks, etc. But the primary function of dashboard is answering the question: "are my users getting the service?" Additiona…

Dashboards are not just for "Is my product working for my users?"

They have variety of purposes. For e.g our webapp is an open book, we display live dashboard (https://my.infocaptor.com) of our very app itself because it is meant to be a dashboard platform

The main purpose of dashboards is to keep a birds eye view and take action when something is off.

Don't track metrics where you don't have time or resources to take action against.

Re: Your Startup Needs a Dashboard

#27

Post by chucklarge » Fri, Jun 01, 2012, 3:33 PM UTC

I suggests everyone read these whitepapers on dashboard design.

http://www.perceptualedge.com/articles/Whitepapers/Dashboard... http://www.perceptualedge.com/articles/Whitepapers/Common_Pi...

The author also has written a great book on dashboard design.

Operations dashboards are essential for any (tech?) company.

Business dashboards are another beast all together.

Re: Your Startup Needs a Dashboard

#28

Post by sophacles » Fri, Jun 01, 2012, 3:37 PM UTC

I worked at a place doing network appliance stuff. We managed and developed the machines, and product to end users. A simple dashboard increased our effectiveness by an order of magnitude. Some simple graphs and deep knowledge of the code really really helped. Some data from other departments combined with this was amazing. A simple color coded call queue count told us there was a problem before the front-line tech support people could even report it. Simple analysis of that queue told us where to look for problems: a predominance of one area code told us to examine specific machines (and which subset!), while a wide array of area codes told us that it was probably in billing or comms infrastructure. A couple of graphs of network traffic and system load told me what what the problem could be. I don't know how, I couldn't write code to do it, but I certainly could get to the problem much faster with that minimal input from graphs.

My working theory on dashboards is this: the human mind is amazing at pattern recognition. We're wired for it. We can make intuitive leaps see patterns that may be very hard to describe in math/stats or code. Particularly visual ones. So if you provide that data to your brain to crunch, you are enabling and augmenting your natural tooling. The graphs and stats should be as specific an pre-thought out as possible, but they aren't perfect. Fortunately, as time progresses you learn what "looks right" and what "looks like a problem in subsystem Foo".

This isn't a silver bullet, but certainly it is a great tool. Since then, I've tried to never do work without some sort of visual feedback I can background my innate pattern matching on. Even if it is just scrolling logs -- these patterns emerge and provide clues even if you can't express what they are.

(Anecdote: I had built a demo a while back, and it hiccuped during the live presentation. Fortunately I was in the back of the room with my logs scrolling, and I noticed the logs looked wrong, so I found out a script had died. I restarted it, causing a weird blip in one of our display graphs, but the presenter noticed it before calling attention to that graph in the course of presentation. He glanced at me and I gave him the thumbs up and the audience never even noticed. What was the pattern? The scrolling in one of my log windows slowed down....)

Re: Your Startup Needs a Dashboard

#29

Post by ArbitraryLimits » Fri, Jun 01, 2012, 3:52 PM UTC

taligent wrote: post #14

tomgallard wrote: post #3

I have to say, I'm in two minds about this. On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong. On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy…

I have to respectively disagree. Metrics are also about the state of your business. What is your growth rate ? What is your churn ? Are people only clicking on certain features ? Where should you improve performance ? These sort of questions are essential for startups especially in the early stages where you need to be careful where to allocate precious resources. Yes you can sort of gauge that with Google Analytics…

I think your parent post is saying that metrics are for control and not planning. Once you've identified what your business depends on, and have identified good metrics that (at best) are proxies for those things, a dashboard is a great idea. If on the other hand you don't actually understand your business model, your first goal is to start to understand it, and fixating on a particular set of metrics will only make that harder IMO.

Re: Your Startup Needs a Dashboard

#30

Post by MicahWedemeyer » Fri, Jun 01, 2012, 4:15 PM UTC

At any given time, there are a near infinite number of things your business "needs" to do: add a dashboard, A/B test your landing pages, test your database backups, fix your security holes, improve your customer service, add more metrics tracking, etc, etc. The list goes on and on.

A big part of running your business involves looking at this list and picking the ones that make sense for your business, not what the blog-post-du-jour is telling you.