Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 107 posts

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#51

Post by spark1212 »

One of the hardest challenges is ensuring alignment with the end user from ideation to delivery. It can be tough to figure out what the end user needs in the first place, let alone the details of how to define individual metrics or slice the data. This is a huge pain point for both externally and internally facing deliverables, but it's especially tough for external clients because you're likely a lot more limited in your ability to communicate ad-hoc to clarify things down the line. And once you've delivered something that's either irrelevant or inaccurate, then it can end up being game over for the engagement (if you're working externally) or your counterpart's trust in your output (if you're working internally).

So it's super important to get on the same page RE: goals and expectations and keep that alignment going to the end - so that there aren't any unpleasant surprises at the delivery stage. Some more on who to get involved and how here: https://www.avo.app/blog/who-should-be-involved-in-tracking-...

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#52

Post by happytiger »

amcaskill wrote: post #41
itsoktocry wrote: post #29

I've worked in many analytics projects across a number of companies as a consultant. I'm a big believer in "decision support systems". Find out what decisions your customers need to make, repeatedly, to their job. Quantify the heuristics and visualize that information (and that information only) in an easy to consume manner. More often than not that's an email or PDF . Another advantage is that by supporting the busi…

I helped build the analytics group at a PE fund, and this really fits with my experience. Good decision support is where most of the value is, and it’s about building things that draw conclusions, not just throwing the data over the fence with 50 filters and expecting the end consumer to do the actual analysis. I now work on an open source, code-based BI tool called Evidence, which incorporates a lot of these ideas,…

Agree with both of you, and would add that knowing who is using the system, and what they need to get out of it is really the key to making them shine.

Too many systems have too much data for too many customer categories and end up being useless to everybody.

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#53

Post by grvdrm »

conductr wrote: post #45
grvdrm wrote: post #8

I'll enjoy watching this thread evolve. Some thoughts from my experience: - Everyone asks to translate simpler spreadsheets and Excel charts/graphs into dashboards in your BI tool of choice. As soon as it's there, they'll ask you why they can't export to manage the data themselves. This vicious cycle can sometimes be stopped but is a slow-motion drag on productivity in lots of orgs. - Build in validations, and/or wor…

I work closely on BI projects but from a finance perspective. The concept I like to explain to the BI teams is that the dashboard is always just a snapshot of “what” is happening. But the underlying base level data is always needed to understand “why” it’s happening. And without the why, there’s no actual intelligence gain. Take a metric like Average Order Value (AOV). It may be ; total sales / order quantity. But as…

Similarly titled towards finance. I specialize in what I'll call decision analytics for insurance underwriters.

> Which means, the don’t even understand how un-intelligent their BI tool is or why the execs likely aren’t feeling like investing in BI has been worthwhile

And this relates to what I was thinking about in my first comment. I once was conversing the COO of my company (my last job), at a 1000+ person company, and asked him if he thought more concise requests for things would drive productivity. He, point blank, said: "sometimes I don't even know what I'm asking for"

I've remembered that moment for years. In so many situations, the actual BI/dashboard is the least important part of the puzzle. Instead it's all of the conversation and discovery to understand the real need(s)

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#54

Post by syndicatedjelly »

Tons of great advice in the comments. At risk of repeating others, here's what I've learned working on business intelligence tools for an engineering group:

- What users ask for and what users really want are often extremely different.

- Engineering executives like to place their "thumbprint" on every business analytics dashboard. They want evidence that the "intelligence" being reported has been customized by them. It's their way of imparting branding on the organization.

- UI/UX is far more important to users than how you handle the technical details. When discussing implementation with them, start with the UI so that they have a mental model to build from.

- Leave space to create cool things that you/your team want to make. The developers of BI dashboards often have excellent ideas for visualizing data that an end-user would not immediately think of. Leave room to "delight".

- Never assume the data is clean or accurate (even when there are regulatory reasons for it to be either of those things)

- Not everyone's opinion is equally valuable.

- Beware of corporate politics. I once had an analytics project completely shut down because it would expose certain weaknesses in the business that were not acceptable to discuss publicly.

Bonus: Read "Envisioning Information" by Edward Tufte.

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#55

Post by syndicatedjelly »

RyanHamilton wrote: post #10

80% of the time people should display a table, 15% a time-series or line chart. The other 5% is probably wrong. Anyone that asks for pie charts, 3d charts,... isn't a real data user ;)

Unless you're displaying amount of a pie or pizza that's been consumed so far, don't use a pie chart

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#56

Post by tailspin2019 »

grvdrm wrote: post #8

I'll enjoy watching this thread evolve. Some thoughts from my experience: - Everyone asks to translate simpler spreadsheets and Excel charts/graphs into dashboards in your BI tool of choice. As soon as it's there, they'll ask you why they can't export to manage the data themselves. This vicious cycle can sometimes be stopped but is a slow-motion drag on productivity in lots of orgs. - Build in validations, and/or wor…

Brilliant summary - mirrors my thoughts and experience quite closely.

Validation/testing has always been a challenge, especially given that dashboards are by definition quite “full stack” implementations where testing just the front end or back end is not sufficient and testing both in isolation can also often be challenging due to the huge possible variations in input data.

Mocking data is also hard because dashboards may also lean a lot on database-side calculations/filtering.

All of this has lead me to take quite a full-fat approach to testing dashboards, by using a real DB populated with test data, and testing the full complete application stack (driven by something like Playwright or Cypress) as well as more granular unit tests where a mocked data layer may be used.

I’m also looking at introducing visual regression tests next time I work on this kind of thing. The visual aspects of dashboards can easily drift over time even if the data is correct. You’re often theming charting libraries for example and the compliance of the theme can drift slightly if you update the library without really checking every detail of the visual appearance/layout every time. Or you may not even notice the “visual drift”…

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#58

Post by tylerhillery »

The biggest help we got was meeting directly with our customers and asking them “What would it take for you to login everyday to view this dashboard” and they clearly provided metrics and trends they care about but have a hard time getting access to the data. Also don’t get fancy with our visuals. Lots of big metric kpi visuals, tabular visuals, line charts & bar charts. They should be able to glance at the visuals and immediately known what’s going on and get sense of what the visual is conveying.

Another thing customers love is the dynamic ability we give them to be able to switch how certain visuals are grouped or what value is being displayed. We can’t for see all the different ways users will want to slice and dice the data so giving them that ability was huge.

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#59

Post by gorbachev »

#1 mistake people first dabbling with dashboards make is to show absolutely everything.

Don't do that. Show only the things users need to act on what's on the screen. Minimize the information, make it "glanceable".

If you have a troubleshooting dashboard, and you're showing 999 items with nothing going wrong, that one item that's actually wrong is not going to pop.

Re: Ask HN: Lessons learned from implementing user-facing analytics / dashboards?

#60

Post by alexpetralia »

* Design matters a lot - if it looks bad, people won't look at it.

* Layout for dashboards is almost completely formulaic. A panel for selected high-level stats (user growth % increase from last year, user % increase from last month, # new users added), a panel for breakdowns (user growth by marketing channel, user growth by registration cohort), a panel at the top for filters ("let's filter the entire dashboard by just this marketing channel, or just this registration cohort") identical to all breakdowns provided, and finally a row-level drill-down ("show me the users in this particular aggregation"). It took me a very long time to learn that this design is entirely cookie-cutter for good reason. Users always want the same things: stats, breakdowns, filters and drill-downs.

* Padding matters, font matters, color palette matters, no typos matter, visual hierarchy matters (i.e. big bold numbers versus smaller grey numbers).

* Always define the key metrics first (based on fact tables). All dimensions and drill-downs in the dashboard will derive from these front-and-center stats.

* Reconcile to existing metrics before broadcasting widely - almost always, people have the same stats in extant technologies (i.e. Excel, Mixpanel, Salesforce) and will instantly find inconsistencies between your figures and the extant ones.

* The vast majority of users will be passive viewers. Very few users will be "power" EDA (exploratory data analysis) users. EDA views will look different from the view that passive viewers want - keep them separate

* Obviously, the more things done in code, which promotes modularity and composability, the fewer data integrity issues you will have

Post reply on HN