Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

mcfunley.com

Post reply on HN
Print view View original

31–40 of 72 posts

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#31

Post by jkaptur »

alexandercrohde wrote: post #28

Generally really good points. However one great use of realtime analytics is as a way to immediately catch production errors (particularly when connected to alarms, e.g. Splunk). My thought is that realtime amazing systems as discussed can be an exceedingly difficult engineering problem or a trivial business problem (use google or another hosted solution). Personally, any engineer who would try to reinvent the wheel…

> It's important to divorce the concepts of operational metrics and product analytics. Confusing how we do things with how we decide which things to do is a fatal mistake.

It sounds like you agree with this quote from the article?

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#32

Post by ACow_Adonis »

gxs wrote: post #16
ACow_Adonis wrote: post #8

"But unless the intention is to make decisions with this data, one might wonder what the purpose of such a system could possibly be." Speaking as an analyst, you'd be amazed (or not) at how much analytics is done either for resume padding, because one guy thinks it's "cool", or to enable marketing or executives to chase their own tails. Indeed, with the general phenomenon of bullshit jobs, infinite instantaneous and…

Would you mind giving examples of some of these metrics? Curious too see exactly what they are.

Almost every metric can be abused by increasing the frequency, resolution and dimensions in which it is measured.

In terms of some real world examples:

Labour force statistics are almost always reported on an instance level by commentators/media, but its trend that's recommended/meaningful.

Net promoter scores and breakdowns are consistently done and reported too frequently on too small a base to establish a trend, and they're usually cross tabulated and spoke on far too many dimensions.

Customer opinions and employee ratings/surveys.

Generally it's just fundamental statistics: don't reason too much about the general from the specific. And as you get greater resolution and move towards real time events, your specific becomes smaller and smaller, which means that extrapolations onto the general result in greater and greater errors...

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#33

Post by danieltillett »

konschubert wrote: post #24
inopinatus wrote: post #13

Earlier quoted context omitted.

Have you considered increasing the price? I'm serious. I'm not a mobile app publisher, I don't know anything about app discovery through the Play store, but I have experienced the paradox of increasing sales by increasing prices in other channels. I'm not super up on the psychology at play, but I've read it's because people use price as a proxy for quality/utility in uncertain markets.

I think that the price is pretty high already for app store standards :) And honestly, if you look at the app, a higher price would feel like a ripoff.

Well it might be worth a try - you at least don’t have to worry about killing your sales :)

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#34

Post by panarky »

sskates wrote: post #21
jakobegger wrote: post #2

When I started my business, I looked at sales number every day. I got my hopes up when I sold 15 copies of my app on a good day, only to feel completely devasted when I sold only 4 copies the next day. After some time I stopped looking at daily sales numbers, and switched to weekly numbers instead. But even that was too often. Now I fetch my numbers roughly every other week, and don‘t worry at all about an individual…

That's definitely one of these things you have to learn as part of running a business. You tend to over-index on things you can measure and under-index on things that you can't. After a while you realize there's a lot more randomness in what happens than you might expect and so you can't weight any particular data point too highly. If you look at patio11's blogging of Bingo Card Creator, you'll notice he has little i…

The same principle applies to other information.

I used to spend a lot of time reading news stories and checking stock prices every day. Turns out almost all of that is random noise that doesn't add up to actionable patterns.

Better to ignore the hourly and daily stuff and look at longer timeframes with long-form articles and books. Even fiction can be pretty good at identifying real macro patterns.

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#35

Post by rdiddly »

jakobegger wrote: post #2

When I started my business, I looked at sales number every day. I got my hopes up when I sold 15 copies of my app on a good day, only to feel completely devasted when I sold only 4 copies the next day. After some time I stopped looking at daily sales numbers, and switched to weekly numbers instead. But even that was too often. Now I fetch my numbers roughly every other week, and don‘t worry at all about an individual…

Also see: The stock market.

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#36

Post by qq66 »

In my experience, 99% of the value of real-time analytics has been in identifying service disruptions that monitoring tools don't find, and 1% of the value has been in informing business decisions. If some change badly broke your HTML rendering, but didn't throw any errors, you may not see it in service monitoring but you will definitely see it in the signups per day.

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#37

Post by davedx »

nerfhammer wrote: post #22

> The turnaround time also imposes a welcome pressure on experimental design. People are more likely to think carefully about how their controls work and how they set up their measurements when there's no promise of immediate feedback. This seems like a cranky rationalization of the lack of a fairly ordinary system. Sure, you shouldn't draw conclusions for potentially small effects on What this means is you can't tel…

> What this means is you can't tell whether an experiment launch has gone badly wrong.

Personally I prefer automated testing to tell me if a feature has gone badly wrong, not conversion numbers. Then I find out before I launch too.

Or do you mean that the UX is so badly designed the users cannot use your software anymore? In which case, maybe there are bigger problems than real time analytics

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#39

Post by watt »

konschubert wrote: post #24
inopinatus wrote: post #13

Earlier quoted context omitted.

Have you considered increasing the price? I'm serious. I'm not a mobile app publisher, I don't know anything about app discovery through the Play store, but I have experienced the paradox of increasing sales by increasing prices in other channels. I'm not super up on the psychology at play, but I've read it's because people use price as a proxy for quality/utility in uncertain markets.

I think that the price is pretty high already for app store standards :) And honestly, if you look at the app, a higher price would feel like a ripoff.

Economically, if you deliver value, you deliver value. Does not matter the price, if it's worth doing and generates profit for me.

Re: Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

#40

Post by anuragojha »

ACow_Adonis wrote: post #32
gxs wrote: post #16

Earlier quoted context omitted.

Would you mind giving examples of some of these metrics? Curious too see exactly what they are.

Almost every metric can be abused by increasing the frequency, resolution and dimensions in which it is measured. In terms of some real world examples: Labour force statistics are almost always reported on an instance level by commentators/media, but its trend that's recommended/meaningful. Net promoter scores and breakdowns are consistently done and reported too frequently on too small a base to establish a trend, a…

Interesting. Thanks for sharing. Do you know of any good writing on the web or print that talks more about such abuses in the analytics and corporate world? Interested to learn how to spot when kooky tactics are at play in a tech/corporate setting, specially when its willful.
Post reply on HN