Live data from Hacker News

Is BI dead? – On dismantling data's ship of Theseus

benn.substack.com

31–40 of 118 posts

Re: Is BI dead? – On dismantling data's ship of Theseus

#31
The ugly truth of BI tools is that they are of most benefit to the organizations which are least capable of using them.

The goal of properly exploiting BI comes with many prerequisites which sound superficially reasonable but turn out to be decade-long side-quests. Things like having all your data in one data model. Things like understanding where your data comes from, and exactly what it means.

These prerequisites are easy for small orgs, but small orgs benefit least from BI and typically get better bang-for-buck from Excel.

Large orgs find themselves mired in the political meta-problems of meeting those prerequisites, like joining up the fiefdoms that own data sources with the cabals that run data governance and the accountants who want a return on the investment of simplifying a sprawling legacy estate.

BI tools are generally incredibly poor at dealing with the bag-of-spanners heterogeneous data landscapes that exist in these organizations, and their analytics nirvana remains largely unattainable.

The trend that the article describes - towards simpler, composable BI components, each with more modest goals - is progress. It helps move focus away from the relatively-easy problem of visualization, to the rest of the data stack.

Re: Is BI dead? – On dismantling data's ship of Theseus

#32
post #30

I’ve spent the last 10 years of my life building “Big Data” platforms for companies (Kafka, Druid, Hadoop, Databricks, Presto, Teradata, Informatica, MicroStrategy, etc etc) and I still have no idea what people need most of this crap for. There’s so much duplicative functionality, and a basically never ending list of OLAP adjacent products emerging every year. I mean it’s good for me personally, it’s pretty much a gr…

Interesting. To me, you seem to be conflating some pieces of data infra plumbing with very sensible and concrete goals (Kafka, Hadoop) with others that are adjacent tools which may or may not make sense. Regardless of any tech details, the idea behind Kafka makes a lot of sense: https://engineering.linkedin.com/distributed-systems/log-wha...

Or did you mean, "I have no idea what business needs all this data for", a much broader question?

Re: Is BI dead? – On dismantling data's ship of Theseus

#33
I do believe that we have seen solutions trying to be a horizontal solution to problems, and I think the evolution will turn into vertical solutions. I have also invested a lot of time, as we are a couple of people who has been building on a vertical BI tool for the SaaS space for about a year.

The main goals are:

- Easy accessible for novice users. We want to make the Google for BI to help empower non-technical people (no more salespeople asking for reports from devs)

- More advanced editors for the more technical people

- Advanced alerts + integrations to 3rd parties

- Later on proactive reports

Hit me up on @philipanderse if you want to test it out.

Re: Is BI dead? – On dismantling data's ship of Theseus

#34
post #31

The ugly truth of BI tools is that they are of most benefit to the organizations which are least capable of using them. The goal of properly exploiting BI comes with many prerequisites which sound superficially reasonable but turn out to be decade-long side-quests. Things like having all your data in one data model. Things like understanding where your data comes from, and exactly what it means. These prerequisites a…

Or that most people are totally clueless about their own companies, processes and data.

Re: Is BI dead? – On dismantling data's ship of Theseus

#35
post #32
post #30

I’ve spent the last 10 years of my life building “Big Data” platforms for companies (Kafka, Druid, Hadoop, Databricks, Presto, Teradata, Informatica, MicroStrategy, etc etc) and I still have no idea what people need most of this crap for. There’s so much duplicative functionality, and a basically never ending list of OLAP adjacent products emerging every year. I mean it’s good for me personally, it’s pretty much a gr…

Interesting. To me, you seem to be conflating some pieces of data infra plumbing with very sensible and concrete goals (Kafka, Hadoop) with others that are adjacent tools which may or may not make sense. Regardless of any tech details, the idea behind Kafka makes a lot of sense: https://engineering.linkedin.com/distributed-systems/log-wha... Or did you mean, "I have no idea what business needs all this data for", a m…

It can become the opposite of sensible, very easily. I've had devs demand to use Kafka for storing 100mb+ blobs (which of course required a custom broker config + cluster)

Re: Is BI dead? – On dismantling data's ship of Theseus

#36
post #2

If you make it to the 8th paragraph, you may learn that "BI" stands for "business intelligence".

My first though was bisexuality, then Basic Income, then glanced at the article and found it was something to do with Salesforce, then found your comment. Thank you so much.

Re: Is BI dead? – On dismantling data's ship of Theseus

#37
I entered in a discussion with Benn on Twitter on Friday around this topic. Here is my take on it in case you are interested in a different perspective: https://jpmonteiro.substack.com/p/a-friday-fight-and-the-int...

TL;DR: I believe the trend will be to have an ecosystem-like approach to BI with tools complementing each other and communicating with each other through open standards.

Re: Is BI dead? – On dismantling data's ship of Theseus

#39
post #22

Earlier quoted context omitted.

I've been doing a fair amount of BI work myself recently. I reckon of the 90% that don't make any data driven decisions with the dashboards they're provided, there are three causes I run into frequently. 1. They can't describe what info they need 2. That info isn't available. 3. They don't make decisions based on data at all. I feel I'm getting better at addressing 1 and 2 the more I do it, but I figure there's a fai…

Yes on 3. One way to solve is to automate the action from insight part and inject into users workflow. I am looking at this now with support of senior sales stakeholders. Simple example: Client X activity has been trending down vs it's peer group, Call Client X to understand what's going on.

I like the idea.

Only works if you've got buy-in from management though.

Re: Is BI dead? – On dismantling data's ship of Theseus

#40
post #32
post #30

I’ve spent the last 10 years of my life building “Big Data” platforms for companies (Kafka, Druid, Hadoop, Databricks, Presto, Teradata, Informatica, MicroStrategy, etc etc) and I still have no idea what people need most of this crap for. There’s so much duplicative functionality, and a basically never ending list of OLAP adjacent products emerging every year. I mean it’s good for me personally, it’s pretty much a gr…

Interesting. To me, you seem to be conflating some pieces of data infra plumbing with very sensible and concrete goals (Kafka, Hadoop) with others that are adjacent tools which may or may not make sense. Regardless of any tech details, the idea behind Kafka makes a lot of sense: https://engineering.linkedin.com/distributed-systems/log-wha... Or did you mean, "I have no idea what business needs all this data for", a m…

Well ideas behind all the other stuff will also make lot of sense if one align with perspective of snake oil salesman. And I do think Kafka/hadoop etc fit in snake oil category.

Linkedin saying Kafka is good for them is not saying much.

Post reply on HN