Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is BI dead? – On dismantling data's ship of Theseus

benn.substack.com

Post reply on HN
Print view View original

51–60 of 118 posts

Re: Is BI dead? – On dismantling data's ship of Theseus

#51

Post by _delirium »

FranzFerdiNaN wrote: post #42
monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

I work as a data engineer in a BI team, with our product being part of a large software suite in healthcare. We got a small number power users who really use the dashboards, who get real result from them and who actively help our development. We also got a larger group of people who use it once in a while. But most people seem to log in once or twice and then never again. And if you ask why it’s often because they co…

> And if you ask why it’s often because they couldn’t find the answer to their incredibly specific one-time only question, so they decided it’s all useless and they continue to work on their own personal homegrown BI-suite in excel.

Hah this is probably me. The university I work for uses a BI tool (MicroStrategy) to track students/majors/etc. But I usually find it easier to use the BI tool primarily as a way to get a CSV or XLS export of enough source data to answer my question, and then do the actual analysis in Python+pandas. I can probably formulate my query in their weird proprietary query builder, but it seems unnecessarily complicated and I already know how to do data analysis in Python.

Re: Is BI dead? – On dismantling data's ship of Theseus

#52

Post by jon-wood »

intrasight wrote: post #10

I remember learing decades ago that "In UNIX, everything is a file" I approach BI now as "In BI, everything is in your database". With everything in the database (SQL Server in my case) I can deliver BI to my users with quality tools like Tableau and Power BI - or just with generic web reports.

It really is amazing how far you can get by just dumping everything into a single database and letting people do joins. We’re doing the same thing with Stitch replicating data from production databases and a few SaaS products into a big Postgres instance and it’s so low on maintenance hassle. The most advanced our “BI pipeline” gets is a few stored procedures run from a cron job to aggregate data in a particularly large table.

Re: Is BI dead? – On dismantling data's ship of Theseus

#54

Post by MangoCoffee »

riskneutral wrote: post #53
MangoCoffee wrote: post #28

>BI should include all consumption we use Microsoft PowerBI. it work great within the Microsoft ecosystem. if you try to pull data from NetSuite with PowerBI. it went to crap.

Why? What is special about NetSuite?

its the ERP software that we use. people needs to do report with data from NetSuite.

Re: Is BI dead? – On dismantling data's ship of Theseus

#55

Post by 0xbadcafebee »

Data is clay. We are still struggling to make bricks. We are still very far from building a house out of it, much less a large building. And we're going to eventually find out that we can't make skyscrapers out of it at all.

Just like bricklayers haven't even remotely been replaced with robots, I doubt the data-mungers will be, either. We are probably on the cusp of a whole new generation of "data people" who will have careers that span a generation, and do nothing but sift through data.

Re: Is BI dead? – On dismantling data's ship of Theseus

#56

Post by tomnipotent »

gav wrote: post #44
monkeydust wrote: post #29

Earlier quoted context omitted.

Yes on 3. One way to solve is to automate the action from insight part and inject into users workflow. I am looking at this now with support of senior sales stakeholders. Simple example: Client X activity has been trending down vs it's peer group, Call Client X to understand what's going on.

This is the direction that I'd like to see more people moving in: pushing insights to users rather than expecting users to come to dashboards and get those insights. Nobody is saying "I'd love another tool that I am expected to check every day", both embedded analytics and insights are how to drive adoption and unlock value from data.

> pushing insights to users

This is part of the expectation of business analysts, that they will surface and communicate data not immediately being asked for by executives or other stakeholders. This usually means running a meeting and presenting findings, and then pairing with another stakeholder to flesh out further opportunity.

If there's no one to pick up the baton, there's very little that can be done about that.

Re: Is BI dead? – On dismantling data's ship of Theseus

#57

Post by d--b »

I am building Jig (based on observable): https://www.jigdev.com

Because I believe that BI should be done by people who have a mixed set of skills between stats and programming.

A programmer alone can’t make the stats, and a mathematician can’t build a reporting tool.

Jig is a shortcut to build reports quickly, but I am somehow convinced by javascript is just not good enough to do that job.

I think I might have to try and design a language that has indexed containers as first class objects…

Re: Is BI dead? – On dismantling data's ship of Theseus

#58

Post by perl4ever »

FranzFerdiNaN wrote: post #42
monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

I work as a data engineer in a BI team, with our product being part of a large software suite in healthcare. We got a small number power users who really use the dashboards, who get real result from them and who actively help our development. We also got a larger group of people who use it once in a while. But most people seem to log in once or twice and then never again. And if you ask why it’s often because they co…

>they continue to work on their own personal homegrown BI-suite in excel.

Why aren't you providing your dashboards via Excel, then? What exactly does the fancy BI software do that Excel won't? I have used Qlikview but not Tableau or Informatica, and I'm a bit fuzzy on what people mean when they say "Power BI". Is using Power Query what you would call Power BI? Or does that mean using the data model stuff? Power Query is the part of Excel that I particularly like because data kind of stays where you put it.

Re: Is BI dead? – On dismantling data's ship of Theseus

#59

Post by b9a2cab5 »

d--b wrote: post #57

I am building Jig (based on observable): https://www.jigdev.com Because I believe that BI should be done by people who have a mixed set of skills between stats and programming. A programmer alone can’t make the stats, and a mathematician can’t build a reporting tool. Jig is a shortcut to build reports quickly, but I am somehow convinced by javascript is just not good enough to do that job. I think I might have to try…

In my experience BI has no business being done by technical people. You need business people who are just technical enough to use the system for what they need. You need domain knowledge and deep product understanding to know what data to be looking for, not a stats and CS whiz. In 99% of cases, SQL backed by a fast distributed analytics database and Tableau+Excel is enough for that.

Even as a software engineer doing basic analytics charts for the service I own I don't want to be writing JavaScript. Ideally I wouldn't be writing SQL either.

Re: Is BI dead? – On dismantling data's ship of Theseus

#60

Post by perl4ever »

_delirium wrote: post #51
FranzFerdiNaN wrote: post #42

Earlier quoted context omitted.

I work as a data engineer in a BI team, with our product being part of a large software suite in healthcare. We got a small number power users who really use the dashboards, who get real result from them and who actively help our development. We also got a larger group of people who use it once in a while. But most people seem to log in once or twice and then never again. And if you ask why it’s often because they co…

> And if you ask why it’s often because they couldn’t find the answer to their incredibly specific one-time only question, so they decided it’s all useless and they continue to work on their own personal homegrown BI-suite in excel. Hah this is probably me. The university I work for uses a BI tool (MicroStrategy) to track students/majors/etc. But I usually find it easier to use the BI tool primarily as a way to get a…

>their weird proprietary query builder

If you're talking about Power Query, the best way to use it is to write code directly once you observe what the interface generates. It's a rather nice kind-of-functional language that really comes into its own when you start using/creating higher order functions.

Power Query gave me an intense craving for a form of SQL with a similar facility for functions.

For example, I have a lot of experience writing Oracle SQL, and Power Query didn't offer "grouping sets". But I realized it could be implemented using functions. It would be so great if SQL supported that.

Post reply on HN