Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is BI dead? – On dismantling data's ship of Theseus

benn.substack.com

Post reply on HN
Print view View original

91–100 of 118 posts

Re: Is BI dead? – On dismantling data's ship of Theseus

#91

Post by Rastonbury »

perl4ever wrote: post #81
edmundsauto wrote: post #74

Earlier quoted context omitted.

PowerBI is kind of a MS Access for BI. It loads data from your warehouse and gives both visual builders and query interfaces to data. IE, it's a specific product name - the data viz tool from MS to accompany their other services.

It seems to be part of Excel since 2016, no? It appears to me that it's morphed over time and I'm not sure to what extent it remains a separate product. I've used both DAX and M (Power Query) via recent versions of Excel. Obviously there are plenty of ways to interface with data. https://en.wikipedia.org/wiki/Power_Pivot says both Power Query and Power Pivot are part of Excel.

It's a separate product, analogous/competitor to Tableau. Just go to the product page

Re: Is BI dead? – On dismantling data's ship of Theseus

#92

Post by javierluraschi »

monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

Would love to hear your feedback for https://hal9.ai -- We are building an open source platform for data analysis based on reusable code blocks and a community to build and monetize their contributions. We are pretty early in our journey, launched our alpha and getting ready for our beta release, but would love to hear your thoughts. You can find me at javier at hal9.ai. Cheers!

Re: Is BI dead? – On dismantling data's ship of Theseus

#93

Post by StreamBright »

monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

I am working as a consultant in the field for some time. The approach what I found that works very well is to create a BI org that is driven by the business needs and only have data for what you actually need.

Less is more.

Re: Is BI dead? – On dismantling data's ship of Theseus

#94

Post by krauses »

SilverRed wrote: post #66
KptMarchewa wrote: post #64

Earlier quoted context omitted.

Are you running analytics on an transactional database?

Yes, Power BI connects up to a read only mirror of the app sql db.

Is the relational source db a denormalized star schema? Are you doing direct query or import in Power BI?

Re: Is BI dead? – On dismantling data's ship of Theseus

#95

Post by toyg »

gumby wrote: post #24
TOMDM wrote: post #22

Earlier quoted context omitted.

I've been doing a fair amount of BI work myself recently. I reckon of the 90% that don't make any data driven decisions with the dashboards they're provided, there are three causes I run into frequently. 1. They can't describe what info they need 2. That info isn't available. 3. They don't make decisions based on data at all. I feel I'm getting better at addressing 1 and 2 the more I do it, but I figure there's a fai…

> there's a fair amount from group 3 who demand the reports nonetheless... It’s a very important function: provide data that can be cherry-picked to support a decision already made. If the decision is too complex for that, hire a big consulting firm to write a report suggesting and justifying the action desired. A decison staple surely going back millennia.

Let's be fair and provide the other side's view: gathering focused data can prove the need for a decision that faces internal (and often irrational) opposition to necessary change.

Re: Is BI dead? – On dismantling data's ship of Theseus

#96

Post by monkeydust »

SilverRed wrote: post #63
monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

We provide power bi access to the data our app runs on and it has been a mixed bag. The customers absolutely love it. But they build dashboards where hitting the refresh button can bring a super expensive db to its knees for half an hour. So then we have to get our db people in to rewrite all of their reports so they don't do this which eliminated the benefit of having the customers do it. In the end there is just no…

OP here.

Interesting as we're looking at going long with PowerBI embedded for a client facing app and ditching our home grown effort which is always playing catch-up and costs a fortune.

Re: Is BI dead? – On dismantling data's ship of Theseus

#97

Post by toyg »

jl6 wrote: post #31

The ugly truth of BI tools is that they are of most benefit to the organizations which are least capable of using them. The goal of properly exploiting BI comes with many prerequisites which sound superficially reasonable but turn out to be decade-long side-quests. Things like having all your data in one data model. Things like understanding where your data comes from, and exactly what it means. These prerequisites a…

> BI tools are generally incredibly poor at dealing with the bag-of-spanners heterogeneous data landscapes

It's not that they are poor, it's that they eventually require some moderately-challenging coding - because there is only so much complexity you can hide behind point&click before things become untractable. Programmers get bored doing BI plumbing, and those commercial ecosystems are quite secretive and often expensive to train in; so the talent pool is small and costs can skyrocket pretty quickly, making the whole proposition unappealing.

Maybe if we accepted that "BI Programmer" is a respectable career and something that is fundamentally unavoidable, providing clear career paths inside companies for it, instead of trying so hard (and failing) to get rid of such figures as soon as they progress a bit, there would be more predictability and less angst in the space.

Re: Is BI dead? – On dismantling data's ship of Theseus

#98

Post by guareber8 »

blakeburch wrote: post #68
monkeydust wrote: post #17

As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them. Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer. We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data mun…

I see things going the same direction. I've always been pretty anti-dashboard because of the lack of long-term utility. If you want to make data-driven decisions, you need to spend the time codifying the decision making process and automating that. Automated actioning off data is far more impactful than automated visualization of data. When it comes to business stakeholders, the biggest obstacle is trust. If you're n…

The way I'm (currently!) approaching this is by implementing the decision engine in 2 steps separated by an evaluation period. I refer to these to the stakeholders as "user approves" and "user reviews".

Essentially, I first implement the engine with the output being an automated list of 1-click actions (typically a set of links on an email, but could easily be a dashboard button or anything else), so the system default is "detect, but don't do". After an evaluation period of the system actions, we move onto the second part, in which the system performs the actions and produces a log of activity, plus a list of 1-click UNDO actions to the user.

The idea is that a) the system earns trust from the stakeholders due to their direct involvement, b) the system can enjoy some supervised learning from someone other than the dev team and c) worst case, if it never earns enough trust, I've still saved a stakeholder dozens of hours of work sifting through dashboards as opposed to taking a look at a pre-filtered list. A few systems have turned out well enough that they never move from Phase1 while still being considered a massive win.

Re: Is BI dead? – On dismantling data's ship of Theseus

#100

Post by dspillett »

haddr wrote: post #27
marcinzm wrote: post #25

Earlier quoted context omitted.

I run a data team. We do two things: * Provide a clean minimal view of the data that other teams can pivot table on top of or download in the BI tool. We do onboarding, support and all that for this data. * Provide dashboards and reports for more complicated ongoing questions that require data that's not cleaned yet. We don't use Looker but based on the sales demo it seems optimized for this type of workflow. The cor…

This is the direction where most BI work is heading for: you need a team of data engineers that can make sure that the data for your BI dashboards is delivered and in a good quality. Everything else that doesn’t need it is probably simple enough to be run as some PowerBI dashboard on top of production DB. I think today most effort in those heavier BI cases is spent on data transformations and making sure that data fl…

> dashboard on top of production DB

A lot of what users need can be done this way in my experience. Also giving their people access to customise reports/exports can go even further. I'd advise against direct access to “real” production though: have the dashboard and custom reports run against a (probably readonly) replica of some sort¹ so runaway reports don't impact live app performance.

[1] via replication, log shipping, etc... — all major databases should support at least one method that can achieve this with minimal latency in updating the replica under normal conditions

Post reply on HN