Live data from Hacker News

Ask HN: Tools to visualize data in SQL databases?

news.ycombinator.com

121–130 of 137 posts

Re: Ask HN: Tools to visualize data in SQL databases?

#121

I use Gephi a good deal to visualize connections between tables and schemas in a messy production MySQL database, mostly with the aim to stop people (read: consultants, junior developers, managers trying their hand writing SQL) from developing ad hoc duplicate tables for their own niche purposes. Gephi lets me show how this kind of table bloat happens over time and helps explain performance degradation.

This is interesting I have seen Gephi before but never used it. Do you have any links or posts explaining how you use it to visualise connections in MySQL? I would like to do something similar with Oracle.

Take a look at the circle pack layout, it's what I use to group by schema

https://blog.miz.space/tutorial/2020/01/05/gephi-tutorial-la...

Re: Ask HN: Tools to visualize data in SQL databases?

#122
One thing that has become apparent to me over the years is that most of these exploration and visualization tools will be pretty ineffective unless the data is modeled correctly. Some of the tools mentioned here will actually do a lot of the modeling before the exploration and visualization kick in, but the tool is probably only doing the best it can but will most likely not understand a lot of the structure and business nuance that have been accumulated throughout its existence.

Moreover, if you plan to adopt and build upon one of these tools that infers and generates the models as well as provide the explore and visualization functionality, you might be painting yourself into a corner and forcing all current and future workloads to use this layer. Otherwise you'll be having to reinterpret and reimplement your models all over the place; one off SQL scripts/reports, web analytics, dashboards/visualizations/reports on other analytics tools. Then you'll also end up having to scale this tool up in both compute and storage to handle the load that grows over time. This can end up being quite costly in time, money and responsibility.

While these tools will offer a lot of value providing visibility and insight into your data, it'll probably be worth circling back and seeing if the data can and should be modeled correctly (semantic layer) before hitching your wagon to your first choice.

Once your data is all modeled, it might be worth re-evaluating all the tools that you started with and see how they manage now that your house is a little more in order.

Remember, your modeling doesn't have to be done by the same tool that does your exploration and visualization.

This is a great article related to these ideas: https://benn.substack.com/p/is-bi-dead

Re: Ask HN: Tools to visualize data in SQL databases?

#123

Free: Power BI is probably going to give you everything you need. It's free, easy to use and provides a lot of features to grow into. Paid: If you have the budget, Aqua Data Studio gives you the database management functionality AND all of the visualizations you'll find in Tableau in the 1 product. (My company shifted from Tableau to Power BI. At first it seemed like a beta product with lower fidelity. But Microsoft…

We use PowerBI. Wouldn't recommend. I also don't believe it's free, but perhaps that's just the version we use.

I'm curious why you don't recommend PowerBI. What are the pain points?

Re: Ask HN: Tools to visualize data in SQL databases?

#124
post #71

Since superset, metabase and redash was mentioned, I'll share some development insights regarding these OSS projects. Note, I've never used these solutions before, so I can't speak to their quality, but I can speak to how active and invested they appear to be. Looking at recently merged pull requests that were less than 120 days old, apache/superset had 89 unique authors, which is very high, as the following shows: h…

With Redash, their team was aqui-hired by Databricks in June 2020 ( https://blog.redash.io/redash-joins-databricks/ ). They gave their SaaS subscribers a bunch of notice that it was being wound up, and some Redash Community members have picked up some of the load for hosting those customers. Development of the main Redash repo though... has an uncertain future. There are people (Community members) improving things, a…

Thanks for the insights.

Re: Ask HN: Tools to visualize data in SQL databases?

#125
post #60

Earlier quoted context omitted.

I second pandas, and would also highlight seaborn, plotly, and dash as complimentary data visualization libraries. Tableau is fine for what it is, but I've found that the requests from stakeholders often grow to a point where you either can't do it in Tableau or have to move mountains to get it to work... so, in essence, sunk cost fallacy makes tableau millions.

PSA: If you haven't already learned pandas, learn dplyr/ggplot2 instead. Yes, R is a pretty clunky language (but it's closer to lisp than pretty much anything else as popular) but ggplot2 and dplyr are 100% the best currently available way to visualise tables in SQL.

I started with R, but then switched to Python because all pipelines were already written in Python (web-scrapers, some data-processing scripts, REST APIs), so I just learned pandas and it's been fine, although I do think dplyr's syntax is great and I prefer it to pandas'.

Re: Ask HN: Tools to visualize data in SQL databases?

#126

Earlier quoted context omitted.

PSA: If you haven't already learned pandas, learn dplyr/ggplot2 instead. Yes, R is a pretty clunky language (but it's closer to lisp than pretty much anything else as popular) but ggplot2 and dplyr are 100% the best currently available way to visualise tables in SQL.

I started with R, but then switched to Python because all pipelines were already written in Python (web-scrapers, some data-processing scripts, REST APIs), so I just learned pandas and it's been fine, although I do think dplyr's syntax is great and I prefer it to pandas'.

Yeah, totally. I spend a lot more of my time writing Python for anything that isn't data exploration/analysis, for exactly that reason.

I still refuse to learn pandas enough to replace dplyr though, as it's just so painful to use the API compared to how easy this stuff is in R.

Re: Ask HN: Tools to visualize data in SQL databases?

#127
post #10

This may be an unpopular opinion, but if you have US$70/mo to spare, it's hard to beat Tableau for this exact use case. "Connect to an arbitrary database, create a view that joins numerous tables (including foreign tables, via blending) together, load to columnar storage on a local SSD for performance if necessary, add arbitrary derived columns (including well-defined lateral lookups for things like 'annotate this ac…

[deleted]

Re: Ask HN: Tools to visualize data in SQL databases?

#129
post #89

I love Metabase ( https://www.metabase.com/ ) and used it successfully with business users, too (small hedge fund). If it's not too much data, getting it out as .csv and using R + data.table + dplyr, etc; = incredible productivity. See [1] Use Excel, QlikSense and Tableau if Business Users need visualization. Excel pivot tables = OG data reshaping. Resist more complicated solutions : do you really need more than Exce…

If you like the idea of asking questions from your data in plain English, there are also other alternatives to Power BI, like Tableau's Ask Data, ThoughtSpot and Veezoo (free for 5 users). Here is a series of comparison videos to show their strengths: https://veezoo.com/compare-solutions/ Disclaimer: I'm one of the co-founders of Veezoo.

Oh, ThoughtSpot. I spent a couple of months testing that out. At the time, it was hard to deploy amd didn't "scale down". That is, the smallest size we could deploy was something like 750 Gigs.

Also, the UI looked half baked, but I'm sure they have fixed that by now

Re: Ask HN: Tools to visualize data in SQL databases?

#130
post #57

Earlier quoted context omitted.

We use PowerBI. Wouldn't recommend. I also don't believe it's free, but perhaps that's just the version we use.

PowerBI is not free to use, its part of Office 365 (Microsoft 365 or whatever it is called today). I remember 2 companies ago (in 2019) some data scientists had to get a license for it approved.

For personal analytics the desktop version is free. The web service has different pricing tiers (which can be very expensive)

Edit...just to add to this. I personally pay for it and it's $5/mth. Premium capacity can be a lot more...but if the Data Scientist needed a license, it may have only been $5/mth, paltry compared to a data scientist salary.

Post reply on HN