Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

mprove.io

Post reply on HN
Print view View original

31–40 of 46 posts

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#31

Post by sandGorgon »

akalitenya wrote: post #24
sandGorgon wrote: post #5

this is super cool and i would pay for this. Bigquery is a cheap alternative to a lot of the mobile analytics tools. Quick point however - why do you need a new database ? You can use a table inside bigquery itself. It seriously reduces the dependencies required.

Mprove creates permanent derived tables in BigQuery if the user wants it. But you can't use BigQuery for OLTP.

No - I'm talking about MySQL being needed as a dependency.

https://github.com/mprove-io/mprove/blob/master/deploy/docke...

Can you not use biquery database itself. Create tables for your internal use instead of MySQL ?

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#32

Post by akalitenya »

segah wrote: post #28
akalitenya wrote: post #22

Earlier quoted context omitted.

Yes it should, there is special "unnest" parameter for fields in BlockML reference - https://mprove.io/docs/blockml/fields/dimension

wow, impressive! how about symmetric aggregates (e.g. being able to do correct summation/aggregation on numeric values despite a one_to_many join)?

Yes, look at this page - https://mprove.io/docs/blockml/fields/measure. You need to specify measure "type" and "sql_key" that will be used to avoid counting duplicates.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#33

Post by akalitenya »

sandGorgon wrote: post #31
akalitenya wrote: post #24

Earlier quoted context omitted.

Mprove creates permanent derived tables in BigQuery if the user wants it. But you can't use BigQuery for OLTP.

No - I'm talking about MySQL being needed as a dependency. https://github.com/mprove-io/mprove/blob/master/deploy/docke... Can you not use biquery database itself. Create tables for your internal use instead of MySQL ?

The MySQL image specified in the docker-compose file you mentioned. It is used for the internal data of the Mprove application (users, projects, members, etc.). Each user action in the web client (angular) can initiate several queries to this database through an backend request. Delays here are crucial. network latency - so you need to keep the database as close as possible to your server side, read / write delays - for queries such as finding a user / setting a username / creating a member, etc.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#34

Post by g14i »

segah wrote: post #20
g14i wrote: post #13

Earlier quoted context omitted.

Please, do you mind sharing what exactly makes a BI solution difficult/struggling for 1,000+ users? Is it something technically related or more business/feature related?

mmmm, both! Tech-wise, data stacks are complicated. Hundreds of pre-existing tools, with many owners and diverse interests. BI is where they all meet. With some organizations its easy to build a central data warehouse, but in a large corporate environment that's a dream. Imagine how many data warehouses an organization like Microsoft might have? OK, MS is too large - how about Expedia or Zillow? Thousands? So you con…

Wow. Thanks for putting great energy for this answer. Great perspective.

Again, if you don't mind sharing, what would be a dream solution for you, something that could solve most of the problems you mentioned?

I understand that some of them are cultural problems (like people no talking to each other), but maybe somehow a technology could solve this?

If you could think of a new approach, what would that be?

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#35

Post by sandGorgon »

akalitenya wrote: post #33
sandGorgon wrote: post #31

Earlier quoted context omitted.

No - I'm talking about MySQL being needed as a dependency. https://github.com/mprove-io/mprove/blob/master/deploy/docke... Can you not use biquery database itself. Create tables for your internal use instead of MySQL ?

The MySQL image specified in the docker-compose file you mentioned. It is used for the internal data of the Mprove application (users, projects, members, etc.). Each user action in the web client (angular) can initiate several queries to this database through an backend request. Delays here are crucial. network latency - so you need to keep the database as close as possible to your server side, read / write delays -…

Hmm..I would prefer to not have it. In production, managing database persistence is very hard. Especially when you go down the kubernetes road.

I would take higher latency, but avoid pulling in a whole database infrastructure.

Plus a huge number of us use postgresql..so that becomes another set of a mess. I would strongly urge you to do this on the same bigquery database that you would connect to anyways.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#36

Post by akalitenya »

endlessvoid94 wrote: post #26
akalitenya wrote: post #17

Earlier quoted context omitted.

> Some of the Old LookML syntax is an exact copy. I met the online Looker demo a few years ago when I was looking for a business intelligence tool for another project. Looker has a closed source code, so I did not see what algorithm is used to build queries. I kept those LookML features that I understood and liked. However, in some places LookML is confusing (for example - references and aliases). I made them differe…

I used Looker for years and have always wanted a product priced less for the enterprise and more for developers. I even prototyped a similar tool myself. Congratulations on the launch. I will be using mprove and will give you feedback along the way. EDIT: I would love to get in touch with you and learn more about what your plans are. Possibly collaborate if you'd be open to that. I'm dpaola2@gmail.com -- I couldn't f…

Thank you, I sent you an email. I am open to any suggestions - akalitenya@mprove.io.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#37

Post by akalitenya »

danpalmer wrote: post #11

The headlining feature seems to be that it has a dark and a light theme. This isn’t very encouraging for the rest of the product. I’m sure there’s a lot of good stuff here, but themes aren’t important enough to be the first feature mentioned.

Thank you, I plan to add more videos on the page.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#38

Post by akalitenya »

pplonski86 wrote: post #9
segah wrote: post #7

@akalitenya are you even in the clear with this? Some of the Old LookML syntax is an exact copy. But more importantly, the challenge for any such tool is to go beyond use by 2-3 people. At 2-3 people anything will work. Where BI tools (open source and close source) struggle is scale: having all the right features for, essentially, a group of users who actually don't know how to work with data (did I just say that alo…

What is the main reason that LookML is complicated product?

LookML supports different databases as sources. Sometimes Looker adds its own LookML parameters for database - which makes it difficult to read the documentation.

For analysts who work with SQL queries, any wrapper on a language seems complicated. But in the case of LookML and BlockML, in a couple of days you already begin to understand how this works, and after a week you can not tear yourself away from using the product.

Looker and Mprove are in the middle between fully custom DIY solutions and simple query generators tools.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#39

Post by zackt »

akalitenya wrote: post #17
segah wrote: post #7

@akalitenya are you even in the clear with this? Some of the Old LookML syntax is an exact copy. But more importantly, the challenge for any such tool is to go beyond use by 2-3 people. At 2-3 people anything will work. Where BI tools (open source and close source) struggle is scale: having all the right features for, essentially, a group of users who actually don't know how to work with data (did I just say that alo…

> Some of the Old LookML syntax is an exact copy. I met the online Looker demo a few years ago when I was looking for a business intelligence tool for another project. Looker has a closed source code, so I did not see what algorithm is used to build queries. I kept those LookML features that I understood and liked. However, in some places LookML is confusing (for example - references and aliases). I made them differe…

Looker is easy to cr*ck (terrible security) and you can deploy it everywhere for your BI needs. So we dont need similar product.

Re: Show HN: Open-Source Business Intelligence for BigQuery – Looker Alternative

#40

Post by segah »

akalitenya wrote: post #38
pplonski86 wrote: post #9

Earlier quoted context omitted.

What is the main reason that LookML is complicated product?

LookML supports different databases as sources. Sometimes Looker adds its own LookML parameters for database - which makes it difficult to read the documentation. For analysts who work with SQL queries, any wrapper on a language seems complicated. But in the case of LookML and BlockML, in a couple of days you already begin to understand how this works, and after a week you can not tear yourself away from using the pr…

@akalitenya the way I used to explain it to the technical audience is: would you rather program in a low-level language like C or in a high-level language like Java, Python, or Node? Well, if you are writing compilers, you might choose C, but if you are doing almost everything else, you want to choose a language that fits the purpose: python for data science, Node for web backend, etc. Same with LookML - it provides a language that is ideal for doing analytics in-database. SQL is a language for extracting and summarizing data - it is not an ideal language to preserve business logic in just as C would not be an ideal language to preserve backend web dev. work in.
Post reply on HN