Show HN: We wrote a book to help data people build scalable analytics stacks
1–10 of 18 posts
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#2Hey HN. This is something we've been working on for the last three months over at Holistics.
If you're a data analyst, data engineer, or a founder setting up a data analytics stack for the first time, this is a book that will give you a soup-to-nuts overview of an entire field.
Like most books about data analytics, this assumes some amount of technical competence.
Unlike most books in the space, this is mostly about first principles. About the ideas behind the tools, not the tools themselves.
The hope is to give you 'just enough to not get lost'. And the book is written to be read within 2 hours of reading — in some cases, no more than two sittings!
There's probably more than a hundred hours of research and writing that went into this. I'm looking forward to read your comments.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#3Nice, the illustrations look pretty good. I just took a look at the table of contents, it seems to cover a lot of my questions about data analytics for a product guy like me, will spend some time reading this weekend.
Sending to my data team btw, thanks for sharing
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#4I've been looking for something like this for a while. Most of the time when I go online to search for resources on building analytics stack, most of the content is biased towards the vendor's preferred way of doing things. This looks like it will give me a high-level understanding to the why of the proposed approach.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#5I'm a data engineer at a large corporation. At my current company, we use Pentaho to extract and transform our data from Oracle daily. The transformed data is loaded to a staging database where we then model it to a star-schema. Then the final results are bulk loaded to an on-prem DW. The process takes hours to complete.
I’m interested in moving to a model that continually extracts changed data to a data lake, then using the power of a cloud data warehouse to read those files and perform the transformations and modeling in SQL. I guess that's the ELT concept that you mentioned in the book's summary.
Goal being to reduce the latency and allow for the possibility of more frequent batches, as well as making the process more accessible to my team with strong SQL skills and being able to adapt faster to changing business needs.
This book looks like a good foray for me to get a glimpse into those new process. Thanks for putting it together.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#6I'm a data engineer at a large corporation. At my current company, we use Pentaho to extract and transform our data from Oracle daily. The transformed data is loaded to a staging database where we then model it to a star-schema. Then the final results are bulk loaded to an on-prem DW. The process takes hours to complete. I’m interested in moving to a model that continually extracts changed data to a data lake, then u…
I'm one of the authors of the book. Yes you're right. The book outline the transition from the "old world of BI" to the "new world of BI". If you read through the book, you'll see the biases clearly stated there:
- ELT over ETL
- Data Modeling is crucial as part of BI workflow
- Cloud DW over on-premise DW (but this depends on your org's requirements)
- SQL reporting (Redash, Metabase, Looker, Holistics) over non-SQL reporting (Tableau, Qlik)
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#7I've been looking for something like this for a while. Most of the time when I go online to search for resources on building analytics stack, most of the content is biased towards the vendor's preferred way of doing things. This looks like it will give me a high-level understanding to the why of the proposed approach.
This is exactly why we started out writing this book. We actually spoke with a lot of customers, and a fair share of them sharing the same thing: They found a lot of how-to on the web, but none of which is comprehensive, and goes deep into the "why" and the "history" of BI.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#8Really well done! The content looks like a great balance. I'm an engineer working a bit in data engineering, and I find the content of the book relevant to me.
I like how you explain why the methodologies of the pre-cloud era still have lessons learned to apply to today, but implementation best practices have changed thanks to the cost model of the cloud.
The section that stood out the most to me, strangely, was not anything to do with the technology or analytics stack. It was in Chapter 3 – Data Modeling Layer and Concepts where you discuss the dynamic between the CEO and the data analyst and the data. This really articulated quite well how our own dynamic functions at our current company. Even with our current data warehouse, our BI team is a bottleneck, and it is something becoming more and more apparent to me. It is my primary motivation in seeking out how best to re-architect our analytics stack.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#9I'm a data engineer at a large corporation. At my current company, we use Pentaho to extract and transform our data from Oracle daily. The transformed data is loaded to a staging database where we then model it to a star-schema. Then the final results are bulk loaded to an on-prem DW. The process takes hours to complete. I’m interested in moving to a model that continually extracts changed data to a data lake, then u…
I'm one of the authors of the book. Yes you're right. The book outline the transition from the "old world of BI" to the "new world of BI". If you read through the book, you'll see the biases clearly stated there: - ELT over ETL - Data Modeling is crucial as part of BI workflow - Cloud DW over on-premise DW (but this depends on your org's requirements) - SQL reporting (Redash, Metabase, Looker, Holistics) over non-SQL…
I'd toss SeekWell[1] in the "new world" category, but we've taken a different approach. Instead of forcing people to use a whole separate platform for BI, we decided to tightly integrate with the tools people were already going to for data (e.g. Google Sheets, Excel, Slack, etc.). We've found teams stay better informed when the data is in places they're already hanging out.
Re: Show HN: We wrote a book to help data people build scalable analytics stacks
#10Great book and spot on of the problem statement. One interesting note to point out is I didn't see any mention of testing your data models or version control? This should be part of the process of the modern analytics stack to ensure data quality.
A suggestion of approaches and tools could be useful. Whether it's via tools such as dbt for expected field values or with frameworks such as GreatExpectations. What happens to data that doesn't conform to expected values? How should you handle it? and how can the testing process be automated? This forms an important part of ensuring data quality and reliability of the analysed output.