Guide to data tools landscape for developers
1–10 of 38 posts
Re: Guide to data tools landscape for developers
#2Now I'll be thinking of "L" in ETL as "Land" and not "Load".
Although the article doesn't propose that but uses a lot of "Land" terminology.
"Load" => "load where? or FROM where?" - ambiguous
"Land" => "land where?" - clear
Re: Guide to data tools landscape for developers
#3Data is the new oil
Re: Guide to data tools landscape for developers
#4Apache Avro has 2 encodings: binary AND json.
Re: Guide to data tools landscape for developers
#5> A data warehouse on the other hand is an OLAP database and is optimized to work on columns
A bit of a pedantic nit here: a data warehouse is a usage pattern. It’s not necessarily tied to any specific technology, however it is commonly implemented with OLAP systems like Snowflake, BigQuery, etc. But there’s nothing stopping you from building out your data warehouse in Postgres or MySQL. If you’re stitching together disparate datasets to build a unified model for analytics, you’ve got yourself a data warehouse no matter what system it lives on.
Re: Guide to data tools landscape for developers
#6Apache Avro has 2 encodings: binary AND json.
Are you sure? I can't find any reference indicating that.
Update: Huh, TIL https://avro.apache.org/docs/%2B%2Bversion%2B%2B/specificati...
Re: Guide to data tools landscape for developers
#7It's a good all-round primer, well written.
Would love to hear more about larger-than-memory tasks and running local Dask clusters. I processed many-a-dataset that way that would normally make pandas choke.
Re: Guide to data tools landscape for developers
#8This was great, thanks for writing it up. Even as someone in the data space for a long time, I learned quite a bit.
Re: Guide to data tools landscape for developers
#9Now I'll be thinking of "L" in ETL as "Land" and not "Load". Although the article doesn't propose that but uses a lot of "Land" terminology. "Load" => "load where? or FROM where?" - ambiguous "Land" => "land where?" - clear
always understood "land" as the raw data layer though, i.e only the bronze layer in the article.
Re: Guide to data tools landscape for developers
#10Great post! Also, I dig your site -- it's attractive and highly usable, and the "personal" toggle in the footer is a clever affordance I haven't seen before for separating professional content.