Live data from Hacker News

Use SQL for data pipelines, Airflow Sucks for MLOps, Draw plots on the terminal

ingestthis.substack.com

1–4 of 4 posts

Re: Use SQL for data pipelines, Airflow Sucks for MLOps, Draw plots on the terminal

#3
I know it’s not the focus of the article, but a personal pet-peeve of mine is when pandas gets mentioned in articles and discussions regarding alternatives to sql and data pipeline tools. Pandas was never meant to be a replacement for sql, it was meant to be a replacement for excel in highly dynamic quickly changing financial and econometric excel models. It just so happens there’s a lot of overlap between this and more proper data pipeline tools. But for its original purpose I don’t know if anything that still comes close to the useability of pandas.

Re: Use SQL for data pipelines, Airflow Sucks for MLOps, Draw plots on the terminal

#4
post #3

I know it’s not the focus of the article, but a personal pet-peeve of mine is when pandas gets mentioned in articles and discussions regarding alternatives to sql and data pipeline tools. Pandas was never meant to be a replacement for sql, it was meant to be a replacement for excel in highly dynamic quickly changing financial and econometric excel models. It just so happens there’s a lot of overlap between this and m…

You're right, but pandas today is used a bit everywhere, and in data pipelines too, even if it wasn't its main goal.