Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Source Python ETL

amphi.ai

Post reply on HN
Print view View original

1–10 of 127 posts

Re: Open Source Python ETL

#3

Post by thibautdr »

Hi everyone, thanks for posting Amphi :)

To give some context, Amphi is a low-code ETL tool for both structured and unstructured data. The key use cases include file integration, data preparation, data migration, and creating data pipelines for AI tasks like data extraction and RAG. What sets it apart from traditional ETL tools is that it generates Python code that you own and can deploy anywhere. Amphi is available as a standalone web app or as a JupyterLab extension.

Visit the GitHub: https://github.com/amphi-ai/amphi-etl Give it a try and let me know what you think

Re: Open Source Python ETL

#5

Post by thibautdr »

v3ss0n wrote: post #4

What's the difference compare to Windmill.

Hi, thanks for your question. I'm not familiar with Windmill, but after checking it seems to be an open source developer platform to build applications. Amphi is a low-code tool to develop data pipelines (or ETL pipelines).

Re: Open Source Python ETL

#6

Post by tayloramurphy »

I'm curious as to the story of how things like this come to be. It seems like there are already a ton of "open source python ETL" tools on the market. Was this a passion project by the author? Was this born out of academia? Was there a specific problem they were trying to solve that others didn't? It's not necessary to answer these questions in the docs but it is useful for folks who may be familiar with the other options out there.

Re: Open Source Python ETL

#7

Post by esafak »

thibautdr wrote: post #5
v3ss0n wrote: post #4

What's the difference compare to Windmill.

Hi, thanks for your question. I'm not familiar with Windmill, but after checking it seems to be an open source developer platform to build applications. Amphi is a low-code tool to develop data pipelines (or ETL pipelines).

Windmill is a low-code workflow engine: https://www.windmill.dev/flows

Re: Open Source Python ETL

#8

Post by thibautdr »

tayloramurphy wrote: post #6

I'm curious as to the story of how things like this come to be. It seems like there are already a ton of "open source python ETL" tools on the market. Was this a passion project by the author? Was this born out of academia? Was there a specific problem they were trying to solve that others didn't? It's not necessary to answer these questions in the docs but it is useful for folks who may be familiar with the other op…

Thanks for your comment, those are valid points. I come from the industry, having worked for an ETL vendor for 6 years. I've personally witnessed a need for a low-code (graphical) ETL for Python environments. In short, traditional ETLs are GUI ETLs for Java environments while modern data tools are either focusing on the EL part or are code-oriented (dbt). With Amphi, I want to offer a low-code graphical alternative to develop Python-based pipelines. I also believe that modern data stack tools don't effectively address use cases for unstructured data, which is another focus of Amphi (with extensive file integration and RAG support).

Re: Open Source Python ETL

#10

Post by iblaine »

Low code ETL tools (informatica, Appworx, talend, pentaho, ssis) were the original services for ELT/ETL. A lot of progress was made to go towards ETL-as-code starting with Airflow/Luigi. Going back to low code seems backwards as this point.

(I have used all of the above tools in my 15+ yr career. Code as ETL was a huge industry shift)

Post reply on HN