Live data from Hacker News

Viewing profile — thibautdr

thibautdr

HN member
Joined
Wed, Mar 27, 2024, 10:11 PM UTC
HN karma
27
Public activity
20 items

About thibautdr

No profile information was provided.

Recent public activity

  1. story
    Show HN: Amphi, visual data transformation based on Python

    Hi everyone, I’d like to introduce Amphi, a free and source-available visual data transformation tool. Amphi is a low-code tool designed for data preparation, manipulation and ETL …

  2. comment
    Comment #40781020

    I sure will!

  3. comment
    Comment #40781001

    I wrote an article questioning the use of Pandas for ETL. I invite you to read it: https://medium.com/@thibaut_gourdel/should-you-use-pandas-fo...

  4. comment
    Comment #40732611

    Yes, there are similarities, but Elyra allows you to develop orchestration pipelines for Python scripts and notebooks, so you still have to write your own code. With Amphi, you des…

  5. comment
    Comment #40732594

    Using Modin, deploying the pandas code on Dask should be possible: https://modin.readthedocs.io/en/stable/development/using_pan...

  6. comment
    Comment #40732561

    Thanks for the great questions: 1. As far as I know, there isn't a "standard" file format for low-code pipelines. 2. Some formats are more readable than others. YAML, for example, …

  7. comment
    Comment #40731350

    Great, thanks for sharing. I was familiar with Dask and cudf separately but not this one.I was planning to implement dask support through Modin but I'll definitely take a look at d…

  8. comment
    Comment #40730779

    Thank you! Inputs components are pre-built for now but the ability to add custom inputs is coming soon!

  9. comment
    Comment #40729651

    Hey, Amphi's developer here. Those two tools are great, big fan of dlt myself :) However, Amphi is a low-code solution while those two are code-based. Also, those two focus on the …

  10. comment
    Comment #40729190

    Hey, not sure I get your point here. I believe the abstraction provides what you're describing. You can swap a file input with a table input without touching the rest of the compon…

  11. comment
    Comment #40725796

    Thanks for pointing that out, it's actually mentioned (Extract, transform and load ...) in the very first sentence below the tagline, but if you didn't get it then it's not clear.

  12. comment
    Comment #40725594

    Thanks! Don't hesitate to give it a try and reach out if you need anything :)

  13. comment
    Comment #40725563

    Thanks for your question. Amphi generates Python code using Pandas and can scale on a single machine or even multiple machines using Modin, but the process is manual for now. Futur…

  14. comment
    Comment #40725527

    Amphi generates Python code, so you can definitely orchestrate them through Airflow but it doesn't generate "Airflow code" so to speak. Now, in the future we might develop Airflow …

  15. comment
    Comment #40724727

    Thanks! Being based on JupyterLab also allows Amphi to benefit from the vast ecosystem of extensions already available, such as the Git extension or using different file systems (S…

  16. comment
    Comment #40724493

    Thanks for your comment! I do believe it depends on who you ask and ultimately both will co-exist. I also think low-code solutions democratize access to ETL development offering a …

  17. comment
    Comment #40724299

    Thanks for your comment, don't hesitate to share your use case! Also, you can reach out on Slack if you have any questions or need help.

  18. comment
    Comment #40724242

    Thanks for your comment, those are valid points. I come from the industry, having worked for an ETL vendor for 6 years. I've personally witnessed a need for a low-code (graphical) …

  19. comment
    Comment #40724101

    Hi, thanks for your question. I'm not familiar with Windmill, but after checking it seems to be an open source developer platform to build applications. Amphi is a low-code tool to…

  20. comment
    Comment #40723576

    Hi everyone, thanks for posting Amphi :) To give some context, Amphi is a low-code ETL tool for both structured and unstructured data. The key use cases include file integration, d…