Ask HN: What tools do you use for data munging and merging?

Problems with Alteryx come when you need to share a workflow with other people, or need any functionality outside of the basic tools, or need a loop, or need good documentation, or your internet is down which completely blocks this otherwise-desktop application, or need compatibility with an older version of Alteryx...

I could go on but my point is that beyond the most basic tasks the functionality of Alteryx is limited, the user experience is horrendous and the price isn't justified whatsoever.

Would you say that the main reason to use a software instead of code is to onboard non-coding business partners?

edit: After looking at the website, I can't help but feel that they are surfing on the Asteryx clothing brand to overprice their products...

The python ecosystem is really good here.

I do a lot of exploratory coding in ipython, though the threshold for "switch to a real editor and run git init and poetry new" is pretty low.

Want to munge CSV? stdlib, or pandas(https://pandas.pydata.org/pandas-docs/stable/)

Want to munge JSON? stdlib.

Want to munge a database? pandas, stdlib, anything that speaks sqlalchemy

Want to validate your json/sql/CSV/whatever and have it come in as a bunch of structured classes not std types? jsonschema https://pypi.org/project/jsonschema/, attrs+marshmallow, attrs+cattrs. http://www.attrs.org/en/stable/, https://desert.readthedocs.io/en/latest/ https://marshmallow.readthedocs.io/en/stable/ https://github.com/Tinche/cattrs

Want to transform csv into sqlite? https://github.com/simonw/csvs-to-sqlite

Want to serve sqlite as a REST api? https://datasette.readthedocs.io/en/stable/

Want to stuff simple things into a database really fast? https://dataset.readthedocs.io/en/latest/

Want to flip numerical data around? Numpy.https://docs.scipy.org/doc/numpy/reference/

Want to model it? Scipy https://www.scipy.org/docs.html, pandas

Want to plot it? seaborn https://seaborn.pydata.org/, plotnine https://plotnine.readthedocs.io/en/stable/

Want to futz around with structured data in a sensible way? glom https://glom.readthedocs.io/en/latest/, python-lenses https://github.com/ingolemo/python-lenses.

Want to spit out tabular data? Tabulate https://github.com/astanin/python-tabulate

Want to figure out where not to eat in chicago? built-ins: counter, defaultdict, comprehensions: https://www.youtube.com/watch?v=lyDLAutA88s

There's a LOT you can do pretty fast, and I can more or less hammer out a basic ETL cli script in my sleep at this point.

A project I'm using, which I'm also one of the authors of, is OctoSQL[1].

It allows you to transform and join data from different data sources (including databases like mysql, postgres, redis, and CSV or JSON files, more to come) using a familiar interface - plain SQL.

Other than that, I like Exploratory[2].

And for nested JSON data I use another project of mine, jql[3] - an easier to use (in my opinion) lispy alternative to jq. (originally inspired by a hackernews comment actually)

[1]:https://github.com/cube2222/octosql

[2]:https://exploratory.io

[3]:https://github.com/cube2222/jql

Alteryx is very good, especially if you want to hand off maintenance and operation of data processing flows to non coders.