Earlier quoted context omitted.
Excel has an entire ETL engine called PowerQuery tucked up under the hood. Originally part of SQL Server Analysis Services, it got strapped into Excel about a decade ago, and more recently tons of other products[1]. The Web[2] connector is what you're after. It can consume a variety of formats, including csv, xml, and json. It supports HTTP Basic Auth pretty well, but if you want to put the data source behind oauth i…
Thank you for this!. It's why i continue to read HN.
I'm not an expert on it, but I am a technical person currently working on the business side of the IT fence. And I frequently do (data-engineering heavy) consulting for clients who are similarly on the business side of IT. So none of the nifty ETL tools, data storage capabilities, or computing environments I have available when I'm on the IT side.
Power Query is a godsend in that scenario. It's a well featured ETL engine with integrations for a variety of systems, services, and databases (including generic JDBC/ODBC support). And has the ability to use direct HTTP calls when that's more appropriate/useful.
It also stores the data internally in a highly-compressed and optimized columnar store, independent of the "Excel data" on sheets. Which you can then either sync to a worksheet, or leave it in a state where the raw data isn't visible but can be accessed through a pivot table connected to it. So you can abuse it for far heavier work than you'd expect to be able to do in Excel.
And it's already there, sitting on virtually every business person's computer everywhere. Completely sidesteps the security, IT, procurement, and legal hassles you have to jump through to get a proper system. Not to mention user training - you can architect things in a way where all users have to do is maybe tweak a cell or two, then hit the Refresh button that's in the Ribbon. So the complexity of "learning something new" is completely absorbed on your side, and you're free of pesky support questions and hassles since you isolated them from being the new stuff (also making it harder for "accidental/I didn't press anything!" changes from users).
It's not perfect by any means and has a number of warts, usually falling short of more purpose-built solutions when those are options. But it's still pretty solid, and on balance has saved me far more frustration than it's caused. And without having to dip into the dreaded world of macros and VBA.