Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to level up beyond ETLs

ezzeriesa.com

Post reply on HN
Print view View original

1–10 of 37 posts

Re: How to level up beyond ETLs

#3

Post by zurfer »

What an honest an refreshing perspective! I loved the quote from Donald Knuth:

    If you find that you’re spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you’re spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice.

Re: How to level up beyond ETLs

#5

Post by omegabravo »

johngladtj wrote: post #4
floorballchamp wrote: post #2

Perhaps start by explaining what ETL stands for and what it is.

Extract Transform Load

This is correct from my understanding, and glossing over the article and links is precisely what they're talking about.

Pipeline of taking data from one system, transforming it and loading it into another. There is a whole industry full of software products that facilitate this such as MuleSoft. It is referred to as ETL

I was surprised to see your succinct correct answer greyed out.

Re: How to level up beyond ETLs

#7

Post by dijksterhuis »

jstx1 wrote: post #6

Isn't this very very basic? I'm not seeing the "beyond" element, it's more like "How to level up to almost not zero".

I was a bit confused myself.

Then I realised this is probably a good reference for “things to be think about” as a decent-ish data engineer.

Re: How to level up beyond ETLs

#8

Post by dijksterhuis »

Not sure if it’s just me, but the whole mention of the Stitch Fix blog post and ‘democratising data pipelines’ really threw me off.

> The idea here is the engineering team creates ‘Lego blocks’ that consumers then assemble into end-to-end data pipelines.

Might be my personal biases, but it doesn’t really seem to set the scene for the rest of content?

The rest of it was good enough common sense “non exhaustive list of stuff to always be working on” as a data engineer or something.

Re: How to level up beyond ETLs

#9

Post by floorballchamp »

omegabravo wrote: post #5
johngladtj wrote: post #4

Earlier quoted context omitted.

Extract Transform Load

This is correct from my understanding, and glossing over the article and links is precisely what they're talking about. Pipeline of taking data from one system, transforming it and loading it into another. There is a whole industry full of software products that facilitate this such as MuleSoft. It is referred to as ETL I was surprised to see your succinct correct answer greyed out.

Thanks for your explanation.

Maybe the greyed out was because just resolving the acronym is not very helpful if from the words it's not clear what that is. Like in this case.

Post reply on HN