Earlier quoted context omitted.
Wow - that sunroof thing is cool, I bet that was a fun area to work on. There’s a lot to be said for data engineering when done at scale - systems design, devops, cloud engineering etc. I’m not sure that’s going yet… Building the individual units (reliable ETL pipelines) might be on the chopping block in the near future - I’m not convinced there’s 100 competing ways to build these. Still - best way to get started is…
> that sunroof thing is cool, I bet that was a fun area to work on Yeah, it was (and is) really cool! To be clear, I didn't work on that, just something similar but much more primitive and with much simpler datasets. Google did a really good job there. > locally deployed Airflow instance I'm pretty comfortable with basic ETL stuff, but never used Airflow. Will have to look into that, thanks! > May I ask the same of w…
Yeah, I'm agreed on most of the AI stuff - becoming a more architect-shaped engineer is now more important than ever.
I've been flicking through a copy of https://www.amazon.co.uk/Structured-Design-Fundamentals-Disc... lately to get my brain into gear on this subject.
Thanks for all the web dev advice and tech callouts. Some real neat examples there which I knew nothing about.
Regarding niches, I think as a data engineer, becoming fluent in an interactive front-end data-viz library might be the slam dunk that I'm probably looking for.
Having looked at D3, I am quite impressed! Do you have any experience or knowledge of this lib?
Again, thank you for sharing :)