Become a better data engineer on a shoestring (free resources)
dataengineering.academy
Become a better data engineer on a shoestring (free resources)
1–10 of 26 posts
Re: Become a better data engineer on a shoestring (free resources)
#2Can anyone vouch for the quality on this site? I've been looking for data engineering resources for my team, but don't want to share anything shoddy.
Re: Become a better data engineer on a shoestring (free resources)
#3Can anyone vouch for the quality on this site? I've been looking for data engineering resources for my team, but don't want to share anything shoddy.
My title says Senior Data Engineer* and from what I can tell it looks like a collection of interesting things but nothing so revolutionary. You're better off identifying what aspects your team is weak in and seeking out experts or training in that specifically.
* titles are kinda shit analogs for skill
Re: Become a better data engineer on a shoestring (free resources)
#4I‘m not aware of any decent data engineer for whom money is the bottleneck. It’s always time and mental capacity.
Re: Become a better data engineer on a shoestring (free resources)
#5Can anyone vouch for the quality on this site? I've been looking for data engineering resources for my team, but don't want to share anything shoddy.
Looking over the resources, it doesn't seem worthwhile.
The difficult part is not learning how to code, or work with SQL. The hard part is learning the platform and tooling you need to operate at scale. The ecosystem is full of tools that are great for certain workloads, but terrible for others.
Your best bet is to start by getting an overview of the tools available for your team. If you're using AWS, GCP, or Azure, they each have data engineer-oriented certifications. So take a look at what tools those certification courses cover and start there.
If you are not in a cloud environment, take a look at Apache Airflow, Beam, Storm, or Hadoop. Most of the tooling provided by the big cloud providers is either a rip off of one these products, or is merely a hosted version (i.e, GCP Cloud Composer is managed Airflow).
Re: Become a better data engineer on a shoestring (free resources)
#6I‘m not aware of any decent data engineer for whom money is the bottleneck. It’s always time and mental capacity.
Wouldn't the target audience for these resources be those who are not already decent data engineers?
Re: Become a better data engineer on a shoestring (free resources)
#7Does anyone have a list of high quality data engineering, that aren't free? I don't mind paying, but there is so much out there and most of it is not great.
Re: Become a better data engineer on a shoestring (free resources)
#8Can anyone vouch for the quality on this site? I've been looking for data engineering resources for my team, but don't want to share anything shoddy.
Looking over the resources, it doesn't seem worthwhile. The difficult part is not learning how to code, or work with SQL. The hard part is learning the platform and tooling you need to operate at scale. The ecosystem is full of tools that are great for certain workloads, but terrible for others. Your best bet is to start by getting an overview of the tools available for your team. If you're using AWS, GCP, or Azure,…
This is a great reply! The greatest difficulty I face is convincing my data engineers to stop reinventing the wheel and leverage existing (and appropriate!) tools. Writing code to directly manage events and batch jobs should be a thing of the past by now. Pick a tool. Configure your jobs, retry policy, etc. and be done with it.
Re: Become a better data engineer on a shoestring (free resources)
#9Can anyone vouch for the quality on this site? I've been looking for data engineering resources for my team, but don't want to share anything shoddy.
Looking over the resources, it doesn't seem worthwhile. The difficult part is not learning how to code, or work with SQL. The hard part is learning the platform and tooling you need to operate at scale. The ecosystem is full of tools that are great for certain workloads, but terrible for others. Your best bet is to start by getting an overview of the tools available for your team. If you're using AWS, GCP, or Azure,…
I really got a lot of use out of taking the GCP data engineer course on coursera (the one by google aimed at the cloud cert) and then later taking the actual certification.
With that being said it was very focused on BigQuery, and my impression is that it is that all their certs are now basically different variations of a kubernetes certification.
Re: Become a better data engineer on a shoestring (free resources)
#10Any tips for getting started as a data engineer? I actually like cleaning stuff so maybe it's the career for me