Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a data team at a mid-stage startup

erikbern.com

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Building a data team at a mid-stage startup

#31

Post by herodoturtle »

IMTDb wrote: post #9

What would be the name of the position/profile of someone in charge of building the data warehousing architecture/ETL pipelines? I my view, they need make sure the warehouse model is a correct representation of the business and that it can be leveraged to answer basic or not-so-basic questions using SQL. They also need to promote it's usage internally by ensuring it is accessible and easy to use and guide other team…

You pretty much described my job in a nutshell, and they call me "the database guy".

Re: Building a data team at a mid-stage startup

#32

Post by czep »

PragmaticPulp wrote: post #29
czep wrote: post #24

This is so eerily familiar I swear I've had many of these exact conversations word for word. The only way this doesn't turn into a complete nightmare of a cluster is if the exec team "gets it". If so, you just might stand a chance at building a data team that gels with the rest of the org. But if the exec team simply hired you for window-dressing, expect to be treated like a scapegoat and a punching bag. Any mistakes…

This applies to most specialties. Companies tend to have a few teams that lead the charge and expect everyone else to follow. Knowing which teams get the authority and which teams are along for the ride at a company is important for knowing what your job experience will look like. > However, if E-team does give you the authority to call Product's bullshit, and tell Finance to stuff it, and not take direction from Eng…

Thanks for the tips! One mantra I've tried when starting at a new job is "for the first 3 months say yes to everything, for the next 3 months say no to everything." The idea is you first immerse yourself in everything, to find out what works and what doesn't. Then you dedicate time to fix the broken processes so that hopefully when you hit 6 months your team is better positioned to be more efficient. Obviously you can't be too rigid, but it seemed to work for me when I had buy in. Curious if you think that approach sounds good.

Re: Building a data team at a mid-stage startup

#35

Post by plank_time »

This is probably the singly best written and most realistic article I’ve read on HN ever and I’ve been on HN for a long long time. It’s so realistic I wonder if the author took it from his diary or something. Everything about it is supersaturated with authenticity and teaches better than any other article I’ve read. Kudos to the author, and I would love to see this style of article take off.

Re: Building a data team at a mid-stage startup

#36

Post by PragmaticPulp »

czep wrote: post #32
PragmaticPulp wrote: post #29

Earlier quoted context omitted.

This applies to most specialties. Companies tend to have a few teams that lead the charge and expect everyone else to follow. Knowing which teams get the authority and which teams are along for the ride at a company is important for knowing what your job experience will look like. > However, if E-team does give you the authority to call Product's bullshit, and tell Finance to stuff it, and not take direction from Eng…

Thanks for the tips! One mantra I've tried when starting at a new job is "for the first 3 months say yes to everything, for the next 3 months say no to everything." The idea is you first immerse yourself in everything, to find out what works and what doesn't. Then you dedicate time to fix the broken processes so that hopefully when you hit 6 months your team is better positioned to be more efficient. Obviously you ca…

Good advice as long as you don't take it too literally.

The most important thing is to work closely with your manager on expectations. If someone from another department comes to you with a proposal, an ask, or a directive, you don't want to say yes without first consulting with your manager. Depending on company politics, some managers might try to rope new employees into doing work that isn't actually part of their job description.

Discovering expectations and then proactively managing those expectations is key in any role.

Re: Building a data team at a mid-stage startup

#37

Post by Orou »

IMTDb wrote: post #9

What would be the name of the position/profile of someone in charge of building the data warehousing architecture/ETL pipelines? I my view, they need make sure the warehouse model is a correct representation of the business and that it can be leveraged to answer basic or not-so-basic questions using SQL. They also need to promote it's usage internally by ensuring it is accessible and easy to use and guide other team…

I would also vote for "data engineer" (it's my current job title).

You very likely don't want a data scientist to be doing a data engineer's job (and they probably don't want to be doing it themselves!). While there are similarities, data engineering tends to be a lot closer to software development than data science. If you're advertising for a data scientist role, don't expect them to be happy if 80% of their job is writing ETL scripts and cleaning datasets.

I think the reason there has been a flattening in data scientist job growth more recently is that lots of companies hired data scientists to build cool ML applications but had no infrastructure in place to support advanced data analysis. These companies didn't realize they needed to walk before they could run, and that what they really wanted was data analysts and engineers to build the foundation for a strong data science function.

Tools like dbt have been great for advancing an ELT approach to managing data pipelines, where modeling for BI tools, business users, and data scientists alike can all happen in the warehouse and ensure consistency in data usage across the company.

Re: Building a data team at a mid-stage startup

#38

Post by mjirv »

IMTDb wrote: post #9

What would be the name of the position/profile of someone in charge of building the data warehousing architecture/ETL pipelines? I my view, they need make sure the warehouse model is a correct representation of the business and that it can be leveraged to answer basic or not-so-basic questions using SQL. They also need to promote it's usage internally by ensuring it is accessible and easy to use and guide other team…

Analytics Engineer is a clear one for this, as teej said.

The title is strongly associated with the dbt community, so it could imply you’re using dbt for your data modeling (not necessarily a bad thing, as it sounds like it would be a good tool for your use case).

Re: Building a data team at a mid-stage startup

#39

Post by maileslin »

plank_time wrote: post #35

This is probably the singly best written and most realistic article I’ve read on HN ever and I’ve been on HN for a long long time. It’s so realistic I wonder if the author took it from his diary or something. Everything about it is supersaturated with authenticity and teaches better than any other article I’ve read. Kudos to the author, and I would love to see this style of article take off.

Erik is a legend in the modern data world. Wrote Luigi and built Spotify's first recommendation engine. He has the ground-level experience to lean on
Post reply on HN