Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a data team at a mid-stage startup

erikbern.com

Post reply on HN
Print view View original

61–70 of 91 posts

Re: Building a data team at a mid-stage startup

#61

Post by WastingMyTime89 »

czep wrote: post #24

This is so eerily familiar I swear I've had many of these exact conversations word for word. The only way this doesn't turn into a complete nightmare of a cluster is if the exec team "gets it". If so, you just might stand a chance at building a data team that gels with the rest of the org. But if the exec team simply hired you for window-dressing, expect to be treated like a scapegoat and a punching bag. Any mistakes…

> However, if E-team does give you the authority to call Product's bullshit, and tell Finance to stuff it, and not take direction from Eng leads, then you actually might be able to accomplish something really cool.

So what's the business case for having a data team independent of product, business and engineering?

Because as I see it the data team is a support function not q core part of the business. I'm sure it can be cool for you but if you are at odd with all the people actually creating value, what exactly do you bring to the table?

Re: Building a data team at a mid-stage startup

#62

Post by dijksterhuis »

sails wrote: post #58
Orou wrote: post #37

Earlier quoted context omitted.

I would also vote for "data engineer" (it's my current job title). You very likely don't want a data scientist to be doing a data engineer's job (and they probably don't want to be doing it themselves!). While there are similarities, data engineering tends to be a lot closer to software development than data science. If you're advertising for a data scientist role, don't expect them to be happy if 80% of their job is…

IMO data engineer roles are further subset into: 1. kafka / streaming oriented software engineering 2. data warehouse and ETL/ELT development for analytics

A good data engineer understands and can work with both of these.

They're both "data in, data out" mental models that are part of the Lambda architecture which every data engineer should at least know about [0].

But if you want a specialist streaming person to optimise all the streaming pipelines, then sure hire a specialist.

[0]: https://en.m.wikipedia.org/wiki/Lambda_architecture

Re: Building a data team at a mid-stage startup

#63

Post by zippy5 »

lifeisstillgood wrote: post #54
zippy5 wrote: post #51

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place. This company makes 10M and spends 3M on the team and infrastructure to make data a core competency? A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten an…

As someone who reaches for code if they need to blow their nose, what is a 3rd party vendor going to supply that a “English-to-SQL translators” wont do? (I have not finished the article, but the idea that devs / data scientists can be replaced by some vendors makes me wonder what I have missed) Edit: Also love the Uranium quote :-)

So my assumption is that for a given business model, like e-commerce or Saas business much of the highest value analysis is fairly standardized and can be templated. For example breaking down conversion rate by weekly cohort is something that can be pretty easily be done in google analytics.

The problem with English to sql translators or most coders in general are the assumptions we make, in particular about the underlying data. For example, say we want a join two tables, so we write a query to join on two columns and often call it correct which it is from a logical or schema perspective it is. However, null values, defaults like 0, many to one relationships vs one to one relationships, issues with instrumentation such as networking timeouts or bot detection, etc all can impact the down stream metrics. My point is that when there are 500 lines of sql in a query such as those mentioned the article, there’s a lot of ways to be mostly correct but to cumulatively be wrong.

Like many popular enough open source tools, 3rd party vendors get battle tested, issues get found before you, and they can justify devoting more resources to rigorously ensure correctness than the average analyst has the time or energy todo because their business depend on you trusting the outputs.

I’m not saying you couldn’t do all this yourself. But given the sheer number of analytics tools that are reasonably priced, you might have chosen to spend your time on something more specialized like a recommendation system.

Re: Building a data team at a mid-stage startup

#64

Post by dijksterhuis »

Orou wrote: post #37
IMTDb wrote: post #9

What would be the name of the position/profile of someone in charge of building the data warehousing architecture/ETL pipelines? I my view, they need make sure the warehouse model is a correct representation of the business and that it can be leveraged to answer basic or not-so-basic questions using SQL. They also need to promote it's usage internally by ensuring it is accessible and easy to use and guide other team…

I would also vote for "data engineer" (it's my current job title). You very likely don't want a data scientist to be doing a data engineer's job (and they probably don't want to be doing it themselves!). While there are similarities, data engineering tends to be a lot closer to software development than data science. If you're advertising for a data scientist role, don't expect them to be happy if 80% of their job is…

Seconded.

I was a bit sad to not see any mention of a data engineer anywhere in the article.

Like, if you gave me access to all the prod tables and the warehouse I'd be having a whale of a time and (hopefully) delivering enough business value to automate some of the more regular "English to SQL" translations.

> You very likely don't want a data scientist to be doing a data engineer's job.

100%. This is one of those things that would make "disgruntled ML people" in the article want to leave.

Re: Building a data team at a mid-stage startup

#65

Post by dgb23 »

dijksterhuis wrote: post #60
GlennS wrote: post #55

I liked this article, but I have two questions: 1. Is it definitely a good idea to build a separate data team, rather than embedding people with analytics knowledge in feature teams? Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data? 2. Is A/B testing and driving your business by metrics really a good idea? My (uninformed) impression is that data-driven is respon…

> Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data? It's important to get the core centralised data infrastructure up and running (even if it's dirty af) as that helps with the bulk of the data work. The oft quoted not completely true but kinda true statistic is that 70% of data work is finding, cleaning and storing the data. Analysis and modelling is the easy b…

Interesting. I'm a bit of a hybrid, CLI/GUI user. There are things that I find easier to to in a CLI (or with text in general) and things were a GUI is more natural.

CLIs are finicky and force you to think in terms of text, whether it is appropriate or not. GUIs can be more expressive and haptic, but are typically very idiosyncratic and can get in the way of things.

The data-driven approach to UI seems a bit crazy?

If I think about the problems of any UI, I think in terms of communication, intent, learning, psychology and aesthetics. All of those things are human to human or human to computer related issues.

I think data-driven (as in statistical data derived from user behavior) approaches are or can be useful in terms of "what" to present, prioritize and so on. But much less so on "how", because I think this should be based on experiences derived from direct interaction and needs to be induced by creativity.

And I mean creativity from both sides, the implementer and the user. One thing that CLIs generally do better is to provide composable tools within a adaptive and simple system (pipes, text etc.), whereas it is hard to impossible to let GUIs talk to eachother and compose them to a user tailored whole.

I think we should empower "non-technical" users with the freedoms and sound principles we have come to enjoy ourselves, instead of letting statistical data dominate their experience.

Re: Building a data team at a mid-stage startup

#66

Post by iamacyborg »

alzaeem wrote: post #59
GlennS wrote: post #55

I liked this article, but I have two questions: 1. Is it definitely a good idea to build a separate data team, rather than embedding people with analytics knowledge in feature teams? Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data? 2. Is A/B testing and driving your business by metrics really a good idea? My (uninformed) impression is that data-driven is respon…

I share the frustration with how many A/B testing driven development processes end up. Leads to a very iterative process with lots of small changes, rather than big bets. Also, trying to get statistical significance from iterative changes when you don’t have a ton of data is problematic.

I think that’s just down to a lot of folks who think ab testing is the answer to every problem not necessarily having a background in maths or stats. I see it all the time in marketing teams where people’s are so conditioned to think of testing as the default that they don’t understand what they’re doing or why.

Re: Building a data team at a mid-stage startup

#67

Post by roenxi »

zippy5 wrote: post #51

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place. This company makes 10M and spends 3M on the team and infrastructure to make data a core competency? A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten an…

Having unique data is quite valuable. If your organisation can make decisions based on signals that other people can't detect then it can gain a decisive edge.

I do wonder at the anecdotes in this article though. In businesses that I've seen, the data team is usually the biggest impediment to a data-driven culture because they have databases full of numbers and no real grasp of how that links to the decision making process that makes the business money.

Beefing up the team doesn't help. In data, as in business more generally, the important think is not trying to guess what job your doing and spend a lot of time talking to customers about what job they need done. If the data team is where that work happens in a business then that can be helpful - but the grunt work of SQL/reporting/basic analysis is almost never where the value appears from.

Re: Building a data team at a mid-stage startup

#68

Post by tharkun__ »

PragmaticPulp wrote: post #36
czep wrote: post #32

Earlier quoted context omitted.

Thanks for the tips! One mantra I've tried when starting at a new job is "for the first 3 months say yes to everything, for the next 3 months say no to everything." The idea is you first immerse yourself in everything, to find out what works and what doesn't. Then you dedicate time to fix the broken processes so that hopefully when you hit 6 months your team is better positioned to be more efficient. Obviously you ca…

Good advice as long as you don't take it too literally. The most important thing is to work closely with your manager on expectations. If someone from another department comes to you with a proposal, an ask, or a directive, you don't want to say yes without first consulting with your manager. Depending on company politics, some managers might try to rope new employees into doing work that isn't actually part of their…

Very good advice. I've also seen this from new ICs (incidentally from one of our new data guys). I bet he said yes but he shouldn't have.

New guy, knows nothing about the company and product yet but was asked to "get KPI X by end of day". He obviously has no idea how to get this done so goes to various people and throws around the "VP XYZ wants this by end of day, help me now or else!".

Needless to say I, as politely as I could, told him to shut it, look at his data and what he could get from it and stop interrupting dev with mid day, two days after start of a sprint, requests to do his work for him (dude I don't even have access to your data storage, don't know what data you have or don't etc). And do it by end of day. Sure.

The guy is burned for me now. He will have to do a LOT of sucking up to dev now for his try at "do my job for me or else"

Re: Building a data team at a mid-stage startup

#69

Post by lifeisstillgood »

zippy5 wrote: post #63
lifeisstillgood wrote: post #54

Earlier quoted context omitted.

As someone who reaches for code if they need to blow their nose, what is a 3rd party vendor going to supply that a “English-to-SQL translators” wont do? (I have not finished the article, but the idea that devs / data scientists can be replaced by some vendors makes me wonder what I have missed) Edit: Also love the Uranium quote :-)

So my assumption is that for a given business model, like e-commerce or Saas business much of the highest value analysis is fairly standardized and can be templated. For example breaking down conversion rate by weekly cohort is something that can be pretty easily be done in google analytics. The problem with English to sql translators or most coders in general are the assumptions we make, in particular about the unde…

can you point me at some of the vendors - I am missing a chunk of knowledge i suspect.

Or is this - for exmaple - people taking google analytics and producing analysis on top of that.?

Re: Building a data team at a mid-stage startup

#70

Post by higeorge13 »

WastingMyTime89 wrote: post #61
czep wrote: post #24

This is so eerily familiar I swear I've had many of these exact conversations word for word. The only way this doesn't turn into a complete nightmare of a cluster is if the exec team "gets it". If so, you just might stand a chance at building a data team that gels with the rest of the org. But if the exec team simply hired you for window-dressing, expect to be treated like a scapegoat and a punching bag. Any mistakes…

> However, if E-team does give you the authority to call Product's bullshit, and tell Finance to stuff it, and not take direction from Eng leads, then you actually might be able to accomplish something really cool. So what's the business case for having a data team independent of product, business and engineering? Because as I see it the data team is a support function not q core part of the business. I'm sure it can…

Engineering is building some schema, creates and uses multiple data stores , message queues, etc, eventually the queries do not longer work properly as the company scales and gets more and larger customers and hundreds of other issues. Doesn’t engineering need a proper data engineering team/dba/you name it to handle those?
Post reply on HN