Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a data team at a mid-stage startup

erikbern.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: Building a data team at a mid-stage startup

#51

Post by zippy5 »

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place.

This company makes 10M and spends 3M on the team and infrastructure to make data a core competency?

A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten and setup with 3rd party software for an order of magnitude cheaper.

I can only imagine what this hypothetical startup could have learned if they spent that money actually talking to customers, and running more experiments.

I’ve heard people talk about data as the new oil but for most companies it’s a lot closer uranium. Hard to find people who can to handle / process it correctly, nontrivial security/liabilities if PII is involved, expensive to store and a generally underwhelming return on effort relative to the anticipated utility.

My take away was that startups benefit tremendously from a data advisor role to get the data competency, as well as the educational and cultural benefits, but realistically the data infrastructure and analytics at that scale should have been bought not built. Obviously there are a couple of exceptions such regulatory reasons like hippa compliance for which building in-house can be the right choice if no vendor fits your use case.

Re: Building a data team at a mid-stage startup

#52

Post by te_chris »

This is a good write-up, but for the sort of insights they’re getting they’re over staffed and overpaying. A combination of a cloud dw (big query, e.g), cloud etl (stitch, fivetran) and dbt for the T in ELT to build useful reporting tables, along with some sort of sql based BI (mode, in our case), could deliver the same insights for a fraction of the price. Throw in a sub to Heap or similar for ad-hoc product analytics as a cherry on top.

I concede, of course, that they’re rescuing a bad situation, not starting from scratch, but still.

Re: Building a data team at a mid-stage startup

#53

Post by chupchap »

zippy5 wrote: post #51

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place. This company makes 10M and spends 3M on the team and infrastructure to make data a core competency? A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten an…

> it’s a lot closer uranium

Love this analogy!

Re: Building a data team at a mid-stage startup

#54

Post by lifeisstillgood »

zippy5 wrote: post #51

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place. This company makes 10M and spends 3M on the team and infrastructure to make data a core competency? A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten an…

As someone who reaches for code if they need to blow their nose, what is a 3rd party vendor going to supply that a “English-to-SQL translators” wont do?

(I have not finished the article, but the idea that devs / data scientists can be replaced by some vendors makes me wonder what I have missed)

Edit: Also love the Uranium quote :-)

Re: Building a data team at a mid-stage startup

#55

Post by GlennS »

I liked this article, but I have two questions:

1. Is it definitely a good idea to build a separate data team, rather than embedding people with analytics knowledge in feature teams?

Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data?

2. Is A/B testing and driving your business by metrics really a good idea?

My (uninformed) impression is that data-driven is responsible for rather a lot of rot:

- Extremely irritating websites.

- Businesses ignoring important things because they can't measure them. (Financialisation, hand-in-hand with the MBA types the author decries.)

Re: Building a data team at a mid-stage startup

#56

Post by fouc »

zippy5 wrote: post #51

This was wonderfully written and if your gonna start a data team, this is how you do it. But I can see that I’m the only one who thought it was crazy to start a data team in the first place. This company makes 10M and spends 3M on the team and infrastructure to make data a core competency? A vast majority of wins discussed were lowly differentiated web / mobile / supply chain analytics which they could have gotten an…

> spends 3M on the team and infrastructure

You're making a pretty big assumption on cost of team & infrastructure there. This company could have 100+ people with that kind of revenue (I've worked at a company this size before). The data team is only about 6 people. The cost of the data team & infrastructure is likely less than $1M

Re: Building a data team at a mid-stage startup

#57

Post by roystonvassey »

This is a perfect encapsulation of my career as a data-guy square peg in a round hole, filled with jargon and misplaced understanding of data in general.

Despite all that you read and hear about data science advancing, you’ll be surprised to see how poorly leveraged, or worse, billions of dollars are sought to implement the latest tool that promises to change the world. Tech and data as we imagine it be in the FAANG kind of companies is far different than how it is in older industries. It’s not just systems that need upgrading, company cultures do and that’s never an easy or fast process. I’ve been in the data Analytics space for 16 years now and I still feel, more often than not, I’m part of the minority, working to demonstrate true data use-cases

Re: Building a data team at a mid-stage startup

#58

Post by sails »

Orou wrote: post #37
IMTDb wrote: post #9

What would be the name of the position/profile of someone in charge of building the data warehousing architecture/ETL pipelines? I my view, they need make sure the warehouse model is a correct representation of the business and that it can be leveraged to answer basic or not-so-basic questions using SQL. They also need to promote it's usage internally by ensuring it is accessible and easy to use and guide other team…

I would also vote for "data engineer" (it's my current job title). You very likely don't want a data scientist to be doing a data engineer's job (and they probably don't want to be doing it themselves!). While there are similarities, data engineering tends to be a lot closer to software development than data science. If you're advertising for a data scientist role, don't expect them to be happy if 80% of their job is…

IMO data engineer roles are further subset into:

1. kafka / streaming oriented software engineering

2. data warehouse and ETL/ELT development for analytics

Re: Building a data team at a mid-stage startup

#59

Post by alzaeem »

GlennS wrote: post #55

I liked this article, but I have two questions: 1. Is it definitely a good idea to build a separate data team, rather than embedding people with analytics knowledge in feature teams? Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data? 2. Is A/B testing and driving your business by metrics really a good idea? My (uninformed) impression is that data-driven is respon…

I share the frustration with how many A/B testing driven development processes end up. Leads to a very iterative process with lots of small changes, rather than big bets. Also, trying to get statistical significance from iterative changes when you don’t have a ton of data is problematic.

Re: Building a data team at a mid-stage startup

#60

Post by dijksterhuis »

GlennS wrote: post #55

I liked this article, but I have two questions: 1. Is it definitely a good idea to build a separate data team, rather than embedding people with analytics knowledge in feature teams? Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data? 2. Is A/B testing and driving your business by metrics really a good idea? My (uninformed) impression is that data-driven is respon…

> Is it possible to do the latter, but still have end up with a well-curated source-of-truth for your data?

It's important to get the core centralised data infrastructure up and running (even if it's dirty af) as that helps with the bulk of the data work.

The oft quoted not completely true but kinda true statistic is that 70% of data work is finding, cleaning and storing the data. Analysis and modelling is the easy bit.

You could do it the other way around. Hire some data people in each team and get them to meet up every once in a while.

But I'd wager the central data stuff that makes everyone's life easier will get pushed back behind the "urgent" team work every time.

#ConwaysLaw

Edit: it's possible to do both btw. E.g. Have a bunch of centralised data engineers that do the heavy lifting stuff. With data scientist/analysts embedded in teams doing the fine grained modelling stuff. It's not a binary choice (once things are up and running).

> My (uninformed) impression is that data-driven is responsible for rather a lot of rot.

I agree! I was talking to someone else (not a tech head) the other week and realised why they hate tech so much... User interfaces that just... Don't work.

Showed him a terminal cli and he went nuts over it.

Then again, we're two kinda weird ye olde "back in my day" kinda people... So...

Post reply on HN