Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Koheesio: Nike's Python-based framework to build advanced data-pipelines

github.com

Post reply on HN
Print view View original

71–75 of 75 posts

Re: Koheesio: Nike's Python-based framework to build advanced data-pipelines

#71

Post by crowcroft »

hipadev23 wrote: post #64
crowcroft wrote: post #60

Earlier quoted context omitted.

To defend management a little bit, these massive companies have existed through many eras of technology with many different managers. They work with many external companies in many different ways. They have an exceptionally complex, but functioning tech stack, that allows all of these many dependencies to function together. Lastly, they are successful as they are! It's not usually an issue of immaturity, it's just re…

Nike's success in the marketplace has approximately zero to do with their tech stack. That's why their tech literacy is so bad. It just doesn't matter to their business.

This is the most succinct point in defence of management really. Tech stack has no impact on business results, so quite rightly they don't make it a priority to 'fix' it.

Re: Koheesio: Nike's Python-based framework to build advanced data-pipelines

#72

Post by rootusrootus »

MisterTea wrote: post #62
rootusrootus wrote: post #59

Earlier quoted context omitted.

> How many shoes did we sell in-store vs ecommerce last week? That is perhaps not a great example. My brother is a business analyst at Nike (has been for 15 years or more). I just asked him how hard it would be to answer that question and he said it would be pretty easy. Granted, this is the kind of data he works with routinely, so it may be more difficult for other teams that do not.

Is he the person running the query or is he reading a blood spattered report pulled from the dead hands of some data engineer who perished battling the system to retrieve said data?

A little of both? He's a team lead. Definitely runs queries himself, but probably spends more time interfacing with management and other teams on a day-to-day basis.

Re: Koheesio: Nike's Python-based framework to build advanced data-pipelines

#73

Post by drjasonharrison »

steveBK123 wrote: post #33

If I had to guess, a tool like this might be useful in a shop with a lot of inexperienced devs. It's a thin wrapper to make sure everyone walks the same well worn path the same way. You have 2-3 devs work on the tooling, and a much larger team doing rote ETL. I worked at a shop that did this and the trade-off is TTM, as your 2 person tools team is constantly needing to unblock ETL team with new features as they encou…

What is TTM: talk to me or trailing twelve months

Re: Koheesio: Nike's Python-based framework to build advanced data-pipelines

#74

Post by greazy »

drjasonharrison wrote: post #73
steveBK123 wrote: post #33

If I had to guess, a tool like this might be useful in a shop with a lot of inexperienced devs. It's a thin wrapper to make sure everyone walks the same well worn path the same way. You have 2-3 devs work on the tooling, and a much larger team doing rote ETL. I worked at a shop that did this and the trade-off is TTM, as your 2 person tools team is constantly needing to unblock ETL team with new features as they encou…

What is TTM: talk to me or trailing twelve months

I think Time to Market.

Re: Koheesio: Nike's Python-based framework to build advanced data-pipelines

#75

Post by rexpop »

crowcroft wrote: post #70
rexpop wrote: post #63

Earlier quoted context omitted.

> Simplifying the tech stack would probably require simplifying their business operations, which probably means less revenue. I don't understand this perspective. Simplifying the tech stack might mean taking multiple services in multiple languages, and deprecating some in favor of migrating that functionality to the most maintainable codebase. This shouldn't mean "simplifying their business operations", or affecting…

I agree there probably are areas were they can simplify with no impact to operations. But I would imagine there are a lot of pieces of apparent cruft hanging around that is actually there because if you remove it things break. Maybe a large retailer that you rely on requires an integration with an old version of SAP, and then a logistics partner only provides files over FTP, and you need to use OCR to retrieve any da…

"Look, my setup works for me. Just add an option to reenable spacebar heating."[0]

> Management can't just mandate that you will 'simplify the tech stack'.

In my experience, it's the devs who want to do reactors for love of the craft and satisfaction of a job "well" done, while management seeks to sweep as much technical "debt" as possible under the rug, and they'll burn through as many Romanians' nights' sleeps as it takes to reach their numbers.

0. https://m.xkcd.com/1172/

Post reply on HN