Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Build Code at Netflix

techblog.netflix.com

Post reply on HN
Print view View original

61–70 of 140 posts

Re: How We Build Code at Netflix

#61

Post by mkobit »

I'm interested in knowing more about the "25 Jenkins masters" that they have, and how much they have modified/built for Jenkins to make it work for them.

We are currently in a state of "big ball of plugins and configuration". A bunch of plugins have been installed, and lots of manual configuration has been put into jobs so that everybody has what they need to build their software. It has led to Jenkins being a "do everything" workflow system. The easy path that Jenkins provides, to me, seems like the wrong one - it makes it easy to just stuff everything in there because it "can" do it. This seems to leads to tons of copy/paste, drift, all types of different work being represented, and it is starting to become unmanageable.

Have others seen this happen when using Jenkins? How have you dealt with it?

Re: How We Build Code at Netflix

#63

Post by ones_and_zeros »

pitt1980 wrote: post #58
dang wrote: post #52

Earlier quoted context omitted.

Please stop posting these.

why? I'm trying to keep track of things that catch my eye

if you upvote something it'll show up in the list at https://news.ycombinator.com/saved?id=pitt1980. So just upvote things that catch your eye.

Re: How We Build Code at Netflix

#64

Post by TallGuyShort »

pitt1980 wrote: post #58
dang wrote: post #52

Earlier quoted context omitted.

Please stop posting these.

why? I'm trying to keep track of things that catch my eye

You can upvote a story an it will appear under "saved stories" when you click on your profile. You commenting to save stories for you is spamming everyone else. So yes, please stop.

Re: How We Build Code at Netflix

#66

Post by mkobit »

markbnj wrote: post #57

I'm a Netflix fan, as a consumer and an engineer, and this blog post just reinforces my fanboi status. Amidst the descriptions of deployment tools and pipelines one thing stood out for me: the fact that AMI bake times are now a large factor, and that "installing packages" and the "snapshotting process" were a big piece of this. Containers are definitely the answer to this problem. You can deploy base images with the…

I am fairly certain it is on the Spinnaker roadmap. There was a recent talk by Sangeeta Narayanan on "Containers so Far" [1]. At 37:10~ you can see how they will essentially be allowing for containers to replace AMIs in the "bake" process. I'm looking forward to seeing how Netflix start to roll out more container support, and seeing how Spinnaker progresses.

[1] http://www.infoq.com/presentations/netflix-containers

Re: How We Build Code at Netflix

#68

Post by richardwhiuk »

mkobit wrote: post #61

I'm interested in knowing more about the "25 Jenkins masters" that they have, and how much they have modified/built for Jenkins to make it work for them. We are currently in a state of "big ball of plugins and configuration". A bunch of plugins have been installed, and lots of manual configuration has been put into jobs so that everybody has what they need to build their software. It has led to Jenkins being a "do ev…

Yes - we've definitely seen that. The most common thing we've see if that you start with a jenkins job that just runs a build process, and then you end up with a jenkins job that has a huge shell script which calls the build process somewhere in the middle, that's unversioned.

To resolve that we've tried to push the shell script into a file in the repository, which Jenkins then checks out and runs, which makes it easier to maintain and faster to set up new build machines.

Re: How We Build Code at Netflix

#69

Post by christop »

mkobit wrote: post #61

I'm interested in knowing more about the "25 Jenkins masters" that they have, and how much they have modified/built for Jenkins to make it work for them. We are currently in a state of "big ball of plugins and configuration". A bunch of plugins have been installed, and lots of manual configuration has been put into jobs so that everybody has what they need to build their software. It has led to Jenkins being a "do ev…

Netflix have been quite involved in the Jenkins project, including the Job DSL Plugin, which enables the automated creation of new Jenkins jobs, e.g. when a new Git branch is created, by defining the job structure with a simple Groovy-based DSL.

Taking this further, the upcoming release of Jenkins 2.0 is going to put a lot more emphasis on pipelines-as-code, where entire workflows can be defined in code, and version-controlled, as opposed to clicking everything together via the web UI.

See https://jenkins-ci.org/2.0/

Re: How We Build Code at Netflix

#70

Post by lintoma »

mkobit wrote: post #61

I'm interested in knowing more about the "25 Jenkins masters" that they have, and how much they have modified/built for Jenkins to make it work for them. We are currently in a state of "big ball of plugins and configuration". A bunch of plugins have been installed, and lots of manual configuration has been put into jobs so that everybody has what they need to build their software. It has led to Jenkins being a "do ev…

It's really the same Jenkins Ami but sharded for different teams.

One of the things Spinnaker does is plugin Jenkins jobs as a reusable stage that is parametrized and scoped to an application deploy. This has allowed one team to move from 40 Jenkins jobs created via job dsl plugin to just 6

Post reply on HN