Introducing Google Cloud Container Builder • Hacker News Board

Re: Introducing Google Cloud Container Builder

#31

Post by kellyjandrews » Mon, Mar 06, 2017, 10:26 PM UTC

fishnchips wrote: post #16

skj wrote: post #12

Earlier quoted context omitted.

We agree! We maintain the official plugins (which we call "build steps" or "builders") here: https://github.com/googlecloudplatform/cloud-builders Support is limited, more or less, to what I am really familiar with. If you have other use cases that we could support with other builders, I'd love to see a FR on that GitHub repo. Also, you can use literally any container image as a build step. Details about how to make…

I like how Codeship approaches the problem - see here for inspiration https://documentation.codeship.com/pro/getting-started/steps... Still, they're super expensive and not very performant, so I welcome competition. What I'm missing here is an easy way to link a container being tested with external containers (eg DB).

Disclaimer: Codeship employee. Glad you like our approach. We are constantly working on performance, for sure. Since we offer dedicated build machines, there are always options for better performance.

Appreciate the feedback, and feel free to reach out to me directly anytime if you have other questions or thoughts on how we can get better.

Re: Introducing Google Cloud Container Builder

#32

Post by kellyjandrews » Mon, Mar 06, 2017, 10:30 PM UTC

fishnchips wrote: post #6

theCricketer wrote: post #4

Could someone explain specifically what problem Container Builder solves?

I'd imagine it's the same problem companies like Circle CI or Codeship solve with their Docker-focused products - testing, building and deploying containers.

Codeship here again ;).

We love when Google releases new products - it tends to improve the entire space in general.

Our take is that Google Cloud Container Builder will fit in nicely with the Google ecosystem, but Codeship is trying to solve a different problem by being platform agnostic, and flexible enough to solve the holistic CI/CD workflow.

Re: Introducing Google Cloud Container Builder

#33

Post by poofyleek » Mon, Mar 06, 2017, 10:32 PM UTC

tav wrote: post #9

bsaul wrote: post #2

With appengine exposing instances and cloud container facilitating the deployment of software packages, i wonder how long it'll take until the two merge.

That's basically what App Engine Flexible [0] is. It's yet to reach General Availability, but all you have to do is define a bunch of containers and a dispatch.yaml file [1], and App Engine then takes care of deploying/routing/scheduling it for you. [0] https://cloud.google.com/appengine/docs/flexible/ [1] https://cloud.google.com/appengine/docs/flexible/python/how-...

This sounds useful and revived my interest in GAE. The Go example has health check code. It makes sense. Are there equivalent offerings from AWS and Azure? This looks likes a unique GAE feature.

Re: Introducing Google Cloud Container Builder

#34

Post by nchuhoai » Mon, Mar 06, 2017, 10:36 PM UTC

I'm so surprised that both Docker's as well as Google's solution to this problem involves 0 caching.

Much of our container building includes fetching dependencies from various package managers which rarely changes across builds, yet takes up the majority of the build time. Most of the time thats good enough, but for cases where we want to immediately deploy a fix, it can be quite frustrating to having to spend all this time seemingly unncessarily.

Edit: Docker 1.13 has the new --cache-from option so it seems relatively straight-forward to do I guess?

Re: Introducing Google Cloud Container Builder

#35

Post by tjenkinsqs » Mon, Mar 06, 2017, 10:37 PM UTC

skj wrote: post #11

Hi all, I'm on the eng team for this service. Happy to answer any questions, and I'll do my best to monitor the threads.

Very glad that you are offering this service. A key feature needed is speed, i.e. minimal time from check-in through build/unit & integration tests/image/deploy. Questions: - What delay can we expect from check-in to start of build (seconds, minutes?) - What kind of compute power/memory is available for the build? if I run on fast CPU vs slow CPU that makes obviously a difference. What does this service run on? - Is there a way to allocate more vertical resources (more CPU/memory)? - Can parts of the pipeline be parallelized, e.g. run certain tests in parallel (using more than one core/machine)?

Re: Introducing Google Cloud Container Builder

#36

Post by skj » Mon, Mar 06, 2017, 10:40 PM UTC

nchuhoai wrote: post #34

I'm so surprised that both Docker's as well as Google's solution to this problem involves 0 caching. Much of our container building includes fetching dependencies from various package managers which rarely changes across builds, yet takes up the majority of the build time. Most of the time thats good enough, but for cases where we want to immediately deploy a fix, it can be quite frustrating to having to spend all th…

One approach to this problem that I like is storing these dependencies in an intermediate image. Then, you can do an incremental build by referencing (rather than rebuilding) this intermediate image, and pay only the cost of downloading the raw data which should be cheap compared to the normal install process.

Re: Introducing Google Cloud Container Builder

#37

Post by vgt » Mon, Mar 06, 2017, 10:43 PM UTC

thejosh wrote: post #29

Does anybody know when they are launching in Australia? They have listed 2017 as the launch year, and I'm excited for launch :).

We happen to have our major annual conference, Google Next, this week.

Tune in at [0] Wednesday-Friday :)

(Work at Google Cloud)

[0] https://cloudnext.withgoogle.com/

Re: Introducing Google Cloud Container Builder

#38

Post by gkop » Mon, Mar 06, 2017, 10:43 PM UTC

skj wrote: post #36

nchuhoai wrote: post #34

I'm so surprised that both Docker's as well as Google's solution to this problem involves 0 caching. Much of our container building includes fetching dependencies from various package managers which rarely changes across builds, yet takes up the majority of the build time. Most of the time thats good enough, but for cases where we want to immediately deploy a fix, it can be quite frustrating to having to spend all th…

One approach to this problem that I like is storing these dependencies in an intermediate image. Then, you can do an incremental build by referencing (rather than rebuilding) this intermediate image, and pay only the cost of downloading the raw data which should be cheap compared to the normal install process.

Why not leverage Docker's built-in image layer caching?

Re: Introducing Google Cloud Container Builder

#39

Post by skj » Mon, Mar 06, 2017, 10:43 PM UTC

tjenkinsqs wrote: post #35

skj wrote: post #11

Hi all, I'm on the eng team for this service. Happy to answer any questions, and I'll do my best to monitor the threads.

Very glad that you are offering this service. A key feature needed is speed, i.e. minimal time from check-in through build/unit & integration tests/image/deploy. Questions: - What delay can we expect from check-in to start of build (seconds, minutes?) - What kind of compute power/memory is available for the build? if I run on fast CPU vs slow CPU that makes obviously a difference. What does this service run on? - Is…

"speed": We start most builds in under a second, but we don't currently offer an SLA.

"parallelism": You can indicate any kind of concurrency topology you like with the "wait_for" field on a step: see https://cloud.google.com/container-builder/docs/api/build-st...

"different machines / resources": Currently it's n1-standard-1 GCE VMs. It's possible that in the future we may become more flexible.

Re: Introducing Google Cloud Container Builder

#40

Post by tjenkinsqs » Mon, Mar 06, 2017, 10:45 PM UTC

skj wrote: post #39

tjenkinsqs wrote: post #35

Earlier quoted context omitted.

Very glad that you are offering this service. A key feature needed is speed, i.e. minimal time from check-in through build/unit & integration tests/image/deploy. Questions: - What delay can we expect from check-in to start of build (seconds, minutes?) - What kind of compute power/memory is available for the build? if I run on fast CPU vs slow CPU that makes obviously a difference. What does this service run on? - Is…

"speed": We start most builds in under a second, but we don't currently offer an SLA. "parallelism": You can indicate any kind of concurrency topology you like with the "wait_for" field on a step: see https://cloud.google.com/container-builder/docs/api/build-st... "different machines / resources": Currently it's n1-standard-1 GCE VMs. It's possible that in the future we may become more flexible.

Thanks!