Live data from Hacker News

“It's The Future”

circleci.com

401–410 of 536 posts

Re: “It's The Future”

#401

The article perfectly summarizes my frustration and sentiment. These days I hear these buzzwords all time. I work as a consultant for an enterprise product and most people whom I meet they somehow catch these buzzwords and blurt it out in front of everyone during meetings and discussions to either showoff that they know technology and things that are in the market these days(also latest iphone, apple news, tesla, spa…

Our industry is a pop culture. A fad. Computer science is not a real field.

Computer science is a real field. It's just not "developing site/app for BigCo". But - then again - it never has been. :-)

Re: “It's The Future”

#402

Earlier quoted context omitted.

Doesn't statically compiling programs solve the deployment issue better? I mean, as far as I can tell Docker only exists because it's impossible to link to glibc statically, so it's virtually impossible to make Linux binaries that are even vaguely portable. Except now Go and Rust make it very easy to compile static Linux binaries that don't depend on glibc, and even cross-compile them easily. Hell I think it's actual…

Have to admit, as a fellow Go dev, with single binary static compiles, I don't really GET why I need docker... all it seems to offer is an increased workload and complicated build proc

In my humble case, Docker solves the problems I have to manage the systems on which my application runs (and that's mainly it). A single dockerfile of 20-30 lines describes a whole system (operating system, versions, packages, libraries, etc), and cherry on the cake, I can version it in my git repository.

This is not revolutionary in itself, but having the creation and deployment of a server being 100% replicable (+ fast and easy!) on dev, preproduction, and production environments, plus it's managed with my usual versionning tool, that is something I appreciate very much.

Sure, there are other tools to do the same, but docker does the job just fine.

Re: “It's The Future”

#403

Earlier quoted context omitted.

I do have a positive micro-service experience, and although we are still in that process of breaking down our monolith SOA based app, we have seen the benefits already. The more dramatic effect was on a particular set of endpoints that have a relative high traffic (it peaks at 1000 req/s) that was killing the app, making upset our relational database (with frequent deadlocks) and driving our Elasticsearch cluster cra…

Do you think the result could also be a dramatic improvement if you kept old system and do those other things except splitting into microservices? I can't get my head around how people introduce changes to their system if they have to update 12 different microservices at once? It must be horrible. Often you hear stories how people are converting monolithic app to microservices - but this is easy. Rewriting code is ea…

>Do you think the result could also be a dramatic improvement if you kept old system and do those other things except splitting into microservices?

As I said in another thread, the separation in different components was key for resiliency. That allowed independence between the higher volume update and the business critical user facing component.

>I can't get my head around how people introduce changes to their system if they have to update 12 different microservices at once? It must be horrible.

The thing is, if you design the microservices properly it is very rare to introduce a change in so many deployments at once. Most of the time is just 1 or 2 services at a time.

>What I'd like to hear is something about companies doing active development in microservice world. How do they handle things like schema changes in postgres where 7 microservices are backed by the same db? What are the benefits compared to monolithic app in those cases?

We don't introduce new features in our monolith service anymore. So, from that perspective we do all active development in microservices.

>"How do they handle things like schema changes in postgres where 7 microservices are backed by the same db?

The trick is, you want to avoid sharing relational data between microservices. I don't know if it is just us, but we have been able to split our data model so far and in most cases we don't even need a relational database anymore, so having a schemaless key/value store makes seems easy too.

> What are the benefits compared to monolithic app in those cases?"

There are several advantages, but the critical one for me is being able to have a resilient platform that can still operates even if a subsystem is down. With our monolithic app is an all or nothing thing. Another advantage is splitting the risk of new releases.

>It seems to me that microservices can easily violate DRY because they "materialise" communication interfaces and changes need to be propagated at every api "barrier", no?

Not necessarily. YMMV but you can have separation of concerns and avoid sharing data models. When you do have shared dependencies (like logging strategy or data connections) you can always have modules/libraries.

Re: “It's The Future”

#404
post #288

Earlier quoted context omitted.

> No organisation should have more than one regular-use repo Flat out wrong for any organization with multiple products. Which, let's be honest, is most of them.

I guess Facebook, Twitter, and Google are doing things "flat out wrong", then. Yes, that's a weak argument (argument from authority) but it is true that monolithic repositories have major advantages even for organizations with multiple products. Common libraries and infrastructure are much easier to work with in monolithic repositories. My personal take on it, at this point, is that much of our knowledge of how to ma…

Those are huge organizations with commensurately large developer resources, and they simply work at a different scale than most people on HN. "It works for Google" is not an argument for anything.

Monorepos come with their own challenges. For example, if any of your code is open source (which means it must be hosted separately, e.g. on Github), you have to sync the open-source version with your private monorepo version.

Monorepo are large. Having to pull and rebase against unrelated changes on every sync puts an onerous burden on devs. When you're remote and on the road, bandwidth can block your ability to even pull.

And if you're going to do it like Google, you'll vendor everything -- absolutely everything (Go packages, Java libraries, NPM modules, C++ libraries) -- which requires a whole tool chain to be built to handle syncing with upstream, as well as a rigid workflow to prevent your private, vendored fork from drifting away from upstream.

There are benefits to both approaches. There is no "one right way".

Re: “It's The Future”

#405
post #249
post #154

Earlier quoted context omitted.

In practice micro services mean that you turn a function or method call into a network request. This doesn't really limit communication bottlenecks. It is often more difficult to argee on a network interface than on a simple function or object interface. It's also more difficult to change. You introduce a whole new set of failure modes due to going over the network. Debugging is more difficult since you now can no lo…

> You introduce a whole new set of failure modes due to going over the network. A thousand times yes. Distributed systems are hard . > Debugging is more difficult since you now can no longer step through your program in a debugger but rather have an opaque network request that you can't step into. Yes . Folks underestimate how difficult this can be. In theory it should be possible to have tooling to fix this, but I'v…

> No organisation should have more than one regular-use repo (special-use repos, of course, are special). Multiple repos are a smell.

Totally agree with everything else, but gotta completely disagree on this last point. Monorepos are a huge smell. If there's multiple parts of a repo that are deployed independently, they should be isolated from each other.

Why? Because you're fighting human nature, otherwise. It's totally reasonable to think that once you excise some code from a repo that it's no longer there, but when you have multiple projects all in one repo, different services will be on different versions of that repo, and your change may have changed semantics enough that interaction bugs across systems may occur.

You may think that you caught all of the services using the code you refactored in that shared library, but perhaps an intermediate dependency switched from using that shared library to not using it, and the service using that intermediate library hasn't been upgraded, yet?

When separately-deployable components are in separate repositories, and libraries are actual versioned libraries in separate repositories these relationships are explicit instead of implicit. Explicit can be `grep`ed, implicit cannot, so with the multi-repo approach you can write tools to verify that all services currently in production are no longer using an older, insecure shared library, or find out exactly which services are talking to which services by the IDLs they list as dependencies.

While with the monorepo approach you can get "fun" things like service A inspecting the source code of service B to determine if cache should be rebuilt (because who would forget to deploy service A and service B at the same time, anyways...), as an example I have personally experienced.

My personal belief is that the monorepo approach was a solution back when DVCSs were all terrible and most people were still on centralized VCSs like Subversion that couldn't deal with branches and cross-repo dependencies well, and that's just what you had to do, while Git and Mercurial, along with the nice language-level package managers, make this a non-issue.

Finally, there's an institutional bias to not rock the boat (which I totally agree with) and change things that are already working fine, along with a "nobody got fired buying IBM" kind of thing with Google and Facebook being two prominent companies using monorepos (which they can get away with by having over a thousand engineers each to manage the infrastructure and build/rebuild their own VCSs to deal with the problems inherent to monorepos that most companies don't have the resources and/or skills to replicate).

EDIT: Oh, I forgot, I'm not advocating a service-oriented architecture as the only way to do things, I'm just advocating that whatever your architecture, you should isolate the deployables from each other and make all dependencies between them explicit, so you can more easily write tooling to automatically catch bad deploy states, and more easily train new hires on what talks to/uses what, since it's explicitly (and required to be) documented.

If that still means a monorepo for your company's single service and a couple of tiny repos for small libraries you open source, that's fine. If it means 1000 repos for each microservice you deploy multiple times a day, that's also fine (good luck!).

Most likely it means something like 3-10 repos for most companies, which seems like the right range for Miller's Law) ( https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus... ) and therefore good for organizing code for human consumption.

Re: “It's The Future”

#406
post #321

Earlier quoted context omitted.

This is how we manage this problem at the times when Visual Basic was the king and we use instead Visual FoxPro. People want theirs apps to be made with Visual Studio (BTW, FoxPro was part of the package). So they ask: "In what is the app made"? "In Visual, Sir." Done. End of story (like most of the time, obviously some times people are more dangerous and press it ;) ). ---- The point is not focus in the exact word b…

You're exactly right. In the end, the customer is worried about solving their problem and they're asking if you're aware of . It's like when you go to the doctor and say "I've read of an experimental new treatment for X, can't we do that?". The doctor has probably already heard about it.

Yup. An experimental treatment is probably not even available to be prescribed, besides it's ethically questionable to use a treatment when the risks (or benefits) haven't been clearly delineated. Even if it could be used, a responsible prescriber would want to try all "standard" remedies before attempting an experimental method.

That's called practicing conservatively, minimizing chances of bad outcomes. It's a matter of astute clinical judgement to glean optimum risk/benefit ratio in a particular case. Since no two cases are ever exactly the same, good judgement is a constant necessity.

I see that the process of developing software has many parallels and not surprising that everyone experiences so much brokenness. When people complain to me about some mysterious program misbehavior (stuff I had nothing to do with) I empathize with them, and try to help them think logically about the problem they're having.

Only rarely can I offer any real insight, but given the insane proliferation of the alphabet soup of identifiers attached to all the "new things" out there, no one I know in the industry feels they have a handle on what's happening.

Seems like the pace of "innovations" will lead to even greater levels of incomplete and dysfunctional systems and can only lead, sooner or later, to truly catastrophic failures.

Re: “It's The Future”

#407
post #111

Man, I dont think this is the future at all. OK, Docker is good and has its propose, and is very good on what its do: "Run only one process in one brand new kernel", but beyond than that, its just a daemon that uses and abuses of linux containers, you can easily scale, but is a pain in the ass to upgrade apps, also you need to run only one process on that. Does not looks like the future for me to have 30 different li…

Docker containers don't contain a kernel. A container isn't anything special -- it's "just" a namespaced set of processes that are isolated from the host system. If you run "ps" on the host, you will see all the containers' processes.

One process per container is perfectly fine. In fact, that's the common use case. There is absolutely nothing wrong with it, and there is practically zero overhead in doing it.

What you gain is isolation. I can bring up a container and know that when it dies, it leaves no cruft behind. I can start a temporary Ubuntu container, install stuff in it, compile code in it, export the compilation outputs, terminate the container and know that everything is gone. We do this with Drone, a CI/build system that launches temporary containers to build code. This way, we avoid putting compilers in the final container images; only the compiled program ends up there.

Similarly, Drone allows us to start temporary "sidecar" containers while running tests. For example, if the app's test suite needs PostgreSQL and Memcached and Elasticsearch, our Drone config starts those three for the duration of the test run. When the test completes, they're gone.

This encapsulation concept changes how you think about deployment and about hardware. Apps become redundant, expendable, ephemeral things. Hardware, now, is just a substrate that an app lives on, temporarily. We shuffle things around, and apps are scheduled on the hardware that has enough space. No need to name your boxes (they're all interchangeable and differ only in specs and location), and there's no longer any fixed relationship between app and machine, or even between app and routing. For example, I can start another copy of my app from an experimental branch, that runs concurrently with the current version. All the visitors are routed to the current version, and I can privately test my experimental version without impacting the production setup. I can even route some of the public traffic to the new version, to see that it holds up. When I am ready to put my new version into production, I deploy it properly, and the system will start routing traffic to it.

Yes, it very much is the future.

Re: “It's The Future”

#408

Earlier quoted context omitted.

Cloud Foundry is probably what you want. To the point that the Ruby buildpack code is a soft fork of Heroku's (source: I work on the Cloud Foundry buildpacks team). It'll run on AWS, vSphere, OpenStack, Azure, GCP is coming and others to follow. There is, however, still a hump to get over in installation -- you need to learn what BOSH is, install BOSH, then install Cloud Foundry with BOSH. In the long run, for a prod…

thanks for this ! how do you look at deis.io vs Cloud Foundry. Do you see yourself subscribing to one of the popular camps out there... or will you stick to Cloud Foundry in the long term ?

I'm obviously biased.

Most of Cloud Foundry is built the way I like software to be built. Pair programming, TDD, small balanced teams, prioritising for user valu.

That style of development is actually baked into the Cloud Foundry Foundation rules. Companies who join the Foundation are expected to send engineers to ramp up on developing in this style. And voting rights are based on the number of full-time engineers you have assigned to the effort.

The reason I mention all this is that I trust the way we build Cloud Foundry. We still get production bugs and oversights and mistakes. It's around 4 million lines of code that turns into a distributed system of ~50 different interacting processes. We built a fully-featured, robust PaaS, using containers, in about 3 years, starting from scratch.

Nobody outside Google had built a container platform of this level before. Nobody but Heroku had built a fully-featured PaaS of this level before. We are, to my knowledge, the first system to do both of these things. Certainly the first opensource one.

The reason you never hear about Cloud Foundry is because we've already built all the components other folks are trying to roll up into full PaaSes. "It just works, already" is a boring story.

But again, quite seriously: I am obviously very biased.

Re: “It's The Future”

#409

Earlier quoted context omitted.

> add some environment variables, connect a volume with a particular driver to a different storage backend, connect with an overlay to be able to talk to other containers privately across different servers or even DCs, etc. But environment variables already exists without docker. Volumes already exists, aka partitions. "Overlay network" already exists, aka unix sockets or plain TCP/UDP/etc over the loopback interface…

It's about the automation of these things. You now have generic interfaces (Dockerfile, docker-compose, Kubernetes/Rancher templates, etc.) to define your app and how to tie it together with the infrastructure. Having these declarative definitions make it easy to link your app with different SDN or SDS solutions. For example, RexRay for the storage backend abstraction of your container: http://rexray.readthedocs.io/e…

We are closer than ever to true hybrid cloud apps and it's now much more easier to streamline the development process from your workstation to production.

This sounds exactly like the "It's the future!" guy in the original post...

Re: “It's The Future”

#410
post #321

Earlier quoted context omitted.

This is how we manage this problem at the times when Visual Basic was the king and we use instead Visual FoxPro. People want theirs apps to be made with Visual Studio (BTW, FoxPro was part of the package). So they ask: "In what is the app made"? "In Visual, Sir." Done. End of story (like most of the time, obviously some times people are more dangerous and press it ;) ). ---- The point is not focus in the exact word b…

I think the buzzword abuse exists because of people who don't want to take the time to learn to real skill, and just want shortcuts to sound smart and relevant. I am very skeptical of people who are "BizDev" or "Project Managers" or "Managers" or "Scrum Master" they generally don't know what they're talking about and rely on buzzwords.

Not necessarily. I suspect that people that have overlap in hist job positions but still need to use a "shared vocabulary" will diverge in their understanding of each word, not by laziness but because simply is another job.

For example, if a DBA and a JS Developer say "We need to use a scalable database", they probably don't have the same thing in mind about what "scalable" or "database" exactly is, however, both are concerned about provide data at performance.

So, if a naive web developer wanna "a scalable document stored!" you can just give to it postgres and presto! ::troll:: ;)

Post reply on HN