Live data from Hacker News

“It's The Future”

circleci.com

351–360 of 536 posts

Re: “It's The Future”

#351

"-No, look into microservices. It’s the future. It’s how we do everything now. You take your monolithic app and you split it into like 12 services. One for each job you do. That seems excessive" A 100 times yes. We tried to split our monolithic Rails app into micro-services built in Go. 2 years and many fires later, we decided to abandon the project. It was mostly because the monitoring and alerting were now split in…

positive experience with microservices: identify discrete functions that can considered "stateless" (i.e. no side effects, deterministic output for given input) and factor those out into stand-alone microservices. a good example of this that I've used in production at my current $dayjob: dynamic PDF generation. user makes request from our website, request data is used to fill out a pdf template context which is then…

Does it connect to the database to fill in some values in the template? Does it keep connection pool of let's say 5 connection always open (as libraries like to do)? Does it have authentication? Is it public or private API? Who is managing security? Is it running behind it's own nginx or other proxy? Does it have DoS protection (PDF generation can be CPU intense)? What about the schema for request? How do you manage changes to the schema? They need to be deployed together with changes in other services, right? What about changes to database schema - you need to remember to update that service as well and redeploy it at the right time as well - just after successful db migrations - which live in another project.

All of that and much more needs to be replicated for each microservice, right?

Why not just have a module in your monolithic app that does it. The logic will still be separate. In most languages/frameworks you can spawn pdf generation task. Any changes are easier to introduce as well. There's no artificially materialised interface. Updates are naturally introduced. All auth logic is there already, you don't need to worry about deploying yet another service, same with logging etc.

Re: “It's The Future”

#352
There's a theory in economics about the optimal size of a firm. How big should a company be? The optimal size could be infinite, where all of society acts as one firm. Think Communist-style command economy. Or it could be one, where everyone acts as networks of individual contractors or single-owner businesses. Think anarcho-capitalism. But in reality it's neither extreme and falls somewhere in the middle; why?

It turns out that the optimal size depends on the balance between the overhead costs associated with allocating resources within one firm and the transaction costs associated with two firms doing business with each other. The overhead costs are higher with large firms because there's more internal resources, including people, to allocate. On the other hand, transaction costs are higher with small firms because each firm does less themselves so they need to transact more with others to accomplish their goals.

As the relative costs vary over time, the optimal size varies too, and firms in an industry will grow and shrink. If it increases, then you'll see mergers and acquisitions produce larger firms. If it decreases then you'll see firms start splitting or small startups disrupting their lumbering competition.

I suspect a similar thing happens in software, where there's an optimal service size. It could be infinite, where it makes sense to build large monoliths to reduce the cost of two systems communicating. Or it could be one, where it's optimal to break the system at as fine a granularity as possible (function level?).

The optimal size depends on the balance of costs. All else being equal, by drawing a service boundary between two bits of functionality you shrink the services on either side but you increase the number of services and add communication costs for them to exchange data and commands.

How these costs balance out depends on the technology, and there are competing forces at work. As languages, libraries and frameworks improve, we can manage larger systems at lower costs. That tends to increase the optimal service size. As platforms, protocols and infrastructure tools improve, the costs to run large numbers of services decreases. That tends to decrease the optimal service size.

The microservices movement, and to an extent the serverless movement, assume that in the medium- and long-term the technological improvements are going to tip the scales sharply in favour of small services. I agree that's likely the case. But we're not there yet, except in some specialized cases such as large distributed organizations (Conway's law). But it's going to be at least a few years before it's worthwhile to build most software systems in a microservice architecture.

Re: “It's The Future”

#353
post #89

Earlier quoted context omitted.

Why not use Python instead? Shellscript is so... chaotic.

Shell/batch scripting can often be useful in the devops world, where you have no guarantee that any additional tools (python, ruby, perl, powershell, whatever) wil be available. Shell scripts are guaranteed to be runnable on all machines. Unfortunately the shell "language" sucks, but still...

> Unfortunately the shell "language" sucks, but still...

Why do you say that? genuinely curious

Re: “It's The Future”

#354

Earlier quoted context omitted.

Binaries are one thing, but there are the other abstractions that containers bring in regard to networking and storage. You expose what are the network APIs of your apps (e.g open ports), filesystem mounts, variables (12 factors), etc. Your application becomes a block that you can assemble for a particular deployment; add some environment variables, connect a volume with a particular driver to a different storage bac…

> add some environment variables, connect a volume with a particular driver to a different storage backend, connect with an overlay to be able to talk to other containers privately across different servers or even DCs, etc. But environment variables already exists without docker. Volumes already exists, aka partitions. "Overlay network" already exists, aka unix sockets or plain TCP/UDP/etc over the loopback interface…

It's about the automation of these things.

You now have generic interfaces (Dockerfile, docker-compose, Kubernetes/Rancher templates, etc.) to define your app and how to tie it together with the infrastructure.

Having these declarative definitions make it easy to link your app with different SDN or SDS solutions.

For example, RexRay for the storage backend abstraction of your container:

http://rexray.readthedocs.io/en/stable/

You can have the same app connected to either ScaleIO in your enterprise or EBS as storage.

We are closer than ever to true hybrid cloud apps and it's now much more easier to streamline the development process from your workstation to production.

I think it's pretty exciting :)

Re: “It's The Future”

#355

The article perfectly summarizes my frustration and sentiment. These days I hear these buzzwords all time. I work as a consultant for an enterprise product and most people whom I meet they somehow catch these buzzwords and blurt it out in front of everyone during meetings and discussions to either showoff that they know technology and things that are in the market these days(also latest iphone, apple news, tesla, spa…

I have seen excel, SQL, even MS office used as a buzzword. Of course there's also agile, scrum, etc.

Bigdata and machine learning are also hot word. But they are clearly modern engineering. Consultants exist to explain the best way to achieve modern best practices to people without the appropriate background. If someone asks about "Why no Hadoopz plx?", either explain the other technology used instead (maybe spark, storm?) or explain that the scale is small enough for Access to handle. That's a consultant's job.

Re: “It's The Future”

#356

Earlier quoted context omitted.

Yup, that's essentially what looking that byte-size means. However, just because it doesn't fit in memory might not make it big data if it's just poorly engineered. But many times this happens because of wasted or bloated indexes that aren't useful. Or it happens when data types are picked incorrectly. For example, I once worked on a database where the original developer used Decimal(23, 0) as a primary key. This was…

>Since text fields in mysql are stored as separate files Bwuh? Over in MS SQL you just go for an NVARCHAR and forget about it. What is the right way to store this data (if you really do need to store the JSON rather than just serializing it again when you get it out of the DB)

varchar is different than a text field in mysql: http://dev.mysql.com/doc/refman/5.7/en/blob.html

It stores text fields as blobs.

I suppose now the right way would be the json data type. It didn't exist when I was working with these servers though (or they were on a much older version of MySql) https://dev.mysql.com/doc/refman/5.7/en/json.html

Re: “It's The Future”

#357

The article perfectly summarizes my frustration and sentiment. These days I hear these buzzwords all time. I work as a consultant for an enterprise product and most people whom I meet they somehow catch these buzzwords and blurt it out in front of everyone during meetings and discussions to either showoff that they know technology and things that are in the market these days(also latest iphone, apple news, tesla, spa…

I never thought I would appreciate Java...but the industry has really made me. Take your .war file, drop it onto JBoss. It deploys across the cluster in a zero downtime manner, isolates configuration, provides consistent log structure, cert management, deployment. You can deploy dozens of small war's to the same server and they can talk to each other. Load balance across the cluster automatically based on actual load…

Can you do similar stuff with clojure or Scala? Maybe there's a way to avoid the bad parts

Re: “It's The Future”

#358
I wonder if the original HN "Heroku is Dead..." title will cost Heroku money / market share in indirect ways.

When I see titles like that (despite the fact that it was intended as sarcasm), I think to myself, e.g., "I bet at least hundreds of people who scrolled past it thought it was sincere, and now they will have this subconscious 'Heroku is Dead... Docker...' thought at times when deploying projects. Maybe they'll even check out Docker. Maybe these hundreds of people will represent a tipping point of sorts for Heroku->Docker migrations, because one of them will write a really great blog post about it, and it will receive thousands of views..." (alternate endings of the same thought continue to be brute-forced for a few moments).

Along the same vein of thinking, back in 2008 I had this "realization" that Google could control the world by simply showing results based on headline titles (e.g., a search for "Obama" during the election could have resulted in articles / results whose titles have words/phrases whose presences are positively correlated to lower or higher stress levels, assumptions, other emotions, etc., resulting in a net positive or negative sentiment, respectively, about the subject of the search query, all while simply scanning the results to determine which one to click).

Re: “It's The Future”

#359
post #347

Earlier quoted context omitted.

So, why not ditch AWS and use GCP? Kubernetes cluster setup in one command, or use AppEngine. Maybe the problem is AWS.

> So, why not ditch AWS and use GCP? It's a Google product offering, which means it could be EOLed tomorrow. Or this afternoon. Or maybe it already has — better go check their blog.

Amazon and Google have almost the exact same language in their ToS for deprecation policies. Also, k8s is open source, so you can at least go somewhere else if you really must.

GCP isn't "Google Labs".

Re: “It's The Future”

#360

Earlier quoted context omitted.

I never thought I would appreciate Java...but the industry has really made me. Take your .war file, drop it onto JBoss. It deploys across the cluster in a zero downtime manner, isolates configuration, provides consistent log structure, cert management, deployment. You can deploy dozens of small war's to the same server and they can talk to each other. Load balance across the cluster automatically based on actual load…

Can you do similar stuff with clojure or Scala? Maybe there's a way to avoid the bad parts

These days there are workarounds to avoiding a lot of xml configuration in the java ecosystem (although you end up with a decent amount of annotations). Spring boot is a great example of this.
Post reply on HN