Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Infrastructure 2.0: A Wishlist

erikbern.com

Post reply on HN
Print view View original

81–90 of 202 posts

Re: Software Infrastructure 2.0: A Wishlist

#81

Post by pm90 »

nonameiguess wrote: post #20

This is a lot harder than the author wants to realize. Elastic infrastructure deployment in sub second latency is not likely to ever happen. The problem is simply a lot harder than serving http requests. It's not receive some bytes of text, send back some bytes of text. VMs are just files and can be resized, but optimal resource allocation on multitenant systems is effectively solving the knapsack problem. It's NP ha…

It’s true that we’ve made a ton of progress. But I LOVE it when developers demand more and I will bet good money that it’s only a matter of time when what the author describes will actually happen.

We’ve been making tremendous strides in making production software systems easy and accessible for developers (and not sysadmins) to understand and operate. We’re currently in the stage where the devops principle has been somewhat realized; devs love the ability to deploy when they want but absolutely hate that infrastructure provisioning takes so long compared to the tools they use for software development. This is a good thing! I want more devs to demand this; someone may actually solve it.

Re: Software Infrastructure 2.0: A Wishlist

#82

Post by qaq »

erikerikson wrote: post #60
qaq wrote: post #55

Given how much capitalization AWS and Azure account for it's surprising how little progress has being made in many areas. A true serverless RDBMS database priced sanely that basically charged for qry execution time and storage without need to provision some "black box" capacity units and would be suitable for a large scale app. Most of all workflows that require contacting support and arbitrary account limits that fo…

Both Dynamo DB and Cosmos DB offer effectively what you're asking for. Limits are a risk that must be mitigated but they aren't arbitrary, they exist to avoid interference of one customer on to others. They protect the service which, to providers and reasonably so, is more important than any one customer.

DynamoDB is really crappy key value store with poor feature set and really bad limits.I am talking about real SQL RDBMS like Aurora Serverless but not built on top of legacy codebase and shoehorned into the use case. As far as account limits if I consume X+X things across 2 accounts vs 1 account what did it protect exactly? An org can have any number of accounts and add them at any time.

Re: Software Infrastructure 2.0: A Wishlist

#83

Post by mikewarot »

philipswood wrote: post #49
philipswood wrote: post #48

Earlier quoted context omitted.

Agreed. Operating systems abstract resources, the fact that we needed containers and VMs point to the fact that the first set of implementations weren't optimal and needs to be refactored. For VMs, security is the one concern, the others would be more direct access to lower levels and greater access to the hardware and internals than just a driver model. For containers I'd say that the abstraction/interface presented…

(I'm not holding my breath for the refactor though... We're stuck at a local maximum and we're better at adding stuff,rather than removing/refactoring them. Also- just about EVERYTHING is built on top of these current layers)

Genode is a project in Germany I've been watching for a while... right now I think they are the best chance of getting there. They've been making it closer and closer to the point where I can latch onto it and start using it.

There has to be a microkernel at the bottom of everything, which rules out anything Linux based. Genode does use Linux drivers though, in a contained way.

Re: Software Infrastructure 2.0: A Wishlist

#84

Post by mikewarot »

w7 wrote: post #51
mikewarot wrote: post #27

Virtualization eventually will be seen as the unnecessary layer added to make up for operating systems that lack capability based security. It's going to take a decade to refactor things to remove that layer. Once done, you'll be able to safely run a process against a list of resources.

We've had capability based security frameworks aka MAC (ex: AppArmor) in Linux since 1999 or earlier. Containers (which also existed long before docker) have been popularized for convenience, and virtualization would still be useful for running required systems that are not similar to the host. If anything it looks like we're going towards a convergence with "microvms".

Capabilities are like taking a $5 bill out of your wallet, handing it to your child, and letting them buy ice cream with it.

You delegate $5, nothing more than $5 can possibly leave your wallet as a result.

AppArmor is like putting a vault around the ice cream truck and giving a strict list of who is allowed to buy what ice cream. Hardly the same thing.

Re: Software Infrastructure 2.0: A Wishlist

#85

Post by marcus_holmes »

why does this rant start by talking about optimising things for the user, and then end up talking about optimising things for the developer?

I get that they're talking about productivity tools for developers. But missing the point that optimising these tools for developers will mean they're not optimised for the actual user. We're not the customer. The stuff we write should be fast when run in production. Having it be nice to develop on is a bonus.

Having generated configuration that the developer doesn't have to think about means that the system can't be tweaked for the particular use case to make it faster in production. Yes, it's easier. No, that's not better.

Our job is to handle the complexity. To take "simple" business requirements and deal with all the complexity that is needed to make those work. That's the gig.

Re: Software Infrastructure 2.0: A Wishlist

#87

Post by zby »

I have been observing it at my work - the production systems were getting more and more complicated and it was harder and harder to set up dev and test environments. It also started to break the underlying system when you had to install old libs for the project - and then maybe you needed two versions of the libs for different branches of the project. So I started to develop in virtual machines. Then people started using containers for that. That felt like a step back, because it was a different abstraction - and you had to do things differently on the dev machine that on production. Then I learned that people use terraform or ansible to setup the production and I hoped that I could run the same scripts on my dev machine in a vm and have environment just like production. But no - when I encountered a project with terraform and ansible - the scripts contain lots of AWS related stuff, it is impossible to untangle it and run on dev machines. I have never learned kubernets, I don't do much programming any more, with containers both in production and dev that would make sense - but I still don't see scripts that would just spin off a dev environment. Maybe dev environments on a dev machine is really a dead end now - and what we need is a way to spin of dev envs in the cloud.

Re: Software Infrastructure 2.0: A Wishlist

#88

Post by otabdeveloper4 »

jiggawatts wrote: post #47

Years and years ago I saw an advertisement by a SAN array vendor where their gimmick was that they supported very cheap read/write snapshots of very large datasets, in combination with a set of transforms such as data masking or anonymisation. Their target market was same as the OP: developers that need rapid like-for-like clones of production. The SAN could create a full VM server pool with a logical clone of hundre…

> Azure's network team decided to unnecessarily use NAT for IPv6, making it exactly (100%!) as complex as IPv4...

People like the NAT. It's a feature, not a bug. If your selling point for IPv6 is "no more NAT" then no wonder it never went anywhere!

P.S. No, "you're doing it wrong" and "you're not allowed to like the NAT" are not valid responses to user needs.

Re: Software Infrastructure 2.0: A Wishlist

#89

Post by ironicsonic »

js8 wrote: post #75
mikewarot wrote: post #27

Virtualization eventually will be seen as the unnecessary layer added to make up for operating systems that lack capability based security. It's going to take a decade to refactor things to remove that layer. Once done, you'll be able to safely run a process against a list of resources.

It already exists in the IBM mainframe. But nobody wants to write apps for it..

Who would want to use those weird, outdated languages and technologies, though?

It's better to use a modern DevOps stack involving Kubernetes, Ansible, YAML, Jinja2, Python, cron jobs, regexes, shell scripts, JSON, Jenkinsfiles, Dockerfiles, etc.

Re: Software Infrastructure 2.0: A Wishlist

#90

Post by otabdeveloper4 »

philipswood wrote: post #48
mikewarot wrote: post #27

Virtualization eventually will be seen as the unnecessary layer added to make up for operating systems that lack capability based security. It's going to take a decade to refactor things to remove that layer. Once done, you'll be able to safely run a process against a list of resources.

Agreed. Operating systems abstract resources, the fact that we needed containers and VMs point to the fact that the first set of implementations weren't optimal and needs to be refactored. For VMs, security is the one concern, the others would be more direct access to lower levels and greater access to the hardware and internals than just a driver model. For containers I'd say that the abstraction/interface presented…

Most people use containers as a packaging format - instead of .tar.gz.

People run containers with host networking and bind-mounting everything and nobody minds.

(The story for hosting providers is a bit different, but they're not the main driver for Docker adoption.)

Post reply on HN