Software Infrastructure 2.0: A Wishlist • Hacker News Board

Re: Software Infrastructure 2.0: A Wishlist

#51

Post by w7 » Tue, Apr 20, 2021, 4:03 AM UTC

mikewarot wrote: post #27

Virtualization eventually will be seen as the unnecessary layer added to make up for operating systems that lack capability based security. It's going to take a decade to refactor things to remove that layer. Once done, you'll be able to safely run a process against a list of resources.

We've had capability based security frameworks aka MAC (ex: AppArmor) in Linux since 1999 or earlier. Containers (which also existed long before docker) have been popularized for convenience, and virtualization would still be useful for running required systems that are not similar to the host. If anything it looks like we're going towards a convergence with "microvms".

Re: Software Infrastructure 2.0: A Wishlist

#52

Post by _waqas_ali_ » Tue, Apr 20, 2021, 4:10 AM UTC

throwaway823882 wrote: post #23

There will not be a massive productivity boost in the next 10 years. We haven't had a productivity boost in the past 20 years. In fact, we're much less productive now. But I digress. We're still largely making and running software the same way we did 20 years ago. The only major differences are the principles that underpin the most modern best practices. "What the fuck does that mean," you say? It means that I can ta…

Keep your s3 config in version control, terraform apply, if it doesn't work revert the commit and terraform apply again. The DSL isn't that hard: https://registry.terraform.io/providers/hashicorp/aws/latest...

Re: Software Infrastructure 2.0: A Wishlist

#53

Post by dilyevsky » Tue, Apr 20, 2021, 4:28 AM UTC

jiggawatts wrote: post #47

Years and years ago I saw an advertisement by a SAN array vendor where their gimmick was that they supported very cheap read/write snapshots of very large datasets, in combination with a set of transforms such as data masking or anonymisation. Their target market was same as the OP: developers that need rapid like-for-like clones of production. The SAN could create a full VM server pool with a logical clone of hundre…

gcp doesn’t even give you ipv6 at all. Talk about limitations...

Re: Software Infrastructure 2.0: A Wishlist

#54

Post by pgwhalen » Tue, Apr 20, 2021, 4:34 AM UTC

jiggawatts wrote: post #47

Years and years ago I saw an advertisement by a SAN array vendor where their gimmick was that they supported very cheap read/write snapshots of very large datasets, in combination with a set of transforms such as data masking or anonymisation. Their target market was same as the OP: developers that need rapid like-for-like clones of production. The SAN could create a full VM server pool with a logical clone of hundre…

Your comment makes me realize how lucky I am at my job: we do this production environment replication via “SAN magic” and it is so insanely useful.

Re: Software Infrastructure 2.0: A Wishlist

#55

Post by qaq » Tue, Apr 20, 2021, 4:35 AM UTC

Given how much capitalization AWS and Azure account for it's surprising how little progress has being made in many areas. A true serverless RDBMS database priced sanely that basically charged for qry execution time and storage without need to provision some "black box" capacity units and would be suitable for a large scale app. Most of all workflows that require contacting support and arbitrary account limits that force you to split solutions across multiple accounts wtf.

Re: Software Infrastructure 2.0: A Wishlist

#56

Post by anurag » Tue, Apr 20, 2021, 4:44 AM UTC

sebmellen wrote: post #39

anurag wrote: post #5

Testing in 'production' has been possible for a while: Heroku has Review Apps, and Render (where I work) has Preview Environments [1] that spin up a high-fidelity isolated copy of your production stack on every PR and tear it down when you're done. [1] https://render.com/docs/preview-environments

Hey, a question about Render — do you run your own base infra/colocate or do you use a VPS/bare metal setup or do you sit on top of a cloud? I love the service but I’m wondering if you’ll expand outside of the two available regions anytime soon. I understand this might be hard based on your infra setup.

> I love the service

Thanks!

We will absolutely expand beyond Oregon and Frankfurt later this year. Our infrastructure was built to be cloud agnostic: we run on AWS, GCP, and Equinix Metal.

Re: Software Infrastructure 2.0: A Wishlist

#57

Post by erikerikson » Tue, Apr 20, 2021, 4:45 AM UTC

bob1029 wrote: post #2

I love the test-in-production illustration. This is a fairly accurate map of my journey as well. I vividly remember scolding our customers about how they needed a staging environment that was a perfect copy of production so we could guarantee a push to prod would be perfect every time. We still have that customer but we have learned a very valuable & expensive lesson. Being able to test in production is one of the mo…

IMO this still keeps production a previous thing. Next comes multi-version concurrent environments on the same replicated data streams. There is only production and delivery to customers is access version 3 instead of version 2. This can be proceeded by an automated comparison of v2's outputs to v3's and a review (or code description of the expected differences).

Re: Software Infrastructure 2.0: A Wishlist

#58

Post by erikerikson » Tue, Apr 20, 2021, 4:50 AM UTC

zmmmmm wrote: post #30

This post mingles different interpretations of "serverless" - the current fad of "lambda" style functions etc and premium / full stack managed application offerings like Heroku. Although they are both "fully managed" they have a lot of differences. Both though assume the "hard" problem of infrastructure is the one-time setup cost, and that your architecture design should be centered around minimising that cost - even…

Serverless is more than functions. Consider S3, LogicApps, or DataFlow.

Serverless is about starting with highly available, auto scaling, et cetera... assets and concentrating more on delivering business value.

Re: Software Infrastructure 2.0: A Wishlist

#59

Post by intergalplan » Tue, Apr 20, 2021, 4:51 AM UTC

ngngngng wrote: post #21

> but it takes 45 steps in the console and 12 of them are highly confusing if you never did it before. I am constantly amazed that software engineering is as difficult as it is in ways like this. Half the time I just figure I'm an idiot because no one else is struggling with the same things but I probably just don't notice when they are.

1) It's just as shitty for everyone except the person who's selling themselves as a "consultant" for X. And actually—they're lying, it's shitty for them too.

2) A bunch of other goddamn morons managed to use this more-broken-than-not thing to make something useful, so this goddamn moron (i.e. me) surely can.

Both almost always true. After the second or third time you realize something made by the "geniuses" at FAANG (let alone "lesser" companies) is totally broken, was plainly half-assed at implementation and in fact is worse than what you might have done (yes, even considering management/time constraints), and has incorrect documentation, you start to catch on to these two truths. It's all shit, and most people are bad at it. Let that be a source of comfort.

[EDIT] actually, broader point: the degree to which "fucking around with shitty tools" is where one spends one's time in this job is probably understated. This bull-crap is why some people "nope" the hell out and specialize in a single language/stack, so at least they only have one well-defined set of terrible, broken crap to worry about. Then people on HN tell them they're not real developers because they aren't super-eager to learn [other horribly-broken pile of crap with marginal benefits over what they already know]. (nb. I'm in the generalist and willing-to-learn-new-garbage category, so this judgement isn't self-serving on my part)

Re: Software Infrastructure 2.0: A Wishlist

#60

Post by erikerikson » Tue, Apr 20, 2021, 4:54 AM UTC

qaq wrote: post #55

Given how much capitalization AWS and Azure account for it's surprising how little progress has being made in many areas. A true serverless RDBMS database priced sanely that basically charged for qry execution time and storage without need to provision some "black box" capacity units and would be suitable for a large scale app. Most of all workflows that require contacting support and arbitrary account limits that fo…

Both Dynamo DB and Cosmos DB offer effectively what you're asking for.

Limits are a risk that must be mitigated but they aren't arbitrary, they exist to avoid interference of one customer on to others. They protect the service which, to providers and reasonably so, is more important than any one customer.