Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software Infrastructure 2.0: A Wishlist

erikbern.com

Post reply on HN
Print view View original

141–150 of 202 posts

Re: Software Infrastructure 2.0: A Wishlist

#141

Post by otabdeveloper4 »

dijit wrote: post #98
otabdeveloper4 wrote: post #88

Earlier quoted context omitted.

> Azure's network team decided to unnecessarily use NAT for IPv6, making it exactly (100%!) as complex as IPv4... People like the NAT. It's a feature, not a bug. If your selling point for IPv6 is "no more NAT" then no wonder it never went anywhere! P.S. No, "you're doing it wrong" and "you're not allowed to like the NAT" are not valid responses to user needs.

> "you're not allowed to like the NAT" Well, the bigger question might be why do they like NAT? If it's about having a single /128 address so they can do ACLs then that's easily fixed by just lowering the CIDR number. (unless you have an ancient version of fortigate on prem, which likely doesn't work with ipv6 anyway). If it's about not having things poking at your servers through the NAT then the "NAT" really isn't…

Again: "you're doing it wrong" and "you're not allowed to like the NAT" are not valid criticisms.

People like NAT because it's an easy batteries-included way to manage, secure and understand your LAN.

Taking it away and forcing them to migrate to an incompatible zoo of firewall technologies for no benefit is asinine.

> They're just sold to consumers as a single package.

Exactly. How in the world is this a bad thing now? Do we really want to make network security for the average do-it-yourself home LAN harder?

Re: Software Infrastructure 2.0: A Wishlist

#143

Post by lovedswain »

Archelaos wrote: post #121
rurounijones wrote: post #113

Earlier quoted context omitted.

How do you deal with replicating customer data / data the developer / tester should not have access to? Do you copy that into the ephemeral test environment as well? How are permissions managd for accessing that data? Does it have the same restrictions as in prod? (i.e. even in this env the developer / tester has no access to systems / data they would not have in prod).

Using real person-related data for testing was made illegal in Europe by GDPR. Even with explicit consent of each affected person, the tester needs to make sure that the test really requires real person-related data, otherwise it would violate the principle of "privacy by design" from GDPR. Without consent it is entirely forbidden, as of my understanding of the GDPR. Due to GDPR I had to implement an anonymization fe…

This sounds like a needlessly strict interpretation of GDPR. Taken from the UK regulator's site:

> The lawful bases for processing are set out in Article 6 of the UK GDPR. At least one of these must apply whenever you process personal data:

> (a) Consent: the individual has given clear consent for you to process their personal data for a specific purpose.

> (b) Contract: the processing is necessary for a contract you have with the individual, or because they have asked you to take specific steps before entering into a contract.

> (f) Legitimate interests: the processing is necessary for your legitimate interests or the legitimate interests of a third party, unless there is a good reason to protect the individual’s personal data which overrides those legitimate interests. (This cannot apply if you are a public authority processing data to perform your official tasks.)

...

> Legitimate interests is the most flexible lawful basis for processing, but you cannot assume it will always be the most appropriate. It is likely to be most appropriate where you use people’s data in ways they would reasonably expect and which have a minimal privacy impact, or where there is a compelling justification for the processing.

> The processing must be necessary. If you can reasonably achieve the same result in another less intrusive way, legitimate interests will not apply.

> You must include details of your legitimate interests in your privacy information.

I included the legitimate interests bits because they seem most relevant to testing, but even if testing is not considered "necessary" in a particular use case, there still remain at least two more criteria that might satisfy the use of live data in testing, including explicit user consent. Much of the focus of GDPR is on privacy-invasive intrusive processing and prevention of harm, I think a lot of fuss around it can be dispelled when viewed from this angle.

Re: Software Infrastructure 2.0: A Wishlist

#144

Post by jen20 »

throwaway823882 wrote: post #140
jen20 wrote: post #139

Earlier quoted context omitted.

> running terraform apply is unpredictable, due to the nature of the design of the program, even though it was created with the opposite intention Citation needed. Run _with a plan_, terraform apply is entirely predictable as to the operations it will carry out and the order in which it will carry them out.

You are incorrect. edit according to your wording, you are sometimes correct. But terraform plan often can't show values that it can't predict ("Values known after apply"), so you often can't predict what apply will do. And you can never know for sure if the operations will succeed. First of all, what does terraform apply do? It runs API calls. Those API calls may or may not work, based on criteria outside of the con…

> edit actually, according to your wording, you are correct.

That’s uhh... why I chose that wording.

> Use AWS enough and you will run into this weekly.

I _really_ wish you understood how amusing this exhortation is.

I’m under no illusion about the reliability of AWS APIs - much less Azure or any of the other major public clouds.

The easiest way to reduce drift is to prevent read/write access via anything except a designated service account for Terraform. Per one of your earlier replies about a coworker having upgraded TF underneath you, it sounds like you aren’t doing that.

I’m with you on some aspects of your rant - specifically precious resource consumption such as names - for what it’s worth (and am a former core maintainer of Terraform, and still work with that codebase daily elsewhere), but if you’re going to rant about a project whose maintainers (or former maintainers) may be listening, you’d better be technically correct.

I’d suggest issues (yes I know there are a lot of them, I don’t know how they get triaged anymore) or mailing list posts, or contributions as a more helpful way to engage rather than complaining on an internal Slack channel.

Re: Software Infrastructure 2.0: A Wishlist

#145

Post by aliswe »

erikerikson wrote: post #60
qaq wrote: post #55

Given how much capitalization AWS and Azure account for it's surprising how little progress has being made in many areas. A true serverless RDBMS database priced sanely that basically charged for qry execution time and storage without need to provision some "black box" capacity units and would be suitable for a large scale app. Most of all workflows that require contacting support and arbitrary account limits that fo…

Both Dynamo DB and Cosmos DB offer effectively what you're asking for. Limits are a risk that must be mitigated but they aren't arbitrary, they exist to avoid interference of one customer on to others. They protect the service which, to providers and reasonably so, is more important than any one customer.

CostmostDB is not sanely priced IMO!

Re: Software Infrastructure 2.0: A Wishlist

#146

Post by otabdeveloper4 »

w7 wrote: post #100
funcDropShadow wrote: post #93

Earlier quoted context omitted.

What do people like about NAT? I am guessing that the perceived increase in security. But perhaps there are more real or perceived advantages.

What I find common is people conflate NAT with stateful firewalling, and believe that if you lose NAT you lose all forms of edge/perimeter network security. They don't understand that you can still filter and prevent unwanted packets from reaching hosts without NAT.

Of course you can. But why would you? You're replacing something that is simple, easy to understand and works perfectly well with a nebulous something that invites user error and security nightmares.

For example, my (modest) home LAN is five routers, a NAS/media server, a media player, two "smart TVs" and dozens of notebooks and phones connected via Wi-Fi.

What do you propose? Manage a firewall on each of those devices?

I suppose you mean setting up a firewall on the WAN link to block all incoming traffic? How is that different from a NAT? Merely a lack of 'masquerade' setting on the firewall rule? What's the benefit to me and why should I care?

Or do you propose some sort of hybrid scheme to intelligently block traffic while making all my countless devices pingable from the Internet? Not in this timeline, sorry.

Re: Software Infrastructure 2.0: A Wishlist

#147

Post by Throaway838921 »

My biggest problem with modern infrastructure is how much resources they use for simple things. They scale upwards, but they don't scale down.

I tried OpenFAAS in Kubernetes, to see if I could run simple lambda like loads. Default setup took north of 8 Gb of memory!

I just want efficient and simple microservices infrastructure, that doesn't hog 8 Gb of memory to run a simple functions or small microservices.

Re: Software Infrastructure 2.0: A Wishlist

#148

Post by nonameiguess »

mikewarot wrote: post #27

Virtualization eventually will be seen as the unnecessary layer added to make up for operating systems that lack capability based security. It's going to take a decade to refactor things to remove that layer. Once done, you'll be able to safely run a process against a list of resources.

It's not just security. There's incompatible dependencies. Virtualization allows you to run Windows and Linux on the same server, 40 different C libraries with potentially incompatible ABIs, across X time zones and Y locales, and they'll all work.

You can provide the same level of replication and separation of dependencies in an OS, but at a certain point you're just creating the same thing as a hypervisor and calling it an OS and the distinction becomes academic.

Re: Software Infrastructure 2.0: A Wishlist

#149

Post by lnsp »

I am currently working on a project attempting to achieve something like this called Valar (https://valar.dev). It's still quite experimental and in private beta, but feel free to take a look and put yourself on the waiting list. I'm targeting small, private projects so it's not really suited for full-on production workloads yet.

The core of it is a serverless system that shrinks workload quotas while running them (similar to the Google Autopilot system). This way you don't have to rewrite your entire codebase into lambdas and the end user can save a lot of costs by only specifying a resource cap (but probably using a lot less, therefore paying less).

Post reply on HN