Live data from Hacker News

Software Infrastructure 2.0: A Wishlist

erikbern.com

111–120 of 202 posts

Re: Software Infrastructure 2.0: A Wishlist

#111
post #5

Testing in 'production' has been possible for a while: Heroku has Review Apps, and Render (where I work) has Preview Environments [1] that spin up a high-fidelity isolated copy of your production stack on every PR and tear it down when you're done. [1] https://render.com/docs/preview-environments

This is cool - does it also copy over the data and schema of the production db? For instance on an eCommerce site, will I get my production product data to test on in stage?

Re: Software Infrastructure 2.0: A Wishlist

#112
post #79
post #69

Earlier quoted context omitted.

I'm slightly afraid of all the negativity here, but still kinda agree with the sentiment. However, I still want to note that there are many cases where smart people simply don't have enough time to handle all the stupidity in their products. Often times, it's just little issues like communication cost and politics. But, also often, one should care about the revenue of one's own company or clients', which slows down c…

The smartest people can build utterly useless crap if the teams they work on don’t prioritize UX. I think of it as “emergent stupidity”; individually they will optimize for and build the best components but if they aren’t well put together the UX is horrible and the product sucks sucks sucks. On the contrary, users will be ok with suboptimal components if the overall UX is good.

> On the contrary, users will be ok with suboptimal components if the overall UX is good.

Until they -absolutely- need quality components, then no matter good the UI/UX is, you're out of luck.

Re: Software Infrastructure 2.0: A Wishlist

#113

Years and years ago I saw an advertisement by a SAN array vendor where their gimmick was that they supported very cheap read/write snapshots of very large datasets, in combination with a set of transforms such as data masking or anonymisation. Their target market was same as the OP: developers that need rapid like-for-like clones of production. The SAN could create a full VM server pool with a logical clone of hundre…

Your comment makes me realize how lucky I am at my job: we do this production environment replication via “SAN magic” and it is so insanely useful.

How do you deal with replicating customer data / data the developer / tester should not have access to?

Do you copy that into the ephemeral test environment as well? How are permissions managd for accessing that data? Does it have the same restrictions as in prod? (i.e. even in this env the developer / tester has no access to systems / data they would not have in prod).

Re: Software Infrastructure 2.0: A Wishlist

#114

Earlier quoted context omitted.

> Azure's network team decided to unnecessarily use NAT for IPv6, making it exactly (100%!) as complex as IPv4... People like the NAT. It's a feature, not a bug. If your selling point for IPv6 is "no more NAT" then no wonder it never went anywhere! P.S. No, "you're doing it wrong" and "you're not allowed to like the NAT" are not valid responses to user needs.

What do people like about NAT? I am guessing that the perceived increase in security. But perhaps there are more real or perceived advantages.

At home I really appreciate the NAT. I'm glad not every device has a public IP and gets hammered with attacks 24/7. It's not fail-safe but it definitely adds some security..

As for cloud, no idea what the benefit should be there.

Re: Software Infrastructure 2.0: A Wishlist

#115
post #46
post #30

This post mingles different interpretations of "serverless" - the current fad of "lambda" style functions etc and premium / full stack managed application offerings like Heroku. Although they are both "fully managed" they have a lot of differences. Both though assume the "hard" problem of infrastructure is the one-time setup cost, and that your architecture design should be centered around minimising that cost - even…

what would you say is the architectural sacrifice of pushing things out to the edge, or using lambda functions per say in your opinion? There are issues about resolving state at the edge, though Cloudflare has solved some of that with durable objects, and Fastly is working on something too. Think there are memory constraints as of now however. Security too is better in these platforms at the edge - true zero trust mo…

Not the person you asked but for me, the discussion of Edge computing often misses if there are actually enough benefits. Slow response times are nearly always because the application is slow, not because the server is 100ms away instead of 5ms. Saving 100ms (or less for most probably) by deploying logic at the edge introduces much more complexity than shaving that time off through optimization would. Take this page as an example. I get response times of ~270ms (Europe to US-West I presume) and it still loads instantly. On the other hand I have enough systems at my office where I'm I don't say that you don't need edge computing. But it'll always be more complex to run compared to a centralized system (plus vendor lock-in) and performance isn't a valid reason.

Re: Software Infrastructure 2.0: A Wishlist

#116
>Truly serverless

>The beauty of this is that a lot of the configuration stuff goes away magically. The competitive advantage of most startups is to deliver business value through business logic, not capacity planning and capacity management!

Is this exactly what cloudflare's workers is doing? [0]

I love the fact that I only need to think of the business logic.

Development without the need for VMs and obscure configuration scripts feels — and is in fact very productive. I don't think I will want to use anything else for hosting my back ends.

[0]:https://workers.cloudflare.com/

Re: Software Infrastructure 2.0: A Wishlist

#117

Earlier quoted context omitted.

Your comment makes me realize how lucky I am at my job: we do this production environment replication via “SAN magic” and it is so insanely useful.

How do you deal with replicating customer data / data the developer / tester should not have access to? Do you copy that into the ephemeral test environment as well? How are permissions managd for accessing that data? Does it have the same restrictions as in prod? (i.e. even in this env the developer / tester has no access to systems / data they would not have in prod).

This is an interesting question.

I've seen 1:1 copy of prod databases running in test environments. Very convenient from a dev perspective but a no-go for many customers.

I've also seen test envs using fixtures only. More privacy-compliant, but it makes it harder for in-depth testing or to uncover weird edge cases.

Re: Software Infrastructure 2.0: A Wishlist

#119
> The word cluster is an anachronism to an end-user in the cloud! I'm already running things in the cloud where there's elastic resources available at any time. Why do I have to think about the underlying pool of resources? Just maintain it for me.

Because very often, software that runs on clusters, requires thought when resizing.

In fact, I'm trying to think of a distributed software product where resizing a cluster, doesn't require a human in the loop making calculated decisions.

Even with things like Spark where you can easily add a bunch of worker nodes to a cluster in the cloud, there's still core nodes maintaining a shared filesystem for when you need to spill to disk.

Re: Software Infrastructure 2.0: A Wishlist

#120

Years and years ago I saw an advertisement by a SAN array vendor where their gimmick was that they supported very cheap read/write snapshots of very large datasets, in combination with a set of transforms such as data masking or anonymisation. Their target market was same as the OP: developers that need rapid like-for-like clones of production. The SAN could create a full VM server pool with a logical clone of hundre…

> Meanwhile, in the Azure cloud: Practically nothing has a "copy" option. Deploying basic resources such as SQL Virtual Machines or App Service environments can take hours. Snapshotting a VM takes a full copy. Etc...

To add to the list of etc, I can't query across two databases in azure SQL.

Post reply on HN