Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS is not a dumb pipe

matt-rickard.com

Post reply on HN
Print view View original

51–60 of 90 posts

Re: AWS is not a dumb pipe

#51

Post by jonstewart »

> When there's no product differentiation, distribution wins. In the telco case, products build on top of broadband were 10x improvements over products built into broadband. YouTube, Netflix and other internet content providers could do things that cable and telephone simply couldn't. Now, competitors must differentiate on expertise, community, and developer experience.

I don’t disagree, but I think the important thing to understand is the flip side of this argument: products built on top of broadband were 10X better… and that created demand for the dumb pipe. If you wanted these new products, you had to order broadband. And then cable and telcos had local monopolies/oligopolies, so there was no need to differentiate themselves. Others created the demand and they locked customers into renewing contracts. Why be anything other than a dumb pipe in such a scenario?

Re: AWS is not a dumb pipe

#52

Post by denysvitali »

djbusby wrote: post #47
greiskul wrote: post #45

Earlier quoted context omitted.

I hate this practice that some companies do of testing their sql database usage... By using another database. For postgres, if you are using the jvm, you can use the amazing test containers library to test reasonably fast, with a postgres database that you can set up to be the same version as you use in production. Any database vendor should either support having their database used quickly in test like these, or at…

For PG you can just initdb in a tmpfs.

Or (and this can be done with pretty much everything) spin up a container

Re: AWS is not a dumb pipe

#53

Post by FridgeSeal »

glogla wrote: post #33
FridgeSeal wrote: post #13

Earlier quoted context omitted.

Isn’t this a bit of a moot point now though? If you’re going to pay someone for a data warehouse, you can pay AWS some money, or you can pay Snowflake a heinous amount of money, for something not significantly better? Mongo, ES and Redis all come with open source versions, which shifts the appeal back away from the AWS offerings, especially with things like Elastic Cloud for Kubernetes which has made our ES cluster b…

Snowflake is technically far superior to Redshift. The performance and features are somewhere else. Even if you pay more in operating cost (which is not always the case, especially if you count Redshift DBA salary), you get to do what you actually need to do much faster instead of fighting the platform. Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower, not a leader.…

> Snowflake is technically far superior to Redshift. The performance and features are somewhere else

From the discussions I’ve had about this before, I think I’m in the minority when I say I’m categorically unimpressed by Snowflakes performance.

Add that to the hideous cost, and the worlds most aggressive sales/account management team and I’ve less than zero desire to ever deal with them again.

> Snowflake is also technically going to always be far superior to Redshift, because AWS is a follower

While this is true, how many businesses actually need or actually utilise the features they’re paying for with things like Snowflake. Sure it’s got separate storage and compute, but how many places have so much data that they need that? I’ve worked with places that were running multi-node data warehouse clusters, that we migrated to a single ClickHouse or Postgres instance and got equivalent or better performance for a fraction of the economic and operational overhead.

> And if you really care about operating cost, you should be running Trino+Minio or Clickhouse onprem or something as your warehouse.

Can’t disagree here, although I’d say it shouldn’t just be relegated to operating cost: CH blows most alternatives out of the water and is available in hosted options now. Haven’t used Trino recently, last time I used it, it was still called Presto, and it was frustratingly slow, know if it’s improved recently?

Re: AWS is not a dumb pipe

#54

Post by [deleted] »

greiskul wrote: post #45
jrockway wrote: post #41

Earlier quoted context omitted.

Yeah, I would be really careful about relying on something like Aurora and planning to migrate to Cloud SQL or another vendor's managed PostgreSQL. Apps couple very tightly to implementation quirks of their database, and even the tiniest difference between implementations is going to cause problems when you migrate. (For example, we used to test our Postgres app with SQLite. We found that each query engine treats "WH…

I hate this practice that some companies do of testing their sql database usage... By using another database. For postgres, if you are using the jvm, you can use the amazing test containers library to test reasonably fast, with a postgres database that you can set up to be the same version as you use in production. Any database vendor should either support having their database used quickly in test like these, or at…

[deleted]

Re: AWS is not a dumb pipe

#55

Post by shepherdjerred »

glogla wrote: post #28
didip wrote: post #17

But it can be a dumb pipe. More and more companies prefer that. Just deploy everything on EKS and use only foundational services such as S3 and PostgreSQL Aurora. There you go: Dumb Pipes. Edit: When I say dumb, it's more like: Dumb enough so that another public cloud vendor, such as GCP or Azure, can be swapped in.

Honestly, it should be. AWS is much better at the dumb, commodity services like s3, ec2 and rds (that are actually pretty smart) than they are at something like redshift, sagemaker or similar.

Some of their more complex/bespoke services are very good, like Lambda, CloudWatch, and DynamoDB.

Quality declines steeply once you're off the beaten path.

Re: AWS is not a dumb pipe

#56

Post by deanCommie »

xg15 wrote: post #46
oneplane wrote: post #35

AWS isn't a single "thing", or perhaps, it's only a bundled brand. There are many teams, sub-companies, divisions, groups all working on their own parts. Most have to follow a similar set of rules and some attempts are made to make it cohesive, but in the end it's not a single minded entity with a single set of parameters it follows for every service offering. You can run your stuff on AWS without ever touching compu…

Ok, but by that logic it would be impossible to make any kind of analysis about AWS (or other cloud providers) in the sense that the author attempted to make. In the end I believe, the question is about lock-in and control. In a true "dumb pipe" offering, you could just take your software elsewhere and run essentially the same code without changes. Meanwhile, if you use various proprietary APIs, this is not the case.…

I think "lock-in" is a terrible term.

You can avoid all lock-in by ignoring all sorts of proprietary features of any vendor, and leave ENDLESS benefits and optimizations on the table for the hypothetical likelyhood that you might one day have to "free yourself".

Conversely, if you embrace those things wholeheartedly, and at some point find yourself wanting to break the relationship and do a full egress migration, even if your costs end up monumental, I would bet anything that over the time you were "locked in" you will have gained more from that cloud than whatever "one-time migration costs" you have to pay.

If your core business proposition is literally something like storage (e.g. Dropbox) where with time you will want to vertically integrate your whole technology stack rather than relying on a 3rd party, yeah avoid lock-in.

If you have a frivolous C-suite that will want to change cloud providers every 24 months depending on what discount they can negotiate on the golf course, yeah avoid lock-in.

Everyone else? Throw that word out from your vocabulary, and embrace the idiomatic capabilities of your cloud vendor.

Re: AWS is not a dumb pipe

#57

Post by closeparen »

greiskul wrote: post #45
jrockway wrote: post #41

Earlier quoted context omitted.

Yeah, I would be really careful about relying on something like Aurora and planning to migrate to Cloud SQL or another vendor's managed PostgreSQL. Apps couple very tightly to implementation quirks of their database, and even the tiniest difference between implementations is going to cause problems when you migrate. (For example, we used to test our Postgres app with SQLite. We found that each query engine treats "WH…

I hate this practice that some companies do of testing their sql database usage... By using another database. For postgres, if you are using the jvm, you can use the amazing test containers library to test reasonably fast, with a postgres database that you can set up to be the same version as you use in production. Any database vendor should either support having their database used quickly in test like these, or at…

Much better than mocking out cursor objects!

Re: AWS is not a dumb pipe

#58

Post by konaraddi »

> On the other hand, that means that services built over-the-top pay a higher cloud tax. Not sure how this one plays out.

It’s unlikely services will continue to build on top and pay a high cloud tax for two reasons:

1. It’s not safe to bet on any entity remaining at the top forever, applies to corporations being on the top of their game too.

2. Tech tends to be an upward moving pendulum as it converges on an optimal solution. Back in the day, companies owned most of their stack from top down (initial position of pendulum) and now companies own only the top part of their stack by building on top of Azure/AWS/GCP (pendulum swing). There’ll be a pendulum swing back but it’ll be an improvement over the status quo for at least some use cases.

It’s likely new technologies and regulations will make building/owning deep vertical stacks a viable option and it’ll be worth it for some use cases. For example, visual/audio stacks should be deep and owned by an entity, I suspect there’s a large margin of improvement in this area that can be attained with fine-tuned vertical stacks (e.g. https://tonari.no/ is doing some low level stuff to build a better audio/visual experience in meetings). I think companies like https://oxide.computer/ will inadvertently, collectively help make building deep vertical stacks an option on the table.

Re: AWS is not a dumb pipe

#59

Post by gz5 »

A difference between telcos and IaaS is IT is 'baked into' AWS. AWS's IT team (people, software, automation) silently becomes an extension of your team. It makes AWS more than a dumb pipe, and makes it difficult to leave AWS (generally you have to be very purposeful to avoid this lock-in).

Interestingly, considering the OP's comparison, many enterprises are also tethered to AWS via telco dumb pipes (e.g. App user -> WAN -> Private DC -> Colo facility -> Direct Connect (MPLS) -> app in VPC).

Re: AWS is not a dumb pipe

#60

Post by AlphaSite »

throwaway2048 wrote: post #44
tjr225 wrote: post #31

Earlier quoted context omitted.

Running your own Kubernetes cluster is complex and deep. Giving a yaml file to EKS and suddenly you have infrastructure is not complex at all. Its actually pretty sweet.

You are confusing simplicity of usage with simplicity of implementation. "dumb pipes" implement simple services for fixed or low per unit costs, the entire appeal of ready to use complex stacks like EKS is simplicity of usage.

It’s dumb in the sense that you can just move clouds without issue. You’re not using the secret special sauce that binds you tightly to AWS, everyone has a K8s and they’re all broadly the same.
Post reply on HN