Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless Horrors

serverlesshorrors.com

Post reply on HN
Print view View original

61–70 of 503 posts

Re: Serverless Horrors

#61

Post by Ekaros »

fishmicrowaver wrote: post #23
thousand_bats wrote: post #16

Earlier quoted context omitted.

> I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. Agreed about that. I was hired onto a team that inherited a large AWS Lambda backend and the opacity of the underlying platform (which is the value proposition of serverless!) has made it very painful when the going gets tough and you find bugs in your system down close to that layer (in our c…

I've had similar experiences with Azures services. Black boxes impossible to troubleshoot. Very unexpected behavior people aren't necessarily aware of when they initially spin these things up. For anything important I just accept the pain of deploying to kubernetes. Developers actually wind up preferring it in most cases with flux and devsoace.

I recently had customer who had smart idea to protect Container Registry with firewall... Breaking pretty much everything in process. Now it kinda works after days of punching enough holes in... But I still have no idea where does something like Container registry pull stuff from, or App Service...

And does some of their suggested solutions actually work or not...

Re: Serverless Horrors

#62

Post by jsheard »

sherburt3 wrote: post #7
joshstrange wrote: post #4

I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. I scrolled aimlessly through the site ignoring most posts (bandwidth usage bills aren’t super interesting) but I did see this one: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-c... About how you make unauth’d API calls to an s3 bucket you don’t own to run up the costs. That was a n…

I believe they changed that shortly after that blog post went viral: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3...

You have to wonder how many people quietly got burned by that in the 18 years between S3 launching and that viral post finally prompting a response.

Re: Serverless Horrors

#63

Post by pooper »

franktankbank wrote: post #42
pooper wrote: post #37

Earlier quoted context omitted.

> Seems an interesting oversight. I can just imagine the roundtable, uhh guys who do we charge for 403? Who can we charge? But what if people hit random buckets as an attack? Great! It is amazing, isn't it? Something starts as an oversight but by the time it reaches down to customer support, it becomes an edict from above as it is "expected behavior". > AWS was kind enough to cancel my S3 bill. However, they emphasiz…

Even pooper is upset about the stench. Tech is fuckin dumb in the corps, the only logical explanation to me is kickbacks to the CTO or similar.

Pooping at the job is one thing but pooping at the job and trying to sell it as a favor to the customer is a whole different game.

Re: Serverless Horrors

#64

Post by hvb2 »

phoenixhaber wrote: post #43

When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…

> The entire service with it's horrifically complicated click through dashboard (but you can get a certification! It's so complicated they invented a fake degree for it!) just to confuse the customer into losing money.

By that logic, any technology that you can get certified in is too complicated?

Most systems are now distributed and presenting a holistic view of how it was designed to work can be useful to prevent simple mistakes.

Traffic requires a certification (license) too. Must be a fake degree as well because they made it too complicated

Re: Serverless Horrors

#65

Post by jsheard »

kdps wrote: post #51
rwmj wrote: post #25

Earlier quoted context omitted.

Why isn't that the default? I'm glad I use a Hetzner VPS. I pay about EUR 5 monthly, and never have to worry about unexpected bills.

Don't they charge for every TB exceeding the included limit? (website says "For each additional TB, we charge € 1.19 in the EU and US, and € 8.81 in Singapore.")

They do, but the risk of having to pay $1.44/TB after the first 20TB is easier to swallow than say, CloudFront's ~$100/TB after 1TB.

Re: Serverless Horrors

#66

Post by api »

There should also be a general category for "cloud horrors" for things that cost $50k/month to host that would be $1500/month on a bare metal provider like Datapacket or Hetzner.

I'm old enough to remember when cloud was pitched as a big cost saving move. I knew it was bullshit then. Told you so.

Re: Serverless Horrors

#67

Post by emseetech »

This is why when I contract for an early stage startup, I pose the question:

"What if your app went viral and you woke to a $20k cloud bill? $50k? $80k?"

If the answer is anything less than "Hell yeah, we'll throw it on a credit card and hit up investors with a growth chart" then I suggest a basic vps setup with a fixed cost that simply stops responding instead.

There is such a thing as getting killed by success and while it's possible to negotiate with AWS or Google to reduce a surprise bill, there's no guarantee and it's a lot to throw on a startup's already overwhelming plate.

The cloud made scaling easier in ways, but a simple vps is so wildly overpowered compared to 15 years ago, a lot of startups can go far with a handful of digitalocean droplets.

Re: Serverless Horrors

#68

Post by ctvo »

phoenixhaber wrote: post #43

When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…

I took a workshop class and was told to setup a track saw. The course didn't bother explaining how to utilize it properly or protect yourself. I ended up losing a finger. I truly hate Stanley Tools with a passion and if I ever need to use another track saw, I'll use someone else.

Re: Serverless Horrors

#69

Post by hvb2 »

caboteria wrote: post #55

The real serverless horror isn't the occasional mistake that leads to a single huge bill, it's the monthly creep. It's so easy to spin up a resource and leave it running. It's just a few bucks, right? I worked for a small venture-funded "cloud-first" company and our AWS bill was a sawtooth waveform. Every month the bill would creep up by a thousand bucks or so, until it hit $20k at which point the COO would notice an…

You don't think this happens on prem? Servers running an application that is no longer used?

Sure they're probably VMs but their cost isn't 0 either

Re: Serverless Horrors

#70

Post by trillic »

Ekaros wrote: post #61
fishmicrowaver wrote: post #23

Earlier quoted context omitted.

I've had similar experiences with Azures services. Black boxes impossible to troubleshoot. Very unexpected behavior people aren't necessarily aware of when they initially spin these things up. For anything important I just accept the pain of deploying to kubernetes. Developers actually wind up preferring it in most cases with flux and devsoace.

I recently had customer who had smart idea to protect Container Registry with firewall... Breaking pretty much everything in process. Now it kinda works after days of punching enough holes in... But I still have no idea where does something like Container registry pull stuff from, or App Service... And does some of their suggested solutions actually work or not...

Convince them to add IPv6 and you’ll be set for life
Post reply on HN