Live data from Hacker News

GCP Incidents

blog.railway.app

91–100 of 167 posts

Re: GCP Incidents

#91
post #60
post #32

Earlier quoted context omitted.

I’m not the GP but have been building websites off and on since the 1994. Back in the 90s it was hacky, but the extent of those hacks were small enough that even hobbyists could memorise those edge cases. Whereas these days there are so many footguns and edge cases that even seasoned professionals find it impossible to memorise everything. The amount of trial and error it takes to build a modern site is immense. With…

> Back in the 90s it was hacky, but the extent of those hacks were small enough that even hobbyists could memorise those edge cases. You're either misremembering, or have some thick rose-tinted glasses on. The late 90s were times of the Wild West Web. Every browser had custom behaviour and rendered pages differently. JavaScript and CSS were still new, and it took years for the implementations to be standardized. Webs…

> You're either misremembering, or have some thick rose-tinted glasses on.

> The late 90s were times of the Wild West Web.

Or we are talking about different eras. I'm on about early 90s.

> Every browser had custom behaviour and rendered pages differently.

They did. But there was less of spec to have to concern yourself with. The biggest day to day differences was around frames (which was a pain in the arse) and tables (which was easy to work around).

> JavaScript and CSS were still new, and it took years for the implementations to be standardized.

You didn't need Javascript most of the time and CSS incompatibilities were easy to remember (I'm talking about cognitive overhead here)

> Even as late as 2005, projects like jQuery were needed to make web development easier.

That's in the region of 10 years after when I'm talking about. A completely different era. By that point the web had already turned into the shitshow it is now.

> I'd say that it wasn't until the late aughts, well after "web 2.0", that web development didn't feel hacky.

I couldn't disagree more. By that point people had Stockholm syndromed themselves into believing web technologies were actually good. But it wasn't.

> So I get the sentiment that modern web development is difficult, bloated and buggy

Its not a sentiment. It's a fact

> the good news is that the web is fully mature now,

No its not. We're just Stockholm syndromed into thinking its mature. But half the technologies we use are constantly churning. That's not the definition of mature.

> Web developers are spoiled with choice, so a hard task is simply selecting the right technologies to use.

The hard part is finding something that will still be around in 5 years time.

> or we inherit decade-old codebases that make maintenance a nightmare.

I've worked on plenty of decade-old codebases and I'd never rank web development in there precisely because of the aforementioned churn. Web tech goes out of date so quickly that it never gets to live past a decade...never mind multiple decades. It's simply not a mature platform to write software on despite what you claim.

> But it's never been easier to write plain HTML/CSS/JS that will work across browser

Who writes plain HTML and JS? There's so much bloat required to get anything to look modern that nobody writes plain web sites any longer (In fact I did for one of my open source projects and people hated it and rewrote it in Vue).

It was much easier to write plain HTML et al in the 90s. In fact that was exactly how web development back then was done.

> and deploy it as a static site that works reliably at massive scale

That's literally how sites were originally written. It's not a new invention ;) The web was intended to be a static collection of documents. What we've since done is tried to turn it into an application framework. And that's what it sucks at.

> If you need more features, then complexity can creep in, but it is possible to minimize failure points with good engineering.

Sure, but again this isn't a new invention. This was the case right from the inception of the web. It's just gotten a hell of a lot harder to do good engineering for the web.

Re: GCP Incidents

#92
post #60

Earlier quoted context omitted.

> Back in the 90s it was hacky, but the extent of those hacks were small enough that even hobbyists could memorise those edge cases. You're either misremembering, or have some thick rose-tinted glasses on. The late 90s were times of the Wild West Web. Every browser had custom behaviour and rendered pages differently. JavaScript and CSS were still new, and it took years for the implementations to be standardized. Webs…

Agreed. The thing that has changed about the web is the expectations have increased. There is no way that people could make the websites users expect today with only technology from the 90s.

> There is no way that people could make the websites users expect today with only technology from the 90s.

That's my point though. The web should never have been bastardised into making applications. We should have seen the desires of web2.0 and built an entirely new software stack purpose built for online applications.

It wasn't fit for applications then and it still isn't now. It's just we keep trying to convince ourselves it is because we're too locked into the web now to ever move past it.

Re: GCP Incidents

#93

"On December 1st, at 8:52am PST, a box dropped offline; inaccessible. And then, instead of automatically coming back after failover — it didn’t. Our primary on-call engineer was alerted for this and dug in. While digging in, another box fell offline and didn’t come back" This makes no sense. A machine restarted and you had catastrophic failure? VMs reboot time to time. But if you design your setup to completely destr…

Read the article more carefully. The article (the text you quoted, even) clearly states that the machine didn't "restart". It crashed and didn't come back online.

And nowhere in the article do they state that this was a "catastrophic failure" - Railway itself didn't go down entirely. But Railway is a deployment company, so they are re-selling these compute resources to their customers to deploy applications. So when one of those VMs goes down and doesn't automatically failover, that's downtime for the specific customer who was running their service on that machine.

As they state:

> During manual failover of these machines, there was a 10 minute per host downtime. However, as many people are running multi-service workloads, this downtime can be multiplied many times as boxes subsequently went offline.

> For all of our users, we’re deeply sorry.

Re: GCP Incidents

#94

Earlier quoted context omitted.

Agreed. The thing that has changed about the web is the expectations have increased. There is no way that people could make the websites users expect today with only technology from the 90s.

The expectations of apps have skyrocketed too which is why everyone is using Electron to cope. If GUI development were still as easy as the drag and drop builders in Delphi 7 and VB6 with a bunch of static 800x600 monitors and double-click to edit component code-behind, we’d be shoehorning fewer apps into HTML and making more desktop apps. I remember how big of a pain in the ass Windows distribution (alone) was in th…

I was writing resizeable desktop applications in the 90s and it was easy then too. Differing screen resolutions is not a recent thing.

The reason for Electron's popularity is because:

1. web developers are cheaper and easier to hire than people with desktop software development experience

2. Electron is cross platform

Point 2 is a compelling reason. But it's not like other cross platform toolkits haven't existed. They've just been left to languish because of point 1.

If companies only hire web developers then of course everyone is going to invest time into writing Electron apps and banging on about the benefits of web applications. It becomes a self-fulfilling prophesy.

I say this with experience having watched the tides turn, first hand, as a software developer.

Re: GCP Incidents

#95
It's hilarious people are bashing GCP for having one compute instance go down and the author acknowledges it's a rare event. On AWS I've got instances getting forced stopped or even straight disappearing all the time. 99.95% durability vs 99.999% is way different.

If they had the same architecture on AWS it would go down all the time IME. AWS primitives are way less reliable than GCP, according to AWS' docs and my own experiences.

Re: GCP Incidents

#96
post #24

Interesting, I’m starting to think undocumented thresholds are quite common in GCP. I experienced something similar with Clod Run: inexplicable scaling events based on CPU utilization and concurrent requests (the two metrics that regulate scaling according to their docs). After a lot of back and forth with their (premium) support it turns out there are additional criteria, smthg related to request duration, but of co…

Yes, we have also experienced undocumented limits for Cloud Run. For us it was an obscure quota for max network packages per second per instance. Really infuriating and took 6 months to track down what it was. I think it has been documented here now: https://cloud.google.com/run/quotas#cloud_run_bandwidth_limi...

Re: GCP Incidents

#97
post #43

Earlier quoted context omitted.

To be fair, you probably should expect to be less proficient and satisfied embracing a stack that you weren't using before.

Writing GUIs can be easy. It’s just hard on the web. Some complexity is unavoidable, but the web stack is so broken it makes things 5x harder than it needs to be.

Do you have concrete examples that are much harder in the web than in native? When I started developing, I very much liked the web technologies for not being in my way - you have to learn their pitfalls and so on, but once you're up and running, you can do anything you want. Compared to that, every time I touch a native toolkit feels like an absolute nightmare the second you leave the trodden path. Even things like "changing the color of specific parts of a control" are often times either completely impossible without fully re-implementing it, or lead to bugs that make you wish you didn't try.

Re: GCP Incidents

#98
post #94

Earlier quoted context omitted.

The expectations of apps have skyrocketed too which is why everyone is using Electron to cope. If GUI development were still as easy as the drag and drop builders in Delphi 7 and VB6 with a bunch of static 800x600 monitors and double-click to edit component code-behind, we’d be shoehorning fewer apps into HTML and making more desktop apps. I remember how big of a pain in the ass Windows distribution (alone) was in th…

I was writing resizeable desktop applications in the 90s and it was easy then too. Differing screen resolutions is not a recent thing. The reason for Electron's popularity is because: 1. web developers are cheaper and easier to hire than people with desktop software development experience 2. Electron is cross platform Point 2 is a compelling reason. But it's not like other cross platform toolkits haven't existed. The…

I was there too with a Sony GDM-FW900 monitor with 2300x1400 resolution and my experience was very negative. Both as a user and as a developer (though I was just a kid getting started in ‘99). Any software that wasn’t made by a major company was a crapshoot and Delphi layout was brittle for my own apps. Anything that got deployed to university computers with their bulk purchased monitors was mostly fine except for the deployment process.

There are more web developers because it was a lot easier, even before npm existed and frontend frameworks went out of control. That’d didn’t happen in a vacuum - Electron was really late on the scene as far as frontend tech goes. It was originally made to support an IDE and not regular business apps and still won.

Re: GCP Incidents

#99
post #67

Earlier quoted context omitted.

But please don’t make web apps with Flutter. It has determinedly taken the pure-canvas route which makes it very unpleasant to use for a significant fraction of users, and all kinds of things just don’t work the right way and can’t . (I’ve written about the problems here on HN quite a few times, search and you’ll find them.)

For web, Flutter has 2 renderers and offers 3 options: - html: Uses HTML elements and CSS - canvaskit: which, as you mentioned, uses canvas to own the full drawing process - auto: defaults to canvaskit on web but html on mobile. Source: https://docs.flutter.dev/platform-integration/web/renderers

Curious, I haven’t heard of the HTML renderer. Is it any good?

Re: GCP Incidents

#100
post #57

We are a small software company (2 people) and we've also had plenty of issues with Google over the years. Mostly related to Google Adwords. For example: https://successfulsoftware.net/2015/03/04/google-bans-hyperl... https://successfulsoftware.net/2016/12/05/google-cpa-bidding... https://successfulsoftware.net/2020/08/21/google-ads-can-cha... https://successfulsoftware.net/2021/05/04/wtf-google-ads/ If Google have n…

> Google have no interest in providing support

No, they want to do things on a massive scale. Which means it is really difficult to talk to a human for support. And even if you manage it, it might be some badly trained subcontractor. But somehow there are always humans available to ring you up and tell you how you can spend more on Google Adwords.
Post reply on HN