Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What software engineers can learn from the rapid collapse of Fast

newsletter.pragmaticengineer.com

Post reply on HN
Print view View original

121–130 of 482 posts

Re: What software engineers can learn from the rapid collapse of Fast

#121

Post by [deleted] »

lolsoftware wrote: post #33
mwcampbell wrote: post #20

I think the most applicable warning for engineers when it comes to the actual work, as opposed to whether one should join a particular startup, is this: > Engineers calculated the load Fast had in needing to serve their traffic. The Fast button was rendered less than 500,000 times per day - rarely needed to ever serve more than a few requests per second. > One of the few warning signs engineers noticed is how Fast sp…

The TailScale folks caught a bunch of flak for their "do things that don't scale" approach to databases. But, honestly, most startups would be better off following that approach than what Fast did. Sounds like the engineers were just entertaining themselves with shiny toys rather than solving the problems they actually had.

[deleted]

Re: What software engineers can learn from the rapid collapse of Fast

#122

Post by cush »

odonnellryan wrote: post #65
Animats wrote: post #62

"Most engineers joining didn't know much about why the one-click checkout industry has the potential of billions." So why isn't this a standard feature of every shopping cart program, including the cheap ones? The complicated part is that you need "undo", valid for a while after ordering. That's what makes one-click buy feel safe for customers. This complicates inventory management. But you really need "undo" for an…

Lots of sites do not have a good undo for orders. I ordered from remarkable and their solution was to deny the package...

I ordered a piece of furniture from Wayfair, and they had the same response - deny the package.

Well the package arrived while I was at work. After calling them 3 times to come pick it up, due to shipping delays they were never able to, and eventually told me to keep it for free.

Re: What software engineers can learn from the rapid collapse of Fast

#123

Post by [deleted] »

hobs wrote: post #69
lolsoftware wrote: post #33

Earlier quoted context omitted.

The TailScale folks caught a bunch of flak for their "do things that don't scale" approach to databases. But, honestly, most startups would be better off following that approach than what Fast did. Sounds like the engineers were just entertaining themselves with shiny toys rather than solving the problems they actually had.

Pretty sure the reason they caught that flak is because they chose to reinvent the wheel instead of do a normal thing that would work until later - they're on their what - third rewrite of that now? Tootally worth the time /s

[deleted]

Re: What software engineers can learn from the rapid collapse of Fast

#124

Post by amluto »

StillBored wrote: post #119
Spivak wrote: post #35

Earlier quoted context omitted.

And if you think this is undersized for comedic effect I did ops for a company that went from <1mil to 15mil revenue (with the same sales strategy of targeting lots of small clients) on 4 1u white label supermicros. Two app servers and two MySQL dbs. Never in my life had such reliable infrastructure.

Early in my career when in a discussion with my manager at the time, he threw out the "planes with two engines have twice as many failures as single engine planes" line. So in my now close to 30 years of experience later, working on systems expected to run 24/7/365, I can't tell you what percentage of failures and outages were caused by the redundancy/fail-over/etc software and hardware layered into a system, but its…

I once managed a thoroughly non-critical server that lived in a closet. In the couple years I managed it, we had quite a few failures due to the UPS tripping and no unscheduled downtime at all from any other reason.

I’ve had massive problems with fancy (HPE, IIRC, but it wasn’t called HPE then) raid arrays caused by the controller being a POS. Using plain mdraid would have been far more reliable.

Sometimes I wonder if all the effort people put into protocols that run Paxos and Raft could be better spent running a small number of coordinator servers with manual failover. I’m suspicious that, in many use cases, leader elections cause failures more often than the leaders themselves fail.

Re: What software engineers can learn from the rapid collapse of Fast

#125

Post by mywittyname »

xen0 wrote: post #89
lolsoftware wrote: post #33

Earlier quoted context omitted.

The TailScale folks caught a bunch of flak for their "do things that don't scale" approach to databases. But, honestly, most startups would be better off following that approach than what Fast did. Sounds like the engineers were just entertaining themselves with shiny toys rather than solving the problems they actually had.

I wonder, how many engineers had full time jobs just developing and maintaining the various integrations with all their small clients?

I have to imagine they were all stepping on each other and/or reinventing slightly different shaped wheels too.

My experience has been that companies who will customize integrations for you have poorly designed products. By that, I mean, when we ask for a feature, and they bring on an engineer who does some requirements gathering then comes back with some invisible solution. That's a major redflag.

The success of an integration, in my experience, is directly proportional to the amount of the work I'm able to handle as a client.

Re: What software engineers can learn from the rapid collapse of Fast

#127

Post by truffdog »

vosper wrote: post #71
EinsDueTresFour wrote: post #45

Earlier quoted context omitted.

I would add that the "Sales vs engineering and sales winning" would be a very relevant warning also: > [sales] signed up a large number of smaller businesses on the platform. [...] However, integrating these smaller businesses was challenging thanks to several customizations needed for each new customer. The ratio of revenue per each small customer vs the total cost of integration (and very likely on-going maintenanc…

> The ratio of revenue per each small customer vs the total cost of integration (and very likely on-going maintenance) was probably at least an amber flag somewhere for those who had visibility of it. At the last company I worked at (won't call them a startup, just a 10-year old small business that somehow kept raising funding but has never made a profit) I was astonished to learn from a new CPO that in our tenth yea…

> We spent a relatively large portion of our revenue running very expensive servers to respond instantly to searches that no-one ever performed.

The last 7 years of my life as an SRE. It's maddening.

Re: What software engineers can learn from the rapid collapse of Fast

#130

Post by [deleted] »

lolsoftware wrote: post #88
hobs wrote: post #69

Earlier quoted context omitted.

Pretty sure the reason they caught that flak is because they chose to reinvent the wheel instead of do a normal thing that would work until later - they're on their what - third rewrite of that now? Tootally worth the time /s

I don't think that's a fair characterization of what happened. First, they scaled a simple solution for a long time and migrated away from it before it exploded spectacularly. I don't recall them explicitly saying how much time they spent on it, so how is it possibly to say whether it was worth the time or not? I also don't remember if it was here or on Twitter, but one of the engineers listed the many features they…

[deleted]
Post reply on HN