Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What software engineers can learn from the rapid collapse of Fast

newsletter.pragmaticengineer.com

Post reply on HN
Print view View original

191–200 of 482 posts

Re: What software engineers can learn from the rapid collapse of Fast

#191

Post by mwcampbell »

tomp wrote: post #186
SkyPuncher wrote: post #143

Earlier quoted context omitted.

I've done startup engineering for a while. I've seen exactly 1 issue that we couldn't solve easily (N+1) or with simply turning our server count up (often way cheaper than putting an engineer to a problem). Basically, to move development velocity faster we "load everything". Seems like a really bad idea on the surface. In practice, we only ran into an issue with a single customer that was literally 1000x as large as…

Even before using more servers, one should consider using (one) bigger server (and another as backup). Massive servers are laughably cheap these days. E.g. 16 cores, 128GB RAM is €112 from Hetzner.

Damn, wish those dedicated servers were available in Hetzner's US data center. As far as I'm aware, the closest thing to a competitor in the US is OVH, and the server you get for that price is about a quarter of the capacity. That's plenty for my little company, but still...

Re: What software engineers can learn from the rapid collapse of Fast

#192

Post by hn_user82179 »

mwcampbell wrote: post #179
ChrisMarshallNY wrote: post #172

Earlier quoted context omitted.

> My company's strategy is get one extremely solid person across all our verticals (Android / iOS / Backend / Infra) to minimise communication overhead and maximise iteration speed (since we throw out half the code we write anyway). but ... but ... that makes sense ... I have found that having fewer good engineers is much better than lots of bad ones. That approach is treated as heresy, in today's tech industry.

Perhaps the thinking is that it's easier to hire multiple good-enough programmers to get dependable productivity, and to replace one if something goes wrong. In the extreme case, if a company has one programmer who is very good when they're working at peak productivity, but then they have a period of low productivity for whatever reason, that company is in trouble. I've been that person.

Additionally, it's a bigger loss felt when that person leaves.

Re: What software engineers can learn from the rapid collapse of Fast

#193

Post by [deleted] »

mwcampbell wrote: post #20

I think the most applicable warning for engineers when it comes to the actual work, as opposed to whether one should join a particular startup, is this: > Engineers calculated the load Fast had in needing to serve their traffic. The Fast button was rendered less than 500,000 times per day - rarely needed to ever serve more than a few requests per second. > One of the few warning signs engineers noticed is how Fast sp…

[deleted]

Re: What software engineers can learn from the rapid collapse of Fast

#194

Post by robocat »

StillBored wrote: post #119
Spivak wrote: post #35

Earlier quoted context omitted.

And if you think this is undersized for comedic effect I did ops for a company that went from <1mil to 15mil revenue (with the same sales strategy of targeting lots of small clients) on 4 1u white label supermicros. Two app servers and two MySQL dbs. Never in my life had such reliable infrastructure.

Early in my career when in a discussion with my manager at the time, he threw out the "planes with two engines have twice as many failures as single engine planes" line. So in my now close to 30 years of experience later, working on systems expected to run 24/7/365, I can't tell you what percentage of failures and outages were caused by the redundancy/fail-over/etc software and hardware layered into a system, but its…

> "planes with two engines have twice as many failures as single engine planes"

There has to be a smart reply to that, like: 100% of pilots of single engine planes don’t make it home after a single engine failure. 100% of pilots of dual engine planes do make it home after a single engine failure.

Re: What software engineers can learn from the rapid collapse of Fast

#195

Post by woah »

lmeyerov wrote: post #31

Oof, just one of our customers pays about the same as their full revenue, and we're a cockroach team. The real thing here is they're not that surprising. A lot of companies with their kind of funding use enterprise sales to force say $5-10M ARR, but there's only so much VC money can force for a leaky funnel, broken product, and overall incorrect market + fit. I didn't appreciate this until maybe a year or two ago. Va…

> Our new hires are getting paid from customer revenue

Money is fungible

Re: What software engineers can learn from the rapid collapse of Fast

#196

Post by emgeee »

I was an engineer at Fast for over a year and wanted to clarify a few points-- - engineers did have access to data and could write their own queries to check revenue (but few people did until the article came out). - the strategy leadership decided on was to go after massive enterprise sellers which would, in theory, result in step-change functions in revenue. Many people knew about the low numbers but were willing to let the strategy play out for a while. There was a lot of excitement internally around closing the first $1B+ seller - Internally, the culture was something akin to "toxic positivity" which meant that we weren't willing to discuss failures or misses in a productive way - The company spent a lot of money on marketing events including sponsorship deals with the Tampa Bay Lightning and the rumored million dollar concert by the Chain Smokers. People are talking a lot about high engineer salaries but IMO those were far from the biggest problem (possible biased view)

The story is a lot more complicated and nuanced than the headlines you read in publications but I will say that Fast was full of really talented folks who I'd be happy to work with again. For me, a big lesson I took away from this experience is the perils of an overly positive fully remote culture. It's very easy for leadership to hide things from people when information doesn't easily spread across different organizations

Re: What software engineers can learn from the rapid collapse of Fast

#197

Post by lmeyerov »

naiwenwt wrote: post #145
meetups323 wrote: post #78

Earlier quoted context omitted.

May help explain some Meta offers I've seen... 2 years out of college? Sure, come as a Senior! That's not enough? Principal it is!

The problem is sometimes this is done for legitimate reasons and genuinely does lead to stronger outcomes for both employers and employees. It's hard to tell the difference between "stop the bleeding" title/pay inflation and "aggressively competing for talent" inflation.

Sort of. A good question is what is the revenue per employee

Meta is ~$2M/employee. Probably more like $1M once contractors are accounted for? But for a good person, even if the rest of the company has random issues, they can carve out a win.

Fast was like $1K/employee

Consider what that's like for some hypothetical B2B startup BlitzScaler Inc. They're betting on 3x+ growth year-over-year for next 2+ years, 2-3 year sales payback periods, etc, and is probably closer to Fast levels of revenue per employee than Meta's. But hitting $500K is hard, $2M is hard again in different ways, then again to $10M, and again to $20M+ (and increasing range depending if many sales people inflating it). That matters because many aren't there, even unicorns (!). That means, like Uber, they're burning increasing piles of money, except unlike Uber, the revenue is unlikely to be keeping pace. Fundraising buys team, and an artificial sense of fit+growth (kool-aid-drinking team + forced sales that tap out + churn.)

A cockroach team is closer to Amazon style: reinvest ~100% in growth, but don't gamble your employee's stock options.

Re: What software engineers can learn from the rapid collapse of Fast

#198

Post by ChrisMarshallNY »

hn_user82179 wrote: post #192
mwcampbell wrote: post #179

Earlier quoted context omitted.

Perhaps the thinking is that it's easier to hire multiple good-enough programmers to get dependable productivity, and to replace one if something goes wrong. In the extreme case, if a company has one programmer who is very good when they're working at peak productivity, but then they have a period of low productivity for whatever reason, that company is in trouble. I've been that person.

Additionally, it's a bigger loss felt when that person leaves.

...which is why it's so important to foster an environment, where they don't want to leave.

I kept engineers (really good ones) for decades. They had many "life problems" (like divorce, cancer, etc.) during that time, and I kept them on.

It's entirely possible. I have done it. I ran a team that was all "top-shelfers."

Re: What software engineers can learn from the rapid collapse of Fast

#199

Post by munchler »

bps4484 wrote: post #184
encoderer wrote: post #17

Earlier quoted context omitted.

Lol. L6. Look no further folks. This guy Dominic was clearly LARPing a startup. Leveling frameworks before traction is a joke to me.

I completely disagree. They were 450 people with 150 engineers. If I had to guess I would bet the engineers clamored for leveling and it came from bottoms up requests and a need to be fair with compensation when hiring. Now to be clear they shouldn't have been that big (clearly), but leveling people was not the problem.

That simply begs the question of why in god's name a startup with no traction needs 150 engineers.

Re: What software engineers can learn from the rapid collapse of Fast

#200

Post by Jasper_ »

robocat wrote: post #194
StillBored wrote: post #119

Earlier quoted context omitted.

Early in my career when in a discussion with my manager at the time, he threw out the "planes with two engines have twice as many failures as single engine planes" line. So in my now close to 30 years of experience later, working on systems expected to run 24/7/365, I can't tell you what percentage of failures and outages were caused by the redundancy/fail-over/etc software and hardware layered into a system, but its…

> "planes with two engines have twice as many failures as single engine planes" There has to be a smart reply to that, like: 100% of pilots of single engine planes don’t make it home after a single engine failure. 100% of pilots of dual engine planes do make it home after a single engine failure.

Planes are natural gliders and you can recover from engine failure. It's not like they just fall out of the sky.
Post reply on HN