Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building for the 99% Developers

future.a16z.com

Post reply on HN
Print view View original

51–60 of 310 posts

Re: Building for the 99% Developers

#51

Post by deathanatos »

alberth wrote: post #10

We way over complicate things. SQLite can be used for 95% of real world use cases. And whenever WAL2 + BEGIN CONCURRENT get merged into main, it will be able to handle 99% use cases. And don’t fool yourself into believing you’re not in that 1%.

You're not the only commenter saying this, so I gotta ask: whatever VM that SQLite DB is on, is your business cool with a business disruption, dataloss, or both when that VM goes down, when that AZ goes down, or that disk fails? Or am I missing something? If you have some sort of fail-over-to-last-backup plan, is that not just a distributed database with more steps, and why not something like RDS, or CockroachDB?

(From mine, it is an explicit requirement that we weather such events. I keep trying to keep global outages from our platforms off the requirements list…; so, IDK, perhaps your requirements allow different a different approach.)

Re: Building for the 99% Developers

#52

Post by latch »

adra wrote: post #46
urthor wrote: post #38

Earlier quoted context omitted.

It's also that, in the past, the capacity of our monoliths to scale was far lower. There's many apps that need say, a 100GB database in 2022. Those apps also needed 100GB database in 2007, when horizontal scaling was the hottest thing around. Nowadays however, Moore's law has steadily overtaken an order of magnitude of use cases from 2007. Maybe 1/10th of the 2007 use cases still need that kind of enormous, big tech…

Scale vertically and you're still a single point of failure away from a big, potentially fatal outage. The coding and architecture can often be trivial though. Definitely good for internal systems and more green programmers. Scale horizontally and you're often constraining and or complicating your design/architecture, but you can handle outages far better, if not seamlessly. It's much harder to get right, so it's usu…

It's wrong to mix horizontal scaling with availability. They are two different things that solve two different problems and solving one doesn't necessarily mean you've solved the other. You can scale horizontally in a way that decreases your total availability, e.g. by sharding.

Re: Building for the 99% Developers

#53

Post by deathanatos »

bjterry wrote: post #19

99% feels like an exaggeration. I've talked to many developers from non-FAANG companies, and it isn't at all uncommon for them to be using GraphQL or serverless. I guess there is some selection bias since they are usually applying to a unicorn, so they are probably more likely to come from environments that fit the "1%". Whether this distinction is relevant to you depends on where you sit. If you are a startup sellin…

… I'm in a small company, and we "use" serverless. I've never once asked myself "Should I move to serverless?" It's just whether, for some application, it's the right tool.

We run a few Github bots & a function that updates a Route 53 record on serverless. (Security didn't want to give permission to R53; "too much, too broad"; a lambda that exposed only the necessary action to the service that required it was the compromise). But it's all extremely low-frequency stuff with no or little state, where the costs of a VM would far exceed the costs of "serverless". It was the right tool, for those jobs. (& it's usually niche stuff… I'm trying to think if I've ever worked somewhere with something core on Lambda or the like…)

But also got tons of VMs. Lots of VMs. Probably too many VMs.

Re: Building for the 99% Developers

#54

Post by alberth »

deathanatos wrote: post #51
alberth wrote: post #10

We way over complicate things. SQLite can be used for 95% of real world use cases. And whenever WAL2 + BEGIN CONCURRENT get merged into main, it will be able to handle 99% use cases. And don’t fool yourself into believing you’re not in that 1%.

You're not the only commenter saying this, so I gotta ask: whatever VM that SQLite DB is on, is your business cool with a business disruption, dataloss, or both when that VM goes down, when that AZ goes down, or that disk fails? Or am I missing something? If you have some sort of fail-over-to-last-backup plan, is that not just a distributed database with more steps, and why not something like RDS, or CockroachDB? (Fr…

Mentioning RDS is an apples to oranges comparison. RDS is a fully managed service by a 3rd party.

Nevertheless, checkout https://litestream.io - given that SQLite is just a file, it’s incredibly easy to replicate. Failover is coming in the next release, which is already beta.

Also checkout this Expensify blog post on how they use SQLite. It’s scales amazingly well. https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q...

EDIT: I don’t understand, why the downvote?

Re: Building for the 99% Developers

#55

Post by libraryofbabel »

deathanatos wrote: post #51
alberth wrote: post #10

We way over complicate things. SQLite can be used for 95% of real world use cases. And whenever WAL2 + BEGIN CONCURRENT get merged into main, it will be able to handle 99% use cases. And don’t fool yourself into believing you’re not in that 1%.

You're not the only commenter saying this, so I gotta ask: whatever VM that SQLite DB is on, is your business cool with a business disruption, dataloss, or both when that VM goes down, when that AZ goes down, or that disk fails? Or am I missing something? If you have some sort of fail-over-to-last-backup plan, is that not just a distributed database with more steps, and why not something like RDS, or CockroachDB? (Fr…

I’d wager that many of the people pushing SQLite for 95% of use cases have not been involved in running a database for a large always-on business. Just because your SQLite database can handle the traffic doesn’t mean you’ve solved robust database infrastructure.

Now if they said postgres on RDS can be used for 95% of real world use cases, that I could get behind.

Re: Building for the 99% Developers

#56

Post by weq »

newrotik wrote: post #42
bjterry wrote: post #19

99% feels like an exaggeration. I've talked to many developers from non-FAANG companies, and it isn't at all uncommon for them to be using GraphQL or serverless. I guess there is some selection bias since they are usually applying to a unicorn, so they are probably more likely to come from environments that fit the "1%". Whether this distinction is relevant to you depends on where you sit. If you are a startup sellin…

The point the article raises is whether those "non-FAANG engineers" should be using serverless at all. Will their requirements ever scale to a point where it actually makes sense for them to be deployed as serverless services?

So by No-FAANG engineer, you mean someone who isnt a robot ad hawker sell out?

Serverless allows anyone to be scale for minimal up-front cost and only pay for the usage they really need.

So glad that FAANG attract all this rare talent so us no-FAANG engineers can make a buck.

Re: Building for the 99% Developers

#57

Post by snovv_crash »

brandmeyer wrote: post #44
alberth wrote: post #10

We way over complicate things. SQLite can be used for 95% of real world use cases. And whenever WAL2 + BEGIN CONCURRENT get merged into main, it will be able to handle 99% use cases. And don’t fool yourself into believing you’re not in that 1%.

> WAL2 + BEGIN CONCURRENT Woah. There are some gotchas. It still won't support as much concurrency as row-level MVCC. But it'll still be a big improvement. https://www.sqlite.org/cgi/src/doc/wal2/doc/wal2.md https://www.sqlite.org/src/doc/begin-concurrent/doc/begin_co...

The amount of IO we gained in the switch from spinning rust to NVME SSDs means that a SQLite scales serialised 'concurrency' much further than eg. Postgres ever could before.

Re: Building for the 99% Developers

#58

Post by pm90 »

bjterry wrote: post #19

99% feels like an exaggeration. I've talked to many developers from non-FAANG companies, and it isn't at all uncommon for them to be using GraphQL or serverless. I guess there is some selection bias since they are usually applying to a unicorn, so they are probably more likely to come from environments that fit the "1%". Whether this distinction is relevant to you depends on where you sit. If you are a startup sellin…

What you seem to be implying is that there isn’t a profitable market for dev tools for the “rest” of developers (whatever the % might be). 2 observations:

* it’s possible the market isn’t lucrative enough for a VC funded SV company; perhaps other models in other locations might make it more cost effective to serve that market

* putting the profit motive aside: I think there is a lot to learn from understanding how the 99% of developers work and how they may be enabled by tooling to make them more productive. The solutions to the problems faced by that group may actually help build better technologies.

Re: Building for the 99% Developers

#59

Post by wh-uws »

let me calm down and breathe and I'll take the bait ...

look just because you're not Lebron James or Michael Jordan doesn't mean you can't go to your local basketball court and enjoy a game of pick up.

But the same way you don't conflate what you need to do to be one the of the best people to ever play a game to play at all...

your software engineering team doesn't need to feel completely useless because they aren't immediately using .

Everything in software engineering from side project to unicorn startup service is about tradeoffs.

Tradeoffs happen because of constraints.

I don't expect a consulting shop to build code the same way an open source project does or a publicly traded company that has been profitable for 10 or 20 years.

I'm hoping you can read in this the need to be cognizant about manpower tradeoffs.

So yes I fully appreciate this as someone who has consulted, worked a public companies, and is now at a rapidly growing startup...

Just because your team isn't meeting some imaginary ideal for some other form of team with different constraints doesn't mean you can't ever strive for any ideal or give up...

you just have to be realistic about trade offs...

my favorite example of this is thinking about a system like rails vs early react (loooong next.js or even before the context system) or a lot of things built in the python web ecosystem.

With rails you have scaffolds and endless convenience methods that let you be productive quickly even as a small team or shop...

with early react or python web you were largely left to your own devices out side of the core flows they had solved (painting updates to the dom with react or some rudimentary crud stuff with django or flask) ...

Because they were built by and more importantly FOR different types of teams.

Rails by the relatively slender (and famously against VC type scaling) 37 signals consulting crew and react by facebooks behemoth engineering team.

Of course they are going to build for their own trade offs...

why would you not?

(I intially opened this rant with this final piece thata I thought better of starting with but I still stand on the feeling...

The is the biggest piece of curmudgeonly nonsense I've ever read... its like someone training an ml model on last 5 years of HN comments and churning out a click bait blog post)

Re: Building for the 99% Developers

#60

Post by scarface74 »

urthor wrote: post #37
LAC-Tech wrote: post #16

IME most small software companies could be run off SQLite, a $5 linux VPS, vanilla JS, and a single line deploy script. (Pieter Levels seems like the master at this) I'd never bother suggesting it though: - no one likes being told they're small when they think they're big - it's bad for devs careers (Resume-Driven Development) - as TFA points out - it's not what FAANG are doing so I'd lack legitimacy

Strongly disagree about it being (successful) resume driven development mostly. Most developer's resumes would be vastly enhanced by delivering a lean, technically successful product to end users, or 5. A lot of developers think playing with cool technology makes them better developers for some reason. Which isn't true. What makes them better developers is knowing what that technology does , and choosing the appropri…

You seem to be misunderstanding people’s motivations. Most developers aren’t trying to be “better” we are trying to exchange labor for money like everyone else.

“Grinding LeetCode” (tm r/cscareerquestions) doesn’t make you a better developer. But it does get you in the top paying companies. Well I didn’t. But that’s a different story.

On the other side of the fence in corp dev where pattern matching is the norm, employers aren’t going to want to hear that you used Sqllite.

Post reply on HN