Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You don’t need to be “enterprise-ready” or “scalable”

gorelay.co

Post reply on HN
Print view View original

101–110 of 185 posts

Re: You don’t need to be “enterprise-ready” or “scalable”

#101

Post by dominotw »

dub wrote: post #93

Horizontally scalable data storage is generally available these days (cockroach, tidb, vitess, etc.) Rearchitecting from unscalable to scalable data storage is notoriously difficult and expensive. Even the most famously competent companies and teams have struggled with that transition and invested millions and millions of dollars on it. Building on unscalable data storage when scalable data storage is readily availab…

I am suprised how niche these newsql databases are. Is it because they are expensive? why wouldn't everyone use those.

Re: You don’t need to be “enterprise-ready” or “scalable”

#102

Post by FpUser »

winrid wrote: post #92
FpUser wrote: post #57

I've delivered custom backend business servers to a various businesses of healthy size. Almost all were deployed on rented multicore servers with gobbles of RAM from Hetzner / OVH / etc. Some running for many years already. Not a single one required to be "enterprise scaled". Granted all servers are C++ and talking to a local DB. No problems in processing thousands requests per second without breaking much sweat.

Sounds interesting. What's the DB? What framework/type of API are you exposing on the C++ size?

Postgresql. No framework (well there is rather low level one of my own but it is fluid). Using 4 libs for JSON, postgres, HTTP, logging. I expose basically JSON based RPC. Peers / Clients post JSON commands and receive JSON replies.

Upon startup all data (except couple of giant tables) is sucked from DB into RAM into highly efficient data structures hence most reading requests are handled in microseconds. Those 2 giant tables are loaded into the same structs but partially. This is usually enough as RAM keeps last few years worth of data and requests that need something older come just couple of times per month. Only writes touch the database and immediately reflected in RAM as well.

This is just an example. I write game like servers as well and those use different approach. Generally I treat each project / product individually.

Re: You don’t need to be “enterprise-ready” or “scalable”

#103

Post by Gigachad »

grp000 wrote: post #84
fasteddie31003 wrote: post #34

I was interviewing for a platform engineer role for a company that from the outside their application doesn't seem too complex. Through the interview process, I find out that you cannot run much of their app locally to develop against and their staging environment does not reflect their production enviroment well. I asked them how they develop their new features if they cannot run the whole app locally and if they in…

lol. It sounds like they hire smart people. But smart people =/= good engineers. Good engineers keep things as simple as they can.

Simple is easy on a small product. Your simple sql setup doesn’t work so well when you have terabytes of data and need to be running tens of millions of jobs per day.

Re: You don’t need to be “enterprise-ready” or “scalable”

#104

Post by c0mptonFP »

andrewstuart wrote: post #100

As always, you cannot make blanket statements with software such as "you don't need to worry about scale". Perhaps your application does need to worry about scale and then if you don't then you're a fool. Then you launch, succeed and collapse or get beaten by competition because the effort to scale was beyond you in the needed time. The blanket statement that is true about software is "be wary of blanket statements a…

The advice "you don't need scalable architecture" comes from the same place as "performance isn't important, code is all about readability and maintainability"

It's just insecurity and pure coping. They're not dealing with complex performance bottlenecks, so they say these efforts are wasted.

Re: You don’t need to be “enterprise-ready” or “scalable”

#105

Post by xboxnolifes »

s1k3s wrote: post #21
cubes wrote: post #11

Hard agree. History is littered with dead startups that designed for scale before they had enough usage to justify it. Within reason, having users knock your site over resulting in failures like the Twitter Fail Whale is a good problem to have. With that said, you need to be prepared to scale up quickly once you have this problem. There's a reason Facebook counts its users in the billions, and Friendster is a footnot…

Yes, but.. I am a software architect. ~90% of the job offers I get are from CEOs/CTOs who want me to join their small company to help them refactor because they can’t grow anymore. Tech debt kills as well.

If this pattern is successful, it's not killing them.

Re: You don’t need to be “enterprise-ready” or “scalable”

#106

Post by nkuttler »

jsiaajdsdaa wrote: post #24
wereHamster wrote: post #19

I'm working on a project where the backend devs decided to use Cassandra to manage a few hundred entities. Due to different access modes they had to denormalize the data into multiple tables and we're constantly running into consistency issues. A fucking sqlite db with a few tables and indexes would do just fine. Also, the tool has maybe a dozen DAUs. But it's web scale, so, yeah…

Why would a team decide to use nosql to manage "a few hundred" entities? Nosql databases excel at simple lookups and gets by ID, rather than trying to sort and join pretty much ad hoc.

Resume driven design

Re: You don’t need to be “enterprise-ready” or “scalable”

#107

Post by jamesfinlayson »

john_the_writer wrote: post #41
fasteddie31003 wrote: post #34

I was interviewing for a platform engineer role for a company that from the outside their application doesn't seem too complex. Through the interview process, I find out that you cannot run much of their app locally to develop against and their staging environment does not reflect their production enviroment well. I asked them how they develop their new features if they cannot run the whole app locally and if they in…

I'm in that boat. We have a "microservice" code base, and spinning up the whole "app" is a Herculean task. Even if you do manage to get it all running, getting the multitude of database with useable data adds an odyssey to your day. We have some parts automated, but ops cannot keep up with all the changes in the micro apps. Back when I worked on a mono-rails app with a React front end, getting things up and running w…

Yep I've had the same thing happen - a company I worked for went from having a monolith that could run locally, then everyone drunk the microservice Kool-Aid and we ended up with no way to run the microservices locally in a way that integrated with the monolith. There were staging sites with everything but there was one staging site per team.

Re: You don’t need to be “enterprise-ready” or “scalable”

#108

Post by jamesfinlayson »

scarface74 wrote: post #38
fasteddie31003 wrote: post #34

I was interviewing for a platform engineer role for a company that from the outside their application doesn't seem too complex. Through the interview process, I find out that you cannot run much of their app locally to develop against and their staging environment does not reflect their production enviroment well. I asked them how they develop their new features if they cannot run the whole app locally and if they in…

Why is it a big deal to run your app locally? You just give each developer access to a cloud account (either one account or one per developer) with wide enough guard rails. Heck, when I am in a place with bad internet, I spin up a Cloud 9 (Linux) or Amazon Workspace account (Windows) and do everything remotely. I’m not wasting my time trying to use LocalStack or SAM local.

It would be quicker for me to run things locally - in my team we all have access to an AWS account each but to test my code remotely I need to compile code, build my container, push my container to ECR then deploy the container to ECS - that last step takes minutes.

Re: You don’t need to be “enterprise-ready” or “scalable”

#109

Post by jamesfinlayson »

fasteddie31003 wrote: post #34

I was interviewing for a platform engineer role for a company that from the outside their application doesn't seem too complex. Through the interview process, I find out that you cannot run much of their app locally to develop against and their staging environment does not reflect their production enviroment well. I asked them how they develop their new features if they cannot run the whole app locally and if they in…

I've been at a place where the whole stack couldn't be run locally at all but everything is separated by well-defined API boundaries and there are mature mock servers where required.

The architecture was probably more complicated than it needed to be but it was also decades old.

Re: You don’t need to be “enterprise-ready” or “scalable”

#110

Post by noodle »

s1k3s wrote: post #21
cubes wrote: post #11

Hard agree. History is littered with dead startups that designed for scale before they had enough usage to justify it. Within reason, having users knock your site over resulting in failures like the Twitter Fail Whale is a good problem to have. With that said, you need to be prepared to scale up quickly once you have this problem. There's a reason Facebook counts its users in the billions, and Friendster is a footnot…

Yes, but.. I am a software architect. ~90% of the job offers I get are from CEOs/CTOs who want me to join their small company to help them refactor because they can’t grow anymore. Tech debt kills as well.

If they have the money to hire you and they have pent-up growth that refactoring will unlock, that really sounds like close to an ideal path for a startup to take. They're literally paying down their tech debt by paying to hire you. Maybe ideal would be a little earlier so it doesn't block growth, but yeah. In my experience way more startups have died from premature optimization
Post reply on HN