Does it scale? Who cares (2011) • Hacker News Board

Re: Does it scale? Who cares (2011)

#241

Post by manquer » Sun, Apr 09, 2017, 4:56 PM UTC

luord wrote: post #218

user5994461 wrote: post #200

Earlier quoted context omitted.

> You can probably push your solution by an order of one magnitude if you need it quickly. You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. It will almost always bankrupt you in the medium term. The only place I've seen it sustainable is a place that was generating a $100 per user, and there weren't many active users either (thousands, not millions).

> You can always push a python/ruby app an order of magnitude by putting an order of magnitude more AWS instances. Or a Java app or a Go app. Really, if one's working in a domain where the language would become the bottleneck, one deliberately screwed up by going against the grain because that language is little used in that domain. For almost everything else or with very specific exceptions, something else is the bo…

Langauge is rarely the first bottle neck . This also doesn't apply in the same away for data stores and when your app is stateful. Scaling rdbms is not the simple. Query fine tunning and performance optimisation for single slow queries cannot be solved by just more resources. Just from a database pov Handling replication , multi master caching , backup and fail over especially can fuck you over . In b2b a single data loss event can kill your business, same with security .

Re: Does it scale? Who cares (2011)

#242

Post by nickpsecurity » Sun, Apr 09, 2017, 4:57 PM UTC

emcq wrote: post #222

spamizbad wrote: post #216

Earlier quoted context omitted.

> In my experience, dynamically typed languages don't do well for developer productivity. At least one empirical study suggests static typing does not improve programmer productivity. Source: http://courses.cs.washington.edu/courses/cse590n/10au/hanenb... This is just one of the many common industry "wisdoms" that collapse under scientific scrutiny.

That's a cool experiment, but from a quick read they are creating a compiler which is a bit different than web development. Looking at the related work all seem to contradict this finding and suggest preference to statically typed languages, such as one that looked at API development. From my own experience with dynamic types it's not the dev time that increases but maintenance time. When you have a team working on a…

"That's a cool experiment, but from a quick read they are creating a compiler which is a bit different than web development."

They're doing a lot of things wrong. See my reply.

Re: Does it scale? Who cares (2011)

#243

Post by nickpsecurity » Sun, Apr 09, 2017, 4:59 PM UTC

user5994461 wrote: post #212

franciscop wrote: post #205

Earlier quoted context omitted.

But that is basically the point of the whole article and this thread, that it's great for proof of concept that morphs into production when needed. If you can deploy in one day that means you can test 5 ideas/week until one catches on (hopefully not that long). Once one catches on, you can actually put more effort into it and optimize it as needed and not the other way around.

You can't try 5 products or features a week on real users. They don't keep up.

Facebook deploys thousands a week:

https://swreflections.blogspot.com/2013/09/this-is-how-faceb...

Re: Does it scale? Who cares (2011)

#244

Post by cocoflunchy » Sun, Apr 09, 2017, 5:03 PM UTC

cyberferret wrote: post #52

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

> I know of absolutely no service which failed because it couldn't scale. I would say this is because of the simple fact of visibility and adoption. You've probably never heard of these services probably because they ground to a halt with a mere 1000 users, so they never got mainstream enough to be recognised as a viable service. It is a bit like how no one remembers the dozens of people who failed to achieve sustain…

Do you know of resources to learn more about good database design? I've been looking for ways to optimize my current db usage and we're absolutely adding a new field to the Person model every time we feel we need it...

Re: Does it scale? Who cares (2011)

#245

Post by sah2ed » Sun, Apr 09, 2017, 5:04 PM UTC

hopfog wrote: post #175

stevewilhelm wrote: post #140

Earlier quoted context omitted.

I think what the author of the original article would suggest that you need to convert some of the concurrent users to paying customers and use the money to purchase additional Heroku dynos.

I don't think that's possible for me. I have some future monetization plans but that requires features I don't have yet. At the current state I have a hard time seeing anyone would pay for it. Also, due to how the back end is structured it's not possible to scale horisontally at the moment so all I could do with money is beefing up the current dyno, not adding more.

> At the current state I have a hard time seeing anyone would pay for it.

You'll never know for certain until you try. You need to ask put a dollar sign and see how current users respond. Of course only a handful of users will pay, after which you can slowly ramp up prices as you build out the features you have planned.

Re: Does it scale? Who cares (2011)

#246

Post by nickpsecurity » Sun, Apr 09, 2017, 5:05 PM UTC

frik wrote: post #166

grey-area wrote: post #118

Earlier quoted context omitted.

As a counterpoint stack overflow is hosted on windows and did just fine. If your service is printing money you can easily fix tech problems (e.g. Facebook effectively rewriting php), if it's not you may flail around trying rewrites etc but the real problem lies elsewhere. Tech people are quick to find technical reasons for failure but the reasons are usually elsewhere.

1) Friendster had scaling issues. They insisted on keeping the x-degree-of-friendship-calculation which is quite resource expensive and doesn't scale. Instead making the software scalable, they brought in expensive exec who decided to through more and more very high end servers and enterprise storage on it. It eat their startup money, and they were constantly firefighting with very high page load time like 12+ sec, i…

"MySpace took over Friendster because of this."

Friendster was mostly focused on Asia plus lost a ton of users around the same time Facebook gained a ton of users. I don't think it was scaling architecture that did them in. It was the market choosing the competition for the user experience plus what their friends were on. That's for most of the world. I have no idea what contributed to their failure in Asia since I don't study that market when it comes to social media.

EDIT to add: I recall the founder did say they had serious technology problems for a few years that affected them. I'm just thinking Facebook spreading through all the colleges & moving faster on features was their main advantage.

Re: Does it scale? Who cares (2011)

#247

Post by inetknght » Sun, Apr 09, 2017, 5:09 PM UTC

nickpsecurity wrote: post #243

user5994461 wrote: post #212

Earlier quoted context omitted.

You can't try 5 products or features a week on real users. They don't keep up.

Facebook deploys thousands a week: https://swreflections.blogspot.com/2013/09/this-is-how-faceb...

If you're looking to Facebook for your user experience guidelines, you're going to end up angering a lot of actual users.

Re: Does it scale? Who cares (2011)

#248

Post by ne01 » Sun, Apr 09, 2017, 5:15 PM UTC

developer2 wrote: post #194

technosamay20 wrote: post #59

Earlier quoted context omitted.

I mentioned something similar in a separate comment. My initial DB schema was pretty bad. We did at 2 schema rewrites and migrations from the launch to 5M users. Each time it took 2 weeks of sleep less nights. The machines today are really powerful. You can do a lot with 244 GB RAM machines backed by SSD. Someone who doesn't have the skill set to be able to scale once they get traction - it's likely they will not hav…

>> You can do a lot with 244 GB RAM machines Is this a typo of "244 GB" instead of "24 GB"? Nearly any company that has a single machine provisioned with 244 GB of RAM is doing something severely wrong, likely putting the company's ability to grow at risk. Such a machine screams of trying to vertically scale a poorly performing legacy product instead of figuring out to horizontally scale out with 16-64 GB servers. Th…

I know Basecamp uses a server with 2TB of RAM for their single MySQL server. FYI CTO of Basecamp is the creator of Ruby on Rails -- smart people.

Checkout stack overflow stack too.

Their system is faster too.

Re: Does it scale? Who cares (2011)

#249

Post by luord » Sun, Apr 09, 2017, 5:16 PM UTC

sidlls wrote: post #236

luord wrote: post #215

Earlier quoted context omitted.

Exactly, I never understand the static vs dynamic flamewars. Most of the issues usually presented stem from poor developers or poor development processes/environments/tools. Poor code written in JavaScript won't become pretty just by translating to Java, and vice versa. That I've seen anyway.

Many of the "poor development processes/environments/tools" issues are related to choosing dynamic languages in a "ship first, design later (maybe)" paradigm, though.

"Ship first, design later" is completely orthogonal to what language one's using. Using a language that forces one to write int or str before the variables isn't preventing anyone to write software without having a design.

It's not even part of the "static vs dynamic" argument; one can rush things in Java or in C++ just as easily. The only difference is that Java might allow doing it slightly faster.

Re: Does it scale? Who cares (2011)

#250

Post by xiaoma » Sun, Apr 09, 2017, 5:18 PM UTC

technosamay20 wrote: post #15

Couldn't agree with this article more. I built the biggest social network to come out of India from 2006-2009. It was like Twitter but over text messaging. At it's peak it had 50M+ users and sent 1B+ text messages in a day. When I started, the app was on a single machine. I didn't know a lot about databases and scaling. Didn't even know what database indexes are and what are their benefits. Just built the basic produ…

> "I know of absolutely no service which failed because it couldn't scale."

Zooomr did. They came out of nowhere in 2006 and were a real threat to Flickr. They had AJAX-powered editss, geotagging, various other unique features and they were starting to pull in some highly followed followed photographers.

But, the site kept crashing as traffic grew and some scaling problems even lead to data loss. I wanted to see them win but they just couldn't keep up with traffic and eventually Yahoo cloned their features and they became irrelevant.

https://techcrunch.com/tag/zooomr/