Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Old box, dumb code, few thousand connections, no big deal

rachelbythebay.com

Post reply on HN
Print view View original

91–100 of 288 posts

Re: Old box, dumb code, few thousand connections, no big deal

#91

Post by neurostimulant »

rumanator wrote: post #46
bjt wrote: post #44

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…

> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backe…

That's because I'm more productive with Python (been using it for ten years to implement many backend services), never hit any of its performance limitation yet (I mostly use it to develop b2b app with medium traffic at most, and always leverage distributed task queue for anything that might excessively blocks the webserver process), and also didn't want to maintain a fleet of windows servers (at least in the past when .net was windows only).

Re: Old box, dumb code, few thousand connections, no big deal

#92

Post by anonymoushn »

bsder wrote: post #82
anonymoushn wrote: post #64

Earlier quoted context omitted.

It seems like doubling down is the standard thing to do. Here's a video about how Instagram bugs engineers to do fewer string manipulations in Python instead of using a faster language https://youtu.be/hnpzNAPiC0E

You ... are not ... Instagram. Okay. Get over it. Python is fine. Threads are fine. Go is fine. Clojure is fine. Java is fine. PHP is ... okay I won't go that far. :) You fix the problem when the cost required to fix the problem is finally less than the opportunity cost of fixing something else.

I would prefer not to deal with "scaling up" for workloads that could run in a single process on commodity hardware.

I've mainly worked at companies that would be bankrupt if they used Python the way Instagram does.

Re: Old box, dumb code, few thousand connections, no big deal

#93

Post by nexuist »

bsder wrote: post #82
anonymoushn wrote: post #64

Earlier quoted context omitted.

It seems like doubling down is the standard thing to do. Here's a video about how Instagram bugs engineers to do fewer string manipulations in Python instead of using a faster language https://youtu.be/hnpzNAPiC0E

You ... are not ... Instagram. Okay. Get over it. Python is fine. Threads are fine. Go is fine. Clojure is fine. Java is fine. PHP is ... okay I won't go that far. :) You fix the problem when the cost required to fix the problem is finally less than the opportunity cost of fixing something else.

PHP is fine too. Check out https://laravel.com/ and tell me that doesn't look like a joy to work with.

Re: Old box, dumb code, few thousand connections, no big deal

#94

Post by neurostimulant »

heelix wrote: post #77
rumanator wrote: post #46

Earlier quoted context omitted.

> Even if they do need to serve tons of requests per second, it's totally reasonable for them to still choose Python/Gunicorn if the cost of the additional servers is less than the cost of having to support multiple languages. How hard is it to get up to speed on any other tech stack? ASP.NET Core is extremely fast and the learning curve is close to none, for example. If someone was able to wrap his head around backe…

I'll find out myself. Just got handed a codebase that is a mix of Python/Gunicorn/node when this sprint started off. I've seen the word gunicorn in the repo... but still don't know what it does yet. First time for this old back end Java/C++/XQuery programmer, so how hard can it be? (On the plus side, seems my Jetbrain kit includes PyCharm, so I've even got an IDE!)

Gunicorn is just a WSGI server, basically used to spawn a pool of webserver processes for your python backend. A python webserver is more optimal when used in multiprocess configuration (as opposed to multithreaded configuration, where python sorely suck at), and gunicorn will do that for you automatically, routing each http request to available worker process in the pool.

Re: Old box, dumb code, few thousand connections, no big deal

#95

Post by theelous3 »

bjt wrote: post #44

Whether intended or not, there's an undercurrent of "you're all so dumb for using Python" (or Ruby, or PHP, or other similarly performant language) here. I want to surface that and question it a bit. It's totally reasonable for a company to choose the Python/Gunicorn option if they already have a bunch of people who know Python and they don't need to serve tons of requests per second. Even if they do need to serve to…

I don't really get what the point of this post is. Is it really a dig at python? Python can handle thousands of connections in a single thread no problem with basic enough stuff.

Is what the author did supposed to be impressive? Is it supposed to make python look bad?

I don't get it. Seems like run of the mill stuff. Python might struggle at the same level of concurrency (was it like 15k?) but you can still do 10k connections easy enough iirc.

Re: Old box, dumb code, few thousand connections, no big deal

#96

Post by hedora »

wondringaloud wrote: post #19
thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

I'm glad someone else sees this the way I do. What the blog writer did was tinker with something. They didn't engineer it. I was a mechanical engineer prior to switching to software. As a general rule, the things we do in software are very distant from engineering.

I think the analogy to mechanical engineering is something like this:

A seasoned engineer notices that everyone is only building suspension bridges all of a sudden.

They point out that you can span a stream with some bricks or rocks and a bit of mortar, and are ridiculed.

Next, they build a highway overpass out of concrete pylons, and stress test it to 10x the necessary engineering load, and point out it cost 10% as much as a typical contemporary suspension bridge. That’s this article.

Re: Old box, dumb code, few thousand connections, no big deal

#97

Post by 6510 »

modal-soul wrote: post #24
thenewwazoo wrote: post #7

But this isn’t engineering. This is the IT equivalent of building a bridge and driving successively larger trucks over it. In real engineering fields, you can do predictive analyses based on prior empiricism. There’s none of that in our fields until you’re talking about very small systems where, for example, the stack consumption can be determined in advance and the scheduler can give you guarantees about worst-case…

Saw an interesting talk just about this topic [1] The gist of it was: Other engineering disciplines use the techniques you've mentioned because of the the costs, both time and money, associated with getting it wrong. Software engineering lends itself to different methods of development and construction, as the costs associated with getting it wrong or making changes after the fact are much lower. (For most applicatio…

*Assumed to be lower.

What is actually happening we don't know. IT is dramatically changing everything. It's not all bad but it isn't all good either. I hate to give examples since it is really vague what goes in and what comes out and people tend to mistake an example for full coverage but... for example, we have no idea what drives suicide rates.

Re: Old box, dumb code, few thousand connections, no big deal

#98

Post by phreack »

nexuist wrote: post #93
bsder wrote: post #82

Earlier quoted context omitted.

You ... are not ... Instagram. Okay. Get over it. Python is fine. Threads are fine. Go is fine. Clojure is fine. Java is fine. PHP is ... okay I won't go that far. :) You fix the problem when the cost required to fix the problem is finally less than the opportunity cost of fixing something else.

PHP is fine too. Check out https://laravel.com/ and tell me that doesn't look like a joy to work with.

How much of the old "A fractal of bad design" post would you say still applies? That one post showcased so many footguns right at the language level that it spooked me away from it forever.

Re: Old box, dumb code, few thousand connections, no big deal

#100

Post by birktj »

losvedir wrote: post #74

I know pcwalton has been banging the "just use threads" drum for a while now. Rust used to use a green thread approach, but if I recall correctly, they ripped all that out for just native threads with the idea that in most cases it's fast and efficient enough. I remember when C10K was the big challenge, but even the naive approach of spawning a thread per connection now can handle that.

I believe the reason Rust dropped green threads was to get rid of the runtime system and become a more systems language.

Modern fast network code in Rust is based on async/cooperative multitasking distributed over num cpu worker threads which works very well.

Post reply on HN