Live data from Hacker News

Some more things about Django I've been enjoying

jvns.ca

11–20 of 128 posts

Re: Some more things about Django I've been enjoying

#11
post #7

Earlier quoted context omitted.

A $10/month VM might only have a single thread anyway. Some providers like lightsail are really slow too.

You can run multiple OS threads (gunicorn workers) on one VM thread so the workers don’t have to wait for each others request to finish though, right? And if you pay $10/month for a single threaded machine, you’re overpaying by a lot.

True, but in this case with SQLite, there's unlikely to be much of a difference because there isn't the spare time available when waiting for a separate database server. I don't know what providers are good for a $10/month instance these days.

Re: Some more things about Django I've been enjoying

#12
post #6

The Django filter syntax with the double underscores is like fingernails on a chalkboard to me. I find it insane that they didn't just use operator overloading to create a real query expression language.

Or now that python has ~types, this is really an area where things could be improved. Filtering would just be lambda predicate with fields auto complete as seen in .NET, scala, etc

Re: Some more things about Django I've been enjoying

#13

I mostly use Go + SQLite for all the things I used to use Rails, JavaScript, or Python for. I find python django wastes too much resources, just look at memory usage. One of my web app backend (go) is serving approx 100 req/s right now and i look at pprof i see it's not bottlenecked by CPU but mostly IO and i love this. Writing concurrent code in Go is easy, the code i wrote 10yrs ago still compiles with no issue! Th…

Appart from the fact that you find that Python wastes too much memory, what is your point ? I think that any person choosing Django knows that Python by nature will not be the most efficient language. Apart from that Django is battle-tested and can help bring a stable "product" quite quickly.

Re: Some more things about Django I've been enjoying

#14
post #5

> Some light load testing (with (ab -n 1000 -c 1) shows that right now we can serve about 2-3 requests per second (on a ~$10/month VM). > After turning on template caching, it seems like the site can now pretty easily handle 12 requests per second or so without using all of the CPU. I have not carefully benchmarked the before and after but it seems like it’s made a pretty big difference. That seems crazy low, I think…

Those numbers sound... Single-threaded. Like they're using the development runserver instead of uwsgi or gunicorn.

Also, they may be serving static files through Django. Otherwise, it's really, really low.

Re: Some more things about Django I've been enjoying

#15
post #7
post #5

Earlier quoted context omitted.

Those numbers sound... Single-threaded. Like they're using the development runserver instead of uwsgi or gunicorn.

A $10/month VM might only have a single thread anyway. Some providers like lightsail are really slow too.

For a cheap VM, I'd still be expecting in the range of 500-1000 connections a second. Green threads are cheap, even with a single processor.

For a half-decent VM, I'd be expecting multi-thousand.

Single figures a second, is choked to a single connection at a time.

Re: Some more things about Django I've been enjoying

#16
post #13

I mostly use Go + SQLite for all the things I used to use Rails, JavaScript, or Python for. I find python django wastes too much resources, just look at memory usage. One of my web app backend (go) is serving approx 100 req/s right now and i look at pprof i see it's not bottlenecked by CPU but mostly IO and i love this. Writing concurrent code in Go is easy, the code i wrote 10yrs ago still compiles with no issue! Th…

Appart from the fact that you find that Python wastes too much memory, what is your point ? I think that any person choosing Django knows that Python by nature will not be the most efficient language. Apart from that Django is battle-tested and can help bring a stable "product" quite quickly.

if "2-3 requests per second" per author is what you wanna do on $10 server go ahead". My server does 100 RPS on $16 instance

neither you are saving any time, nor money.

>part from that Django is battle-tested and can help bring a stable "product" quite quickly.

this is a myth, you'll not save anytime. Only way you can save time is if you've experience in this but same is true if you write your app from scratch in Go from your learned patterns.

Re: Some more things about Django I've been enjoying

#17
> as a meta comment: I’ve been working on talking about my programming opinions by just saying “THING does not feel good to me, I prefer OTHER THING instead”. That post I linked to says that function-based views are the “right way”. I’m not very invested in whether it’s “right”, but it’s validating to know that other people feel similarly to me about inheritance

I admire this about Julia a lot. Her texts and zines are exploring software in a way that encourages curiosity rather than promoting a singular point of view.

Re: Some more things about Django I've been enjoying

#18
There are so many footguns[1] in async python but it really has stolen the zeitgeist of modern python web. Synchronous django has a lot to commend it. If you deploy it reasonably well (workers, behind a caching reverse proxy etc) it's easy to operate in production even under duress.

It's not going to be the right stack for long lived websocket connections or whatever but for a CRUD-ish or enterprise app, often very productive.

[1] buffer bloat is too easy to sleep walk into

    queue = asyncio.Queue()  # oops
unbounded concurrency

    await asyncio.gather(*(fetch(item) for item in items))  # look mum! no outbound sockets left or maybe even no file descriptors
accidental blocking

    data = requests.get(url).json()  # oops! we're blocking the event loop
and sure you can setup a separate task to measure the event loop latency and alert on that but this is the kinda thing you'll never find in a tutorial, you just need to experience this stuff and figure out a solution you like

I can go on and on about async python (cancellation bugs, leaking a task - that has a cataclysmic failure mode where if you forget to hold a reference to your asyncio.create_task() then it's all weak refs in that machinery so your task can get garbage collected before it ran or completed in production! super tricky forensics. Then there's all the obvious stuff like race conditions, new ways of creating deadlocks, blah blah i really can go on for days.

Re: Some more things about Django I've been enjoying

#19

Earlier quoted context omitted.

You can run multiple OS threads (gunicorn workers) on one VM thread so the workers don’t have to wait for each others request to finish though, right? And if you pay $10/month for a single threaded machine, you’re overpaying by a lot.

True, but in this case with SQLite, there's unlikely to be much of a difference because there isn't the spare time available when waiting for a separate database server. I don't know what providers are good for a $10/month instance these days.

Most of the time you're hitting SQLite, you're just reading from it, and so it doesn't hold anything up.

Re: Some more things about Django I've been enjoying

#20

Earlier quoted context omitted.

You can run multiple OS threads (gunicorn workers) on one VM thread so the workers don’t have to wait for each others request to finish though, right? And if you pay $10/month for a single threaded machine, you’re overpaying by a lot.

True, but in this case with SQLite, there's unlikely to be much of a difference because there isn't the spare time available when waiting for a separate database server. I don't know what providers are good for a $10/month instance these days.

How are you spending any non-negligible time reading from SQLite at 12 requests per second though? That would mean you’re spending something like 50 ms per request on reading from SQLite.
Post reply on HN