41–50 of 239 posts
I think it’s been bog standard practice to run flask via uwsgi or gunicorn with async workers and use multiple process based workers per deployed server unit (eg per pod in Kubernetes). What matters is that the cumulative latency & throughput solve your problem, not how fast you can make one singular async worker thread. I figure most people running complex web services in production would just do an eye roll at this…
Use of ORMs is often a performance choke point. Raw DB queries are often much, much faster. Almost always, the more you abstract, the worse you perform. It's great as a developer but not so great as a user.
I haven't touched an ORM in over 6 years, but unless they've improved since then, I honestly can't think of a single reason why anyone would choose to use one. They're clunky monstrosities that act only as guard-rails for inexperienced developers. Far better to invest a few days (which is realistically all you need) to improve their SQL skills and/or code-review practices.
Earlier quoted context omitted.
I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.
for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa
I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.
Earlier quoted context omitted.
If you're not paying attention it would look that way, yes. If you pay attention you'll notice that just.js is using postgres as their DB, while all of the node benchmarks are handicapped by either using mongoose/mongodb or MySQL. There is no node benchmark with postgres, but all of the fastest benchmarks used it. "lithium" is a good example to show how much of an impact switching to postgres has. All of the 4 lithiu…
What makes postgres special in these cases?
Further, for every framework/language that was tested with both MySQL and postgres (there's quite a few of them), the postgres one always ranks higher.
Earlier quoted context omitted.
If you're not paying attention it would look that way, yes. If you pay attention you'll notice that just.js is using postgres as their DB, while all of the node benchmarks are handicapped by either using mongoose/mongodb or MySQL. There is no node benchmark with postgres, but all of the fastest benchmarks used it. "lithium" is a good example to show how much of an impact switching to postgres has. All of the 4 lithiu…
What makes postgres special in these cases?
In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?
Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.
But IME real-world bottlenecks have more to do with overall architecture. People tend to heavily focus on technical details, such as concurrency architecture--the how. But the biggest opportunities for improved performance usually involve functional aspects of an architecture--the what. (Note that these aren't fixed categories; they're relative positions. A technical detail often becomes a functional model as development progresses.)
12 RPS is a long way from implicating SSL. If you get to the point where SSL is an identifiable bottleneck, you've either made a series of tremendously good decisions or exceptionally poor decisions.
Earlier quoted context omitted.
for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa
Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.
We don't need ORM, we need OQM (object-query mapping)!
Earlier quoted context omitted.
for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa
Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…
I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.
public interface UserDao {
@SqlUpdate("CREATE TABLE user (id INTEGER PRIMARY KEY, name VARCHAR)")
void createTable();
@SqlUpdate("INSERT INTO user(id, name) VALUES (?, ?)")
void insertPositional(int id, String name);
@SqlQuery("SELECT * FROM user ORDER BY name")
@RegisterBeanMapper(User.class)
List listUsers();
}