Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How not to structure database-backed web apps: performance bugs in the wild

blog.acolyer.org

Post reply on HN
Print view View original

71–80 of 319 posts

Re: How not to structure database-backed web apps: performance bugs in the wild

#71

Post by jlebrech »

radicalbyte wrote: post #25
mr__y wrote: post #12

Earlier quoted context omitted.

on the other hand there are savings in development time, which might or might not be more expensive than cpu cycles in question. Especially for smaller scale projects. I'm not saying that 4 seconds to load a page is something to be happy about, but sometimes it's cheaper to "waste" cpu cycles than to spend more time developing faster solution.

It only saves development time for the first version in my experience unless it's being used by an highly capable team. The months lost to the clueless developers massaging their horrible ORM code dominates in time. In the long term properly factored code wins; and lightweight ORMs have a place. Heavy ORMs (Hibernate, old Entity Framework) should only be used if you absolutely must have specific features and are inca…

you could start with an EASY query that's guaranteed to work, then AI could figure out a fast way as your app keeps running. it could be a module for postgresql for example.

Re: How not to structure database-backed web apps: performance bugs in the wild

#72

Post by brobdingnagians »

TekMol wrote: post #45
manicdee wrote: post #43

Earlier quoted context omitted.

That’s how I have seen it done in Perl, PHP, Django, etc as well. It’s not an ORM thing it is a naive programmer thing. The ORM makes it easier to do things like Class.filter(insurance_end_date But hey what do I know, I am not berating the youf of today so I don’t belong in this skit.

Class.filter(insurance_end_date Which ORM is that? It would be cool to compare some approaches to common problems.

C++ has a similar thing with sqlpp11: https://github.com/rbock/sqlpp11

Re: How not to structure database-backed web apps: performance bugs in the wild

#73

Post by adrianmsmith »

doesnt_know wrote: post #29
FooBarWidget wrote: post #17

Earlier quoted context omitted.

I find this question to be very weird. I have never seen ORMs as tools for completely abstracting away the database. I have always seen them as convenience APIs for using the database. They exist to make your code shorter, less repetitive, more readable, easier to reason about, and more maintainable; not to make you forget about the database altogether. I also don't see ORMs as exclusive. It's fine to use ORMs for 95…

> I have never seen ORMs as tools for completely abstracting away the database. Some are advertised that way. Entity Framework Code First, Migrations etc. At our place, our DB dev team is larger than our api team, which in turn collectively dwarfs our front end teams. ORMs in this sort of environment have never really been given a chance, but I feel like it would be easy to justify them in many situations...

> Some are advertised that way

As an example of this, the influential DHH of Basecamp blogged saying just that: https://m.signalvnoise.com/conceptual-compression-means-begi... "Basecamp 3 has about 42,000 lines of code, and not a single fully formed SQL statement as part of application logic!"

Re: How not to structure database-backed web apps: performance bugs in the wild

#74

Post by nicoburns »

TekMol wrote: post #56
dagw wrote: post #47

Earlier quoted context omitted.

That has nothing to do with ORMs. I've seen plenty of people write code like: SELECT * FROM DATA for each r in result if r.x > 12 do_something(r.y) Exactly the same thing without any ORM. If anything ORMs should improve performance for novices since it makes it a lot easier for people to writer better queries that run on the database.

The ORM makes it more likely to do stuff in the application instead of the DB because many queries are hard or impossible to represent in an ORM fashion. Additionally, the ORM makes it much harder to see what is going on under the hood. For example you don't know if a value of an object is stored in the main table or lazy loaded from a related table. So you have no clue if echo "$user.name lives in $user.city" result…

This is only true if you let the ORM design the schema for you. I would hazard a guess that most people only use ORMs for queries (including mutating ones). In which case you very much do know which table a field is in.

Re: How not to structure database-backed web apps: performance bugs in the wild

#75

Post by mavelikara »

crocal wrote: post #4

... Which begs the question: what good is an ORM if it does not prevent by design such issues? Here, we are essentially saying users of ORM must also have in their mind the SQL version. Or call an expert after the mess is done :/...

> Which begs the question: what good is an ORM if it does not prevent by design such issues?

This isn't as terrible as you make it out to be. For example, general purpose programing languages don't prevent the programmer writing a O(2^n) algorithm; yet they are useful.

Re: How not to structure database-backed web apps: performance bugs in the wild

#76

Post by dagw »

TekMol wrote: post #56
dagw wrote: post #47

Earlier quoted context omitted.

That has nothing to do with ORMs. I've seen plenty of people write code like: SELECT * FROM DATA for each r in result if r.x > 12 do_something(r.y) Exactly the same thing without any ORM. If anything ORMs should improve performance for novices since it makes it a lot easier for people to writer better queries that run on the database.

The ORM makes it more likely to do stuff in the application instead of the DB because many queries are hard or impossible to represent in an ORM fashion. Additionally, the ORM makes it much harder to see what is going on under the hood. For example you don't know if a value of an object is stored in the main table or lazy loaded from a related table. So you have no clue if echo "$user.name lives in $user.city" result…

The ORM makes it more likely to do stuff in the application instead of the DB

Again this doesn't match my experience. A mediocre .NET developer who can access the database with LINQ is far more likely to run their queries on the database than a mediorce .NET developer who has to try to write SQL queries by hand.

And the truth is that modern ORMs are often pretty clever with their optimizations, and in many cases the ORM will often outperform an average developer doing the obvious thing is SQL.

Re: How not to structure database-backed web apps: performance bugs in the wild

#77

Post by strken »

grey-area wrote: post #54
TekMol wrote: post #21

Give me an O! Give me an R! Give me an M! What does that spell? SLOW PERFORMANCE! Todays programmers dont understand data. They understand frameworks. To find the nr of all cars that are out of insurance they write: 10 Nr=0 20 Hey framework, give me all cars! Framework: Ok, here are 8001093 business objects representing all the cars in our DB. Each has all the attributes the car has. Color, mileage etc. 30 Thanks! 40…

I use an ORM. I don't perform queries like that because the ORM makes it easy to build complex queries, joins, limiting the data returned etc, then execute them in one query, it's a convenience, not a straightjacket. The problem is not ORMs, the problem is just people not thinking about the resources their query over some data takes, and trying to do things in memory that are better done in the database (as in your e…

One of the biggest problems with ActiveRecord (the ORM in the article) is that it uses extremely obtuse names for extremely common methods. There are three different methods for finding how many things meet a given criteria[0], and none of them do the same thing. There are three different methods for loading data from an associated table[1], and none of them do the same thing either. None of the method names explain how the function is implemented or whether it's going to hit the database.

This is a problem that goes beyond the usual ORM antipatterns and bad database designs and becomes its own special kind of hell, because it makes code far harder to reason about and thus far harder to review, in a way that I don't remember encountering in Hibernate, Gorm, or anything else.

[0] count, length, and size

[1] includes, preload, and eager_load

Re: How not to structure database-backed web apps: performance bugs in the wild

#78

Post by grey-area »

strken wrote: post #77
grey-area wrote: post #54

Earlier quoted context omitted.

I use an ORM. I don't perform queries like that because the ORM makes it easy to build complex queries, joins, limiting the data returned etc, then execute them in one query, it's a convenience, not a straightjacket. The problem is not ORMs, the problem is just people not thinking about the resources their query over some data takes, and trying to do things in memory that are better done in the database (as in your e…

One of the biggest problems with ActiveRecord (the ORM in the article) is that it uses extremely obtuse names for extremely common methods. There are three different methods for finding how many things meet a given criteria[0], and none of them do the same thing. There are three different methods for loading data from an associated table[1], and none of them do the same thing either. None of the method names explain…

Yes there are a few problems with ActiveRecord; it has been well used and has accreted lots of similar functions (a victim of its own success), and also it doesn't make it crystal clear when you are hitting the database.

IMO ORMs should be simple, unsurprising, and only have one or two methods which actually execute the sql and fetch data - they are there to make it simpler to build queries and map rows to objects, not to blur the line between fetching and manipulating data.

Re: How not to structure database-backed web apps: performance bugs in the wild

#79

Post by crocal »

sheepz wrote: post #18
crocal wrote: post #6

Earlier quoted context omitted.

Too much CPU and too much available memory will get you there. When in time of abundance, we humans are not good at optimizing.

It's more of an economics issue. It does not make sense to optimize, because bandwidth and CPU power are so abundant. Optimizing will take time away from adding new features.

Except you can’t add so many features because you have sucked up all resources already ^^

Re: How not to structure database-backed web apps: performance bugs in the wild

#80

Post by [deleted] »

nicoburns wrote: post #74
TekMol wrote: post #56

Earlier quoted context omitted.

The ORM makes it more likely to do stuff in the application instead of the DB because many queries are hard or impossible to represent in an ORM fashion. Additionally, the ORM makes it much harder to see what is going on under the hood. For example you don't know if a value of an object is stored in the main table or lazy loaded from a related table. So you have no clue if echo "$user.name lives in $user.city" result…

This is only true if you let the ORM design the schema for you. I would hazard a guess that most people only use ORMs for queries (including mutating ones). In which case you very much do know which table a field is in.

[deleted]
Post reply on HN