I think the hate comes from two sides:
1) Relational folks who feel that ORMs don't really enable good relational design, and
2) OO folks who feel that ORM's cause a lot of headache regarding relational to operation data structure conversion.
The folks in camp 1 are usually absolutely right. Most ORMs encourage the developer to design the database around the ORM which has to be an antipattern when it comes to relational systems. This isn't always the case. DBIx::Class has done some awesome work on some things like composite primary keys and the like. But generally in most cases it is true.
On the OO side I am less convinced of the costs. So my complaints come from the relational side. Yes object to operational data structure mapping is hard to automate, and it's even harder to automate well. So maybe we shouldn't try. Maybe ORM's should be disregarded in favor of ways of designing the data so that abstractions occur at different points.
In the LedgerSMB project we decided to use stored procedures to accomplish this abstraction. We don't put "all the logic" in the database, far from it. But database operations go through the stored procedures, and these are often well tuned for performance (or as well tuned as they can be). The stored procedures thus create an API for addressing relational data, and this allows an abstraction layer to be formed which maps these into object methods. We'll probably extend this further in the future to tie object properties to complex database types, but we haven't decided the best way to do this yet.
ORMs get a lot of hate because they tackle a very difficult problem and consequently tend to produce results that are unsatisfactory particularly to db folks. They are the product of app designers looking at the db, more often than not, rather than db designers looking at the app.