Live data from Hacker News

What ORMs have taught me: just learn SQL (2014)

woz.posthaven.com

341–350 of 360 posts

Re: What ORMs have taught me: just learn SQL (2014)

#341

Earlier quoted context omitted.

We have far more problems with too many .Includes causing terribly performing queries with bad JOINs than lazy loading problems. Granted this code base is in a bit of a state and we have a complex order structure that can go like 10 layers deep, and ideally we're looking to go even deeper with complex pricing. You do an include with all of that and you're going to get a terrible query. It's generally very cheap to do…

>We have far more problems with too many .Includes causing terribly performing queries with bad JOINs than lazy loading problems. Granted this code base is in a bit of a state and we have a complex order structure that can go like 10 layers deep, and ideally we're looking to go even deeper with complex pricing. You do an include with all of that and you're going to get a terrible query. Wait - are you "Include"ing th…

The example is too trivial, which is why it looks like it might be better. Here's a real world example, not even complete. A restaurant booking might have:

    - A venue associated with it
    - A user who booked it
    - A menu
    -- With courses (starter, main, dessert)
    --- of Menu items (steak)
    ---- with Menu item option groups (think, 'pick one of', 'choose at most 3', etc.)
    ----- of Menu item options ('rare', 'medium', or 'extra chips', 'onion rings')
    - Guests
    -- Guest.User
    -- Guest selections of menu items
    --- Guest selection options
    - And Many more! (payments, events, offers, postal addresses, etc.)
There are loads of things that are optional, or even extremely rarely filled in (say, an associated special area of the restaurant, or perhaps an assigned waiter, or a third-party partner who placed the booking). And we can just let the EF load that in lazily. It knows a nullable int means nothing to load, but if there is a third party supplier, it can go off and load that lazily (and extremely cheaply).

As for the query, you can load them in chunks (which we do), but the way the EF works you're limited on how you can do that.

If you try loading that all in one go, you get a very, very slow query. It's beyond the limits of the execution planner to do it well.

Because of the nature of ORMs, the EF can also make decisions which result in horrible sub-selects, or terrible joins of sub-tables where the execution planner can't use the right indexes, especially when you're trying to do groups, counts, sums, etc.

This makes it often better to load things separately and to selectively use the lazy loader to do certain things.

Other scenarios include where say you have a complex object that you've only partially filled in, but in 1 in 5 cases you want to send an email using that object.

Now you could write your email function to load all the data again, or you could let lazy loading do its thing and, overall, save time and decrease db load, because you've already got 80% of the data, it just needs to fill in the missing 20% with some simple queries.

Answer to earlier question: I use to hand-code my db upgrades as my opinion is that having correctly structured data is king and I understood relational db design. But it turns out EF Migrations are wonderful when you know how to use them. I still check every single one to make sure they're doing exactly what I wanted and expected though, and take them up and down manually. Good way of catching mistakes.

Re: What ORMs have taught me: just learn SQL (2014)

#342
post #50

Earlier quoted context omitted.

Sounds like you disabled lazy loading and that forced everything to be loaded at once. That's a user error, not a EF fault. This is a common theme I've seen with people blaming ORM's for being slow, it's the devs not using them appropriately more than the ORM's themselves. Not to say that they don't have their own issues.

With or without lazy loading enabled the result was the same. Generating the structure took seconds and went OOM with enough tables. LazyLoading impacts what data is retrieved from the database (or more to the point when), this is a structural issue before a query was even sent to the database. It would die while generating the query, not sending the query or populating the result. You likely should have asked for mo…

> LazyLoading impacts what data is retrieved from the database (or more to the point when)

It impacts more than just that, it will impact the query generation as well. If you have a property that is not lazy loaded then it will attempt to join the relation or load it very another query in the same round trip. Turning it off tells the ORM that every single time you want A it needs to go and get B as well. If you have it off universally it will attempt to load the entire database, or as may be the case here, crashing while trying to generate a query to do so.

> You likely should have asked for more information before concluding it was "user error."

Perhaps, but you've got multiple conflicting accounts of what went wrong, some comments indicate that it returned data, others say it never touched the database.

Re: What ORMs have taught me: just learn SQL (2014)

#343

In a recent project we went without an ORM. We took a tack similar to how PostgREST and PostGraphQL are structured. We use views in the public schema to build our objects. Functions constrain our mutations. Triggers respond to events and maintain consistency. It makes our web API code simple and hard to introduce errors that invalidate our customers’ data. Don’t miss having an ORM. Always seemed like more abstraction…

Why didn't you just use stock PostgREST, in which area did you found it lacking?(Curious)

Re: What ORMs have taught me: just learn SQL (2014)

#344
post #85

Earlier quoted context omitted.

Oh, the implementation was fairly straight-forward. I think just sorted arrays are something. It wasn't about speed of execution, but expressiveness when coding. Later on they even added proper type system support. Common operations were things like map/project, extend, filter, join, collect-by-key / expand, etc. Just as Codd pointed out in his original papers, relations allow you to not have to make a choice about a…

> Just as Codd pointed out in his original papers, relations allow you to not have to make a choice about a hierarchy for your data. Or, alternatively, make it really painful when you do actually need to query hierarchies, along the lines of "give me all the tuples above this one in the hierarchy".

Datalog can do those kinds of queries--transitive closures--easily, if memory serves right.

Datalog is a specifically chosen subset of Prolog.

Re: What ORMs have taught me: just learn SQL (2014)

#345

Earlier quoted context omitted.

I expect that everybody with a degree knows SQL and I'm realizing that I could be wrong. Maybe sometimes I'm the only one in the room that knows it. I'll check it next time I'm at a technical event leaning on the backend side.

I haven't written very much SQL in my career, there is quite a lot of development that doesn't use it.

[deleted]

Re: What ORMs have taught me: just learn SQL (2014)

#346
post #168

Earlier quoted context omitted.

Ok, let’s break this down: > Good ORMs are there to automate the repetitive tasks of composing largely boilerplate DML statements, facilitating query composition, providing abstraction for database-specific and driver-specific quirks None of that requires an ORM. A simple query builder will suffice and it will be much easier to debug and much less error prone than an ORM. > providing patterns to map object graphs to…

No, no, no, I know more than you. Here is why. Bitcoin cryptocurrency AI biomedical supply chain networking quantum systems-thinker.

Please don't post unsubstantive comments here.

https://news.ycombinator.com/newsguidelines.html

https://news.ycombinator.com/newswelcome.html

Re: What ORMs have taught me: just learn SQL (2014)

#347
post #340
post #214

Earlier quoted context omitted.

I can get very creative with SELECTs, making use of Prolog style queries, which are fully done server side on the database. Most ORMs will download all the data and evaluate them on the client side, with code that is even more convoluted that the SQL one and thus with less performance.

> Most ORMs will download all the data and evaluate them on the client side I've seen that pattern in lots of homegrown applications, but I've never seen such a thing in a mainstream ORM. Care to provide examples ?

I've seen it a lot in entity beans and the early days of hibernate.

Nowadays I only bother with Dapper, MyBatis, using a mix of SQL and stored procedures.

EF only if the RDMS happens to be SQL Server.

Re: What ORMs have taught me: just learn SQL (2014)

#348
post #107

Earlier quoted context omitted.

> A simple query builder will suffice and it will be much easier to debug and much less error prone than an ORM. What's the difference? To me, an ORM is largely a query builder.

ORM: You ask for something and you don't care _how_ its fetched. Query Builder: You build a query, just not in SQL. So you can get around SQL's limitations (like composition)

This sounds like a distinction that you personally have made, that the wider community probably won't agree on.

Re: What ORMs have taught me: just learn SQL (2014)

#349

Earlier quoted context omitted.

What "enormous" API surface? It's not about "optimizing initial deployment". It's about "optimizing continuous deployment" and having "always releasable Software". I've worked for departments with 15 devs all working on the same codebase and we could release and rollback releases (A/B deployments) every week because we treated the database as a dumb data store. We had multiple branches at the same time, etc. I've als…

I like the idea of stored procedures. I get some of the value they bring. However, the few times I've been on projects where sprocs where the primary focus of logic/truth/app... it was always a pain. * The 'developers' weren't allowed to write the sprocs. We were at the biz meetings, but the DB guys were hardly ever there - their meetings were separate for some reason, but because devs were at the meetings, the devs…

Your complaints about stored procedures seem to be about the nature of the company where they were used rather than any inherent flaw with them.

Re: What ORMs have taught me: just learn SQL (2014)

#350

Earlier quoted context omitted.

> And then what do you do if the ORM is generating junk? If the answer is "use a querybuilder/handcrafted SQL for that one", what's the point of the ORM in the first place? The point may be that 98% of the queries are just fine, and you've saved time vs writing by hand, and it may be easier to read/understand for the next people to have to touch the code.

You haven't saved time compared to using a query builder though; in fact you've lost time due to the hit to readability and debugability.

How do you know this? You have no idea what bugs were encountered or how easy they were to fix.
Post reply on HN