Earlier quoted context omitted.
Great work. I used rezoom in couple of minor projects and like most that I could really trust that query runs if it compiles. Unlike in Entity Framework where you have unlimited possiblities write code which compiles just fine but crash runtime. In principle I don't like an idea that your primary language get compiled to SQL due it's very leaky abstraction. Instead I like expressing queries as data (ideally compile t…
Thanks. I agree composition was a pain point. I had various ideas to make things better, like adding support for "erased" views/functions/TVFs that would be inlined at compile time, but it always felt like it'd be hacky and still not solve enough problems. Type providers are such a cool language feature, but the way developing one works is too damn confusing. Especially when you try to publish one as an easy-to-use p…
To ORM or Not to ORM
291–300 of 300 posts
Re: To ORM or Not to ORM
#292Re: To ORM or Not to ORM
#293I've come to a couple conclusions, over the years. First, when you get down to it, the most-valued feature of ORMs is not the "writing queries in some language other than SQL" feature, it's the "not having to write a mess of mapping code" feature. Second, the biggest drawbacks to ORMs all derive from the "writing queries in some language other than SQL" feature. Fortunately, there are tools out there that solve the "…
I've come to the same conclusion, over the years. In the java/groovy/scala world I've written my own a couple times but have come to the conclusion JDBi [1] hits the sweet spot. 1. http://jdbi.org
Re: To ORM or Not to ORM
#294Earlier quoted context omitted.
Would that be a "RRM" (Record-Relational mapper)? If so we're already on board (in a different language). P.S. I bet you would never guess which typed language with great support for records makes it easy (for the most part) to build most type-safe SQL queries on the fly, even with projections, without explicitly defining types for every possible projection variation.
Any TypeScript libraries you are using?
Re: To ORM or Not to ORM
#295Earlier quoted context omitted.
In my experience, this is (almost) everyone I've ever worked with, in multiple teams, with multiple ORM's, in multiple industries.
I’d love to hear any war stories.
I found this code because I was wanted to fix an issue where if you sort the list in the UI, it only sorted the current page (because the sorting happened in the frontend only). So I wanted to change the query so that sorting and paging are both done in the database. What a rabbit-hole that was! I was able to refactor the code to not over-fetch quite as badly, and to move the filtering into the initial query (so only additional data fetching happens after the initial query now, but it does still over-fetch fields and that's not easy to change since this function is called by many endpoints and the set of fields they all want varies). I had to log all of the queries so that I could construct a new query and then convert that back to ORM code. It was a very tedious task, but I managed to get the query from taking 9 seconds to under a second.
In a previous project, I was tasked with optimising the database, because requests were incredibly slow. The issue was that no care had been taken 1) with indexes -- indexed fields were too large, and 2) with locking. The locking was the big one as there were some requests that caused entire tables or large amounts of rows to become locked, which in turn would prevent other transactions from completing, so otherwise fast queries were now also slow, which in turn... you get the idea. This was in a large well established codebase, so it was not easy to fix and it still wasn't fully fixed when I left that team.
Many of these issues could be avoided by considering the data and access patterns when designing your database model (which doesn't have to be identical to your application model -- database should IMHO be modeled around query patterns and application model around domain features), with some care taken for indexing and locking. You want to make sure that different users usage patterns overlap as little as possible to avoid locking, that your indexes are small, that your queries fetch only what you need (both in terms of columns and rows).
None of these issues are the ORM's fault per se, but the ORM encourages developers to think in terms of their application model and primary programming languages OO facilities, so people (in my personal experience, at least) tend to mix database queries and application logic too much (like what I described above: pull data from DB, do some filtering that maybe should have been in the query, do more queries that maybe or maybe not should have been joins), tend to overfetch (often the ORM pulls in too many columns unless you make it not do so, which often is not done), using indexes as an afterthought and not considering locking at all. I think because ORM make it look like "just some more application code", these things are often overlooked, while with SQL queries, its a little more obvious that it's executed differently. I would have said its down to junior developers or whatever who just haven't learned to take care when writing queries, but I've only ever worked with two people who actually did this and I can't believe that the rest of all of the teams were just inexperienced.
I'm no database expert, but I've managed big wins with small amounts of care and by understanding what queries I'm running.
PS: The book SQL Performance Explained is quite good, in my opinion, and if you use Postgres, then Mastering PostgreSQL 11 is great (and it starts off talking about transactions and locking).
Re: To ORM or Not to ORM
#296Earlier quoted context omitted.
Who said anything about it being a trivial app? It was an ETL stack for distributed networking monitoring, and data reporting.
If you can comfortably use JDBC for data access, then your data access logic isn't very complicated. There's nothing wrong with that; use the right tool for the job.
Re: To ORM or Not to ORM
#297My opinion is to stick to plain SQL, but to use tools around if you like: query builders, transactions utilities, mappers. But no ORM for me, thank you. I have already given.
Re: To ORM or Not to ORM
#298For everyone complaining about orm-something - take a look at elixir's Ecto. It basically allows writing sql via native code. Good for composability, type casting, stuff like that. Account |> where(active: true) |> join(:left, [a], p in Post, on: p.account_id == a.id) |> where([a, p], a.foo == "bar" or p.bar == "foo") |> group_by([a, p], a.id) |> select(...) |> limit(10) ... |> Repo.all() Plus there are changesets fo…
|> where([a, p], a.foo == "bar" or p.bar == "foo")
are these predicates being pushed down to the data base? How can the code expressions be inspected for optimization? Is it using some kind of macro?Re: To ORM or Not to ORM
#299Earlier quoted context omitted.
I’d love to hear any war stories.
In the project I'm currently working on, objects are fetched from the database using ORM queries and then the results are iterated over, filtered further and additional queries are made to fetch more data. This is then all sent to the frontend, where only about two or three of the couple dozen fields are actually used. I found this code because I was wanted to fix an issue where if you sort the list in the UI, it onl…
Re: To ORM or Not to ORM
#300Earlier quoted context omitted.
In the project I'm currently working on, objects are fetched from the database using ORM queries and then the results are iterated over, filtered further and additional queries are made to fetch more data. This is then all sent to the frontend, where only about two or three of the couple dozen fields are actually used. I found this code because I was wanted to fix an issue where if you sort the list in the UI, it onl…
This fits my experience as well. Tracing the queries SQLAlchemy builds based on the models of our Python application the DB is returning all of the columns for every model instead of just the columns that will then be returned via the API to the client. They’re not even just the queries to compute the relation but all the columns from every relation involved and then at return time only a small subset of columns is u…
As an experiment, I changed one to only fetch the fields that it needs and the query ran in approx. 15% of the time. That’s a pretty big improvement! Sigh.