Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our journey in dropping the ORM in Go

alanilling.medium.com

Post reply on HN
Print view View original

111–120 of 157 posts

Re: Our journey in dropping the ORM in Go

#111

Post by zzzeek »

sa46 wrote: post #109

Having written a non-ORM Go-Postgres tool [1] similar to sqlc, I'm a big fan of this article, especially their acknowledgment that using SQL moves the eng culture towards data-centric engineering. Some thoughts: - Application code should not rely on database table structure (like the ActiveRecord pattern). Modeling the database as a bunch of queries is a better bet since you can change the underlying table structure…

say your application has 120 tables. Do you write out 120 "INSERT" statements, 120 "UPDATE" statements, 120 "DELETE" statements as raw strings or otherwise every time you need to do any DML ? if not, and instead you have something that dynamically writes these out given some kind of object state to persist, you are using an ORM.

similarly, when you transfer the states of structures and/or objects to those INSERT/UPDATE/DELETE statements, or the rows from the cursor execution of a SELECT statement into object/structural state, that is also using an ORM.

I tend to consider "a query builder that writes SELECT statements" to be the least important thing an "ORM" can do, and is not even necessary to still have a library defined as an ORM. The "mapping" is mostly to do with automating the movement of data between the objects / database, which is, the relational tuples mapped to object attributes.

Re: Our journey in dropping the ORM in Go

#112

Post by marktangotango »

makeitdouble wrote: post #49

I guess that there’s not many people going to write a long essay on how they use ORMs moderately, knowing the limitations, and it’s perfectly fine. It wouldn’t gather a whole crowd either. What feels wrong to me in this post is the lack of nuance on how they used their ORM. For instance the whole part on how the ORM generated queries they wish were done in a different manner: why didn’t they bother writing either low…

> It looks to me they could have kept their ORM for the 80% cases it works great, and hand tune the 20% that could be problematic. Just like any other optimization problem.

It doesn't make for "good" blog posts I suppose. Personally I find these discussions to be tiresome and depressing. Take for example this seven year old post from the creator of Hibernate[1]. Predates a lot of todays "seniors". This field is clearly stuck in local maxima on this and a few other areas. Will there ever be anything more than this endless trial and error?

[1] https://www.reddit.com/r/programming/comments/2cnw8x/what_or...

Re: Our journey in dropping the ORM in Go

#113

Post by sa46 »

zzzeek wrote: post #111
sa46 wrote: post #109

Having written a non-ORM Go-Postgres tool [1] similar to sqlc, I'm a big fan of this article, especially their acknowledgment that using SQL moves the eng culture towards data-centric engineering. Some thoughts: - Application code should not rely on database table structure (like the ActiveRecord pattern). Modeling the database as a bunch of queries is a better bet since you can change the underlying table structure…

say your application has 120 tables. Do you write out 120 "INSERT" statements, 120 "UPDATE" statements, 120 "DELETE" statements as raw strings or otherwise every time you need to do any DML ? if not, and instead you have something that dynamically writes these out given some kind of object state to persist, you are using an ORM. similarly, when you transfer the states of structures and/or objects to those INSERT/UPDA…

> Do you write out 120 "INSERT" statements, 120 "UPDATE" statements, 120 "DELETE" statements as raw strings

Yes. For example: https://github.com/jschaf/pggen/blob/main/example/erp/order/....

> that is also using an ORM

ORM as a term covers a wide swathe of usage. In the smallest definition, an ORM converts DB tuples to Go structs. In common usage, most folks use ORM to mean a generic query builder plus the type conversion from tuples to structs. For other usages, I prefer the Patterns of Enterprise Application Architecture terms [1] like data-mapper, active record, and table-data gateway.

[1]: https://martinfowler.com/eaaCatalog/

Re: Our journey in dropping the ORM in Go

#114

Post by ramchip »

eric4smith wrote: post #87

Heh. Recently I had to stand up a quick elixir project and decided to write all queries by hand and not use Ecto. It was extremely enjoyable writing every query from the start. Just thinking carefully about what columns I needed, and crafting the best joins and where clauses made efficiency baked in from the beginning. If one is not careful and just be lazy with an ORM you get back all columns all the time - and this…

For some data e.g. users I use two Ecto schemas for the same table, one with the full data and one with a small subset (id, username, full name). This makes it really easy to load just the data that's truly needed in associations etc.

Re: Our journey in dropping the ORM in Go

#115

Post by paulryanrogers »

exdsq wrote: post #85
marcus_holmes wrote: post #68

Earlier quoted context omitted.

this. Don't write your SQL into the database. Write them as scripts (starting with "drop XYX", then "create XYZ", check them into git, and treat them as code. Migrations are for schema changes. Views and functions are not schema.

Wait. People don't do this?

In the dark times even SCM was rare. We were ignorant cowboys strait out of school. We learned things the hard way. Then found out most such problems were solved decades before.

Re: Our journey in dropping the ORM in Go

#116

Post by perl4ever »

marcus_holmes wrote: post #3

So frustrating. You got like 80% of the way there, and then went "nope, too much work" and diverted to add more complexity. The answer is to write the SQL yourself, and the scan methods yourself. Code generation is better than ORM, but still a wrapper, still adds complexity, and still brings problems. Yes it's a pain in the arse to write all that boilerplate in one go (pun intended). But if you'd started without an O…

I spent years writing ad-hoc SQL, and had no occasion to learn about or use ORMs, but I did dabble a little in T-SQL and PL/SQL, and they just seemed wrong to me. Not integrated in a clean manner, not derived from a single vision, and so on. I guess what people call an "impedance mismatch". There is excessive overhead just from switching from SQL to the imperative language. I've gotten four orders of magnitude speedup by reformulating a T-SQL script as a single SQL query.

But recently I learned about Power Query (the "M" language standing for Mashup) and I immediately wanted so badly for SQL to acquire all the good aspects of it. Not as a separate module, but perhaps as a successor language.

In particular, the way it allows functions (and higher-order functions) to be defined is wonderful.

Many people don't like SQL, but I never had the feeling that here is what it's missing until now.

I qualify my comments above by mentioning I have no experience with OLTP or in general SQL that needs to be performant on the millisecond to second timescale.

Instead, I've always been working with queries on the basis that they need to run in 5 minutes, or ten, or fifteen, or an hour, as opposed to a day, or a week, or a month.

Re: Our journey in dropping the ORM in Go

#117

Post by unclebucknasty »

imposterr wrote: post #105
marcus_holmes wrote: post #3

So frustrating. You got like 80% of the way there, and then went "nope, too much work" and diverted to add more complexity. The answer is to write the SQL yourself, and the scan methods yourself. Code generation is better than ORM, but still a wrapper, still adds complexity, and still brings problems. Yes it's a pain in the arse to write all that boilerplate in one go (pun intended). But if you'd started without an O…

> I write a view for each access method (so I can change the schema without worrying about changing every access method), and a function for each update/insert/delete (for the same reasons) I’d appreciate it if you could expand on what this means exactly.

Not OP, but sounds like they're using views for reads and functions (stored procs) for writes.

Idea is to insulate code from schema changes as much as possible. If the underlying schema changes, it's possible to sync the views/functions to those changes so the code itself isn't impacted, as it's just talking to the views and functions. Kind of an exposed interface to the DB.

It's not perfect, and some schema changes will invariably require code changes, but should be minimized.

There are pros and cons to this approach and I personally don't favor it, although I am aware of some shops that made it an absolute requirement--especially pre-ORM.

Re: Our journey in dropping the ORM in Go

#118

Post by ufmace »

kevmo314 wrote: post #24

As a junior dev I had no idea what an ORM was. As a mid-level dev I discovered them and wanted to use them everywhere. As a senior dev I've gone back to manually writing queries. Such is life.

I lean nowadays towards:

For ~90% of your database use, ORMs do basically the same queries and get rid of a massive amount of boilerplate and potential errors. Use 'em so you don't spend all day manually mapping object fields to the right slots in SQL statements.

For the other ~10% of database use, ORMs make it a massive pain to write a query that does what you actually want, or do something boneheadedly inefficient. So use an ORM that makes it easy to drop to pure SQL when you need to, and don't hesitate to do it if the situation calls for it.

Re: Our journey in dropping the ORM in Go

#119

Post by unclebucknasty »

hparadiz wrote: post #98
kevmo314 wrote: post #24

As a junior dev I had no idea what an ORM was. As a mid-level dev I discovered them and wanted to use them everywhere. As a senior dev I've gone back to manually writing queries. Such is life.

I've gone past this and have a very clean and efficient query builder in my ORM. I maintain one for work and another is my open source PHP framework (15 years old now). Mine supports simple joins and relationships but anything beyond that requires writing custom queries.

How does your query builder work? Is it an API that you use to build queries during runtime or do you generate database access code from queries?

Re: Our journey in dropping the ORM in Go

#120

Post by ufmace »

mleonhard wrote: post #86

// updates the whole table, ignoring conditions db.Model( ).Where( ).Updates( ) // updates the rows matching conditions db.Model( ).Updates( ).Where( ) This is a massive footgun in Gorm. Don't use tools that help you make catastrophic errors.

Yeah wow. I don't understand why any ORM would ever let you do that. I don't think I've ever heard of a case where someone actually wanted to update every row in a table from application code.

IMO, any decent ORM should block that entirely and force you to drop to SQL if you actually want to do it, or make you call some obnoxiously named function, like:

db.Model().YesIReallyDoWantToUpdateEveryRowOnTheTable().Update()

Post reply on HN