Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our journey in dropping the ORM in Go

alanilling.medium.com

Post reply on HN
Print view View original

121–130 of 157 posts

Re: Our journey in dropping the ORM in Go

#121

Post by mping »

nauticacom wrote: post #5

Never understood the hate for ORMs. I need to map from my data storage to my domain model somehow, why write all that code myself

ORMs are different from SQL generators; an eg: typesafe sql generator can be a great thing; the problem is mapping sql results (wWhich are basically rows of data) to Objects. Objects have identity, but an SQL row can be anything, even if most of the time it's a full row from a table with a primary key.

As an example, if you do a query for Object o but with partial fields, and later you query for same object but full fields, are they the same object? SQL doesn't care, but in application code you need to ensure they are the "same", because objects will flow in between functions and if you eg: do o.setAge(30), you want both of the live objects to reflect that (for the same transaction).

It's a hard problem with lots of compromises or rules, and I think most people are better off just with a query api.

Re: Our journey in dropping the ORM in Go

#122

Post by kgeist »

We had many problems using ORM's in production:

- performance issues (DB calls in a loop the worst offender)

- various obscure bugs due to the concept of "flushing" (for example, we just deleted an entity, but it's still retrievable from its repository in the same transaction, which breaks logic)

- additional cognitive load because you have to learn ORM's concepts and how it maps to both DB and your domain model

- you have to either add ORM-specific attributes directly to domain objects (which is leakage of infrastructure details to the domain layer) or write tons of functions which convert ORM models to your domain entities and back

So in new code we now use raw SQL and we don't find it all that hard - it's easy to maintain and doesn't take a lot of time. It's problematic with complex aggregates, but our models tend to be anemic.

There's the hypothetical "what if you'll want to change the DB" but all DB calls are encapsulated in the infrastructure layer, so we'd only need to rewrite repository implementations. And you can migrate to a new DB gradually, microservice by microservice, and a typical microservice won't have more than ~20 repositories, so it's pretty manageable.

Re: Our journey in dropping the ORM in Go

#123

Post by noisy_boy »

mcny wrote: post #93
domlebo70 wrote: post #88

Earlier quoted context omitted.

What is the idea behind using views? I have never used them

Not OP but two main things: 1. Access control - you can create new SQL accounts and only give them access to select from specified views 2. Encapsulation and implementation hiding - the underlying table structure can change as long as the views exposed doesn't change. At least that's all I know. I'm sure people will add or correct when they read this.

These are correct - some others are:

- ability to do transformations uniformly e.g. if you want to represent a value in a certain way uniformly instead of having to do that in multiple places

- ability to get generated values (like 'create_ts() - now()' to get the age of the record) directly from the database instead of computing it repetitively

- move the join mechanisms on the database side to do those efficiently instead of the ORM having to do it (sometimes suboptimally); also helps with joins that are repetitive

- ability to apply certain "default" filters e.g. if you are always fetching with predicate 'active = true', then you can include that in the view

Re: Our journey in dropping the ORM in Go

#124

Post by noisy_boy »

Am I the only one who doesn't see this as a binary choice? I've written SQL for decades and I still like ORMs for most scenarios because:

1. I always check the SQL being generated

2. I profile the SQL being generated to ensure that the indices are being used optimally

3. I only fetch what I need and try to reduce trips to database

4. Most of the time, this is good enough and helps reduce boilerplate.

5. Sometimes, after doing the above, I see that the ORM isn't upto the task - then I just drop to SQL and get exactly what I want the way I want it. The ORM doesn't block me from doing so.

Re: Our journey in dropping the ORM in Go

#125

Post by laumars »

dagmx wrote: post #81
kevmo314 wrote: post #24

As a junior dev I had no idea what an ORM was. As a mid-level dev I discovered them and wanted to use them everywhere. As a senior dev I've gone back to manually writing queries. Such is life.

As a senior dev, I use an ORM unless I have a reason not to, and I'd know why or not fairly early. Why do I use an ORM? Because they're way more maintainable for a wider range of devs, with fewer accidental footguns, and the good ones let me do direct queries too, so there's little downside. So I default to using an ORM like sqlalchemy , because I can hand it off to a junior dev afterwards, and they can easily keep i…

If you can’t trust your junior devs to write SQL nor have sufficient peer review (like pull requests on feature branches) to assist with their training then you should be using stored procedures and remove that responsibility from your developers entirely. Either that or sending your juniors on training exercises to level up their SQL experience.

And you don’t need ORMs to abstract away the RDBMS engine from your main codebase. Nearly every programming language these days (even statically compiled ones which support almost nothing in the way of dynamic code, like Go Lang) will allow you to do this.

I’m almost all instances where people think they need ORMs, what they really need is better software development practices in the wider sense and ORMs were just an excuse to hide those larger problems.

Re: Our journey in dropping the ORM in Go

#126

Post by laumars »

cageface wrote: post #70
toast0 wrote: post #52

Earlier quoted context omitted.

ORM makes the easy things easier and the hard things harder. No thanks.

So choose an ORM that lets you easily drop down to plain SQL in the rare cases where it's necessary and you get the best of both worlds.

Or just do away with your ORM entirely and train your staff how to write SQL so they can be effective regardless of the complexity of the queries.

Re: Our journey in dropping the ORM in Go

#127

Post by hparadiz »

unclebucknasty wrote: post #119
hparadiz wrote: post #98

Earlier quoted context omitted.

I've gone past this and have a very clean and efficient query builder in my ORM. I maintain one for work and another is my open source PHP framework (15 years old now). Mine supports simple joins and relationships but anything beyond that requires writing custom queries.

How does your query builder work? Is it an API that you use to build queries during runtime or do you generate database access code from queries?

The framework has 3 layers, Query Builder -> ActiveRecord based ORM -> CRUD Controllers

The CRUD controllers are then mapped with a router to different paths.

When you create a CRUD controller you extend the base class which has everything you need and then you set a single parameter, className which sets the object for which this CRUD controller is for. The base class works out of the box with front ends like agGrid.

The query builder itself is only part of the puzzle.

Every single ORM in existence is converting the object's data from the native types of the language into the types of the database and vice versa. There's almost always some configuration for each object type in whatever format from native code to yaml to xml to plain text. Every ORM has a toDatabaseValue() normalization function that also goes in reverse when pulling data from the database. Some ORMs will call this "data mapper" or "mapping" or "normalization". When this becomes evident the queries themselves become suuuuper simple because what you send into your INSERT or UPDATE queries is a complete array of strings with field values already predetermined. This is necessary because a database engine might have dozens of types like enum, string, text, longtext,smalltext,bigtext,varchar, char but all of them in your language could be "string". Advanced ORMs will know the database engine you use and even warn you if you try to save a string over 255 chars to a (255) char database engine but most don't.

My code is available here. https://github.com/Divergence/framework/blob/6af1b6b0e56b25c...

Where people go wrong is trying to do this field normalization logic inside their query builder. That leads to all sorts of problems. The query builder shouldn't know what is going on with your ORM at all. It's just gluing strings together to form a query.

You can see my dead simple query builder here: https://github.com/Divergence/framework/tree/develop/src/IO/...

I honestly used to not have one as it's so simple but decided it was a good abstraction to assimilate from other frameworks. My query builder does not attempt to sanitize anything. The ActiveRecord class takes care of that through the data mapping conversion functions for all the basic HTTP CRUD functionality.

Re: Our journey in dropping the ORM in Go

#128

Post by laumars »

devoutsalsa wrote: post #80
cageface wrote: post #70

Earlier quoted context omitted.

So choose an ORM that lets you easily drop down to plain SQL in the rare cases where it's necessary and you get the best of both worlds.

100%. You don’t have to trash your 5 passenger car because you occasionally need to transport 6 people.

You also don’t buy a 5 seater car to begin with if you already know you need 6 seats.

Re: Our journey in dropping the ORM in Go

#129

Post by inshadows »

brodo wrote: post #73
GordonS wrote: post #37

Earlier quoted context omitted.

And when you eventually ascend to true mastery, you'll be able to enter a zen-like trance, in which you'll be able to tell when to use one or not, or indeed whether to use both (e.g. ORM for simple CRUD stuff, hand-written queries for everything else).

The problem is that most apps start out as simple CRUD apps. Also each dependency is a liability. People really underestimate how pulling in dependencies can lower your software quality. Most people have security bugs in mind, but there are also performance issues, memory leaks and logging-issues, needles abstractions and incompatibilities.

In my case I always wants to do something that is NOT CRUD, but guides on Internet always want to teach me how to do CRUD apps.
Post reply on HN