Live data from Hacker News

Our journey in dropping the ORM in Go

alanilling.medium.com

131–140 of 157 posts

Re: Our journey in dropping the ORM in Go

#131

Earlier quoted context omitted.

How does your query builder work? Is it an API that you use to build queries during runtime or do you generate database access code from queries?

The framework has 3 layers, Query Builder -> ActiveRecord based ORM -> CRUD Controllers The CRUD controllers are then mapped with a router to different paths. When you create a CRUD controller you extend the base class which has everything you need and then you set a single parameter, className which sets the object for which this CRUD controller is for. The base class works out of the box with front ends like agGrid…

[deleted]

Re: Our journey in dropping the ORM in Go

#132
It’s a good article. At Namely we use Dapper which is a “micro-ORM” developed by the folks at StackOverflow for .NET. I really like it because it gives us the mapping part without the sql code generation part which I think is the thing that ultimately gets you in trouble, and it works really well with Postgres and SQL Server.

Re: Our journey in dropping the ORM in Go

#133

Earlier quoted context omitted.

How does your query builder work? Is it an API that you use to build queries during runtime or do you generate database access code from queries?

The framework has 3 layers, Query Builder -> ActiveRecord based ORM -> CRUD Controllers The CRUD controllers are then mapped with a router to different paths. When you create a CRUD controller you extend the base class which has everything you need and then you set a single parameter, className which sets the object for which this CRUD controller is for. The base class works out of the box with front ends like agGrid…

Interesting. Thanks for sharing!

I wrote something for Java roughly 20 years ago (before ORM was really a thing). Of course I enhanced as needed over the years, but have used it heavily. It supports any DB with a JDBC driver and has easily saved me more time than anything else. There's not even a close second.

It's a pure code generator with a UI on top. You write an SQL select stub that references columns/tables for one or more tables, then provide the class name you want to generate. You then add any number of where clauses (including join conditions and parameterized placeholders). For each where clause, you use checkboxes to indicate what you want generated: select, update, insert, upsert, delete.

When the code is generated, all column-to-object field mappings, including types, are resolved through inspection of metadata (provided by the JDBC driver). Property names for the resulting domain objects are also generated using snake to camel case translations and, of course, methods resulting from parameterized statements include parameters of the correct types.

So, instead of having, say, a User class with a Profile class, which has an Image class, each with its own properties representing the full set of columns for the underlying table, you frequently need a limited subset of each combined into one object. For instance, a ProfileListView might need username, email, first and last from user; profile_pic_id, slug, and short_bio from profile; and thumb_filename from image.

There's more to it, but I took this join approach vs the "one table to one object" model because I found early that you frequently need different views of the data, based on different use cases. So, it's a way to flatten the underlying relational model when needed.

Not perfect, of course, but the best thing going when I built it. And since, I never found an ORM that was worth the switch.

Re: Our journey in dropping the ORM in Go

#134
I tried to use ORMs in Java years ago, but realized that while it makes simple things simpler, sooner or later you will need to do something that the ORM can't do simply. Or you will bang your head against the fact that you are trying to marry object graphs to relational models and you'll burn a lot of CPU, IO and memory in the translation. I have to admit I've been against ORMs ever since. You end up having a weak grasp on what is happening under the bonnet.

You can make much better use of your database, and understand what is going on a lot better, if you let the database be the database.

My approach is to forget about the database, and focus on what the application needs from the persistence layer. Then design an interface for the persistence. Then I implement that interface for each kind of persistence backend I need (SQL, files, memory, etc) while maintaining a single set of unit tests and benchmarks that are written against the interface. This isn't new. A lot of applications and libraries do this. And I think it is a good way to design things.

Usually the database I use to develop a SQL schema is Sqlite3, since it allows for really nice testing. Then I add PostgreSQL support (which requires more involved testing setup, but I have a library that makes this somewhat easier: https://github.com/borud/drydock). (SQLite being in C is a bit of a problem since it means I can't get a purely statically linked binary on all platforms - at least I haven't found a way to do that except on Linux. So if anyone has some opinions on alternatives in pure Go, I'm all ears)

In the Java days JDBC every single method implementing some operation would be a lot of boilerplate. JDBC wasn't a very good API. But in Go that is much less of a problem. In part because you have struct tags, and libraries like Sqlx. To that I also add some helper functions to deal with result/error combos. Turns out the majority of my interactions with SQL databases can be carried out in 1-3 lines of code - with a surprising number of cases just being a oneliner. (The performance hit from using Sqlx is in most cases so minimal it doesn't matter. If it matters to you: use Sqlx when modeling and evolving the persistence, and then optimize it out if you must. I think I've done that just once in about 100kLOC worth of code written over the last few years).

And best of all: I get to deal with the database as a database. I write SQL DDL statements to define the schema, and SQL to perform the transactions. I don't have to pretend it is a object model, so I can make full use of the SQL. (Well, actually, I try to make do as far as possible with trivial SQL, but that's a whole different discussion). The interface type takes care of exposing the persistence in a way that fits the application.

(Another thing I've started experimenting with a bit is to return channels or objects containing channels instead of arrays of things. But there is still some experimenting that needs to be done to find a pleasing design)

Re: Our journey in dropping the ORM in Go

#135

It’s a good article. At Namely we use Dapper which is a “micro-ORM” developed by the folks at StackOverflow for .NET. I really like it because it gives us the mapping part without the sql code generation part which I think is the thing that ultimately gets you in trouble, and it works really well with Postgres and SQL Server.

Yes. The key is designing to the tool’s strength. Build a row oriented database model in the application language that matches the DB schema. This is trivial for the ORM (like Dapper or ActiveRecord) and you save boiler plate. For simple apps this might be enough. For richer domain models it is often simpler to map to the domain model from the database model mapping in application code than fighting the ORM to do it just right from DB to domain model. Another alternative is to design to tools and design your domain model to reduce the friction from the ORM. These trade-offs are specific to each application.

Re: Our journey in dropping the ORM in Go

#136

Heh. Recently I had to stand up a quick elixir project and decided to write all queries by hand and not use Ecto. It was extremely enjoyable writing every query from the start. Just thinking carefully about what columns I needed, and crafting the best joins and where clauses made efficiency baked in from the beginning. If one is not careful and just be lazy with an ORM you get back all columns all the time - and this…

Ecto is pretty damn nice (also technically it's not an ORM - and that's not a trivial distinction). Yeah, it adds a little bit of latency, but that will usually be overshadowed by the DB transactions... Am I mistaken about this? Are you not afraid that your queries will have SQL injections? Or handle unicode poorly, binary blobs, etc? > you get back all columns all the time I think this is why you can map multiple sc…

Ummm we use the PG driver through the Ecto Repo which handles SQL injections.

Re: Our journey in dropping the ORM in Go

#137

Heh. Recently I had to stand up a quick elixir project and decided to write all queries by hand and not use Ecto. It was extremely enjoyable writing every query from the start. Just thinking carefully about what columns I needed, and crafting the best joins and where clauses made efficiency baked in from the beginning. If one is not careful and just be lazy with an ORM you get back all columns all the time - and this…

If you’re not careful and just be lazy with your hand crafted queries you’ll miss updating old queries when you alter your schema. You won’t convert your types the same way in each places you query the data, or you’ll forget checking your data’s consistency where it’s falling in the cracks between your SQL rules and the application’s requirements. The “not careful and just be lazy” way won’t get you in a good place w…

The PG Driver through the Ecto Repo nicely handles and converts all the database types into Elixir types handily. Integers go to Ints and JSON B gots into Maps as one would expect.

Re: Our journey in dropping the ORM in Go

#138
post #25

Earlier quoted context omitted.

Using MikroORM in NodeJS and works wonders. Has the "raw" option for queries I want to handle myself, or even the "knex" builder for something in between

Hmm... Looks like a "active record" pattern (like Hibernate). This is exactly what I don't want. Essentially I only use ORMs as query builders. No magic behinds the scenes.

Author said it isn't[0] a year ago but not sure. I think it allows a couple of strategies to be used.

[0] https://github.com/mikro-orm/mikro-orm/issues/403

Re: Our journey in dropping the ORM in Go

#139
post #50

The actually story here is having the hubris to think you can build a good ORM (in Golang no less, lol) with a handful of engineers at a small company that's not even a technology company. How did this guy become a CTO again? Complete failure of leadership. Mature ORMs (that still often suck) are built by literally thousands of open-source contributors.

So you contradict your already pompous and judgmental initial statement with your last one. Good job.

Re: Our journey in dropping the ORM in Go

#140
post #67

Earlier quoted context omitted.

Why did it not age well? Are there any new findings regarding Vietnam?

To most people in the world, "Vietnam" is not a war, and is not merely a symbol of absurdly misconceived incompetence.

Yes. The author should've said "the Iraq of programming."
Post reply on HN