Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our journey in dropping the ORM in Go

alanilling.medium.com

Post reply on HN
Print view View original

151–157 of 157 posts

Re: Our journey in dropping the ORM in Go

#151

Post by marcus_holmes »

KronisLV wrote: post #149
aidos wrote: post #145

Earlier quoted context omitted.

I’ve worked in all those places (currently in the fully versioned model). We occasionally do a full reset using an export of the current schema as a new base and then start versioning again from there. So there’s not really a good technical reason to stop people from getting a working runnable migration system going - other than social, as you say.

> So there’s not really a good technical reason... One that i can think of is the application depending on some set of data being present in the system, which was added ages ago and that no one has any idea about across the hundreds of tables. Furthermore, you probably can't just do a full export/import of the initial state, because everyone having to download about 2 GB of data a large portion of which is auditing r…

the data in your system is massively more important than the code. Not spending a week exploring and completely understanding your schema is going to hurt you in the long term.

Re: Our journey in dropping the ORM in Go

#152

Post by dagmx »

laumars wrote: post #150
dagmx wrote: post #148

Earlier quoted context omitted.

You say it's not a hypothetical argument, yet the majority of your comment is based on a projected hypothetical. In fact the majority of your argument isn't even focused on the discussion of an ORM but on a situation you made up. ---- That said, let's address your frankly odd rebuttal... Training takes time , and in a lot of cases is going to have a negligible gain over just using the ORM to begin with. Regarding eve…

> In fact the majority of your argument isn't even focused on the discussion of an ORM but on a situation you made up. No, the vast majority was explaining how development best practices resolve the common use case that is being presented here for why ORMs are a good thing (namely “my junior devs are somehow too stupid to learn SQL”). > That said, let's address your frankly odd rebuttal... What’s odd is that you thin…

Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks.

Every thing has an associated cost. An ORM in a familiar language, like sqlalchemy in Python, is easier to learn than learning SQL from scratch, and it provides guardrails like schema enforcement that SQL does not. That means anyone, not just a junior, can drop into a codebase using it and be able to quickly use it

You seem to be under the impression that an ORM has to have a significant performance penalty. While there's of course going to be some, for the vast majority of tasks that I use it for, it is negligible versus the rest of the IO happening. If there is a significant penalty in some edge cases, I can still use direct SQL queries. Again, like my original post, I know when and where to use an ORM and as I've tried to explain to you multiple times, I know it's not a silver bullet solution.

In your previous comment you simply went off the rails talking about branching and CI without tying it into your argument. If your argument is that CI will catch the issues, CI can only catch some issues. It too isn't a silver bullet, and relies on code review to catch some other issues. Why not just use an ORM to add that security up front? It doesn't replace CI or code review, but it greatly reduces the risk of each stage failing.

> they should all be developed using the principles I’ve described

Okay, you do you. You've dissalowed any reply of situational pragmatism for your own fastidious absolutism.

Re: Our journey in dropping the ORM in Go

#153

Post by laumars »

dagmx wrote: post #152
laumars wrote: post #150

Earlier quoted context omitted.

> In fact the majority of your argument isn't even focused on the discussion of an ORM but on a situation you made up. No, the vast majority was explaining how development best practices resolve the common use case that is being presented here for why ORMs are a good thing (namely “my junior devs are somehow too stupid to learn SQL”). > That said, let's address your frankly odd rebuttal... What’s odd is that you thin…

Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks. Every thing has an associated cost. An ORM in a familiar language, like sqlalchemy in Python, is easier to learn than learning SQL from scratch, and it provides guardrails like schema enforcement that SQL does not. That means anyone, not just a junior, can drop into a codebase usi…

> Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks.

Easy conclusion to make when every post is full of reasons why juniors can’t learn SQL ;)

> is easier to learn than learning SQL from scratch,

I very much doubt many juniors are learning SQL front scratch. They’d have learned some already at college. And if they don’t genuinely don’t have any experience with databases what do ever then you bloody well need to skill them up irrespective of whether they use an ORM. So again, your argument here has no teeth.

> In your previous comment you simply went off the rails talking about branching and CI without tying it into your argument.

I’ve explained how that ties in several times now. There’s a difference between you not understanding best practices and me not explaining them

> If your argument is that CI will catch the issues, CI can only catch some issues. It too isn't a silver bullet,

I agree

> and relies on code review to catch some other issues.

Not just code review, automated tests, vets and so forth too. But I also agree that code review is a key part.

> Why not just use an ORM to add that security up front?

Because ORMs aren’t a silver bullet either. Except you’re trading in due diligence with lazy abstractions that add runtime overhead.

> It doesn't replace CI or code review, but it greatly reduces the risk of each stage failing.

It doesn’t reduce the risk of it failing. It reduces the ability for code to be vetted. It adds overhead and runtime costs. It adds complexity and the only benefit is you can keep your juniors blissfully in the dark about how your database works. That strikes me as a stupid compromise.

> Okay, you do you. You've dissalowed any reply of situational pragmatism for your own fastidious absolutism.

I’m not being absolute. I’ve said POCs and hobbies can bypass these best practises. And I’m being pragmatic when I say that every organisation I’ve worked at, both as an engineer and as a leader, has benefited from these best practices. Hence why they’re called “best practices” rather than “pointless processes”. Every organisation I’ve worked at where I’ve seen teams bypassing these best practices have been dysfunctional. That might work for you but it results in shit code — which is likely why you’re dependent on ORMs to fix things for you. And I urge you to read more on this typic rather than just taking my word for it. :)

Re: Our journey in dropping the ORM in Go

#154

Post by dagmx »

laumars wrote: post #153
dagmx wrote: post #152

Earlier quoted context omitted.

Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks. Every thing has an associated cost. An ORM in a familiar language, like sqlalchemy in Python, is easier to learn than learning SQL from scratch, and it provides guardrails like schema enforcement that SQL does not. That means anyone, not just a junior, can drop into a codebase usi…

> Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks. Easy conclusion to make when every post is full of reasons why juniors can’t learn SQL ;) > is easier to learn than learning SQL from scratch, I very much doubt many juniors are learning SQL front scratch. They’d have learned some already at college. And if they don’t genuinely…

Nowhere have I said juniors can't learn SQL either.

This is such a pointless argument anyway. You aren't bothering to read what I'm writing, you're twisting what I'm saying in what little you do read by filling in your own blanks, and you're insinuating that I won't use other best practices just because I won't agree with your subjective definition of best practices.

I have read many arguments about this, and again, I am experienced enough to know when to use it. The issue here is you have a completely condescending view of anyone who's not sharing your exact view. Your posts repeatedly insinuate that your experience outweighs my own, without actually providing anything other than YOU think ORMs are bad.

Not every college teaches SQL, not every junior comes from the same technical background, and you're assuming a lot more about someone else's workplace than you have any basis to.

You're also kind of a jerk by further insinuating that I put up with shit code. Perhaps I just don't put up with shit personalities who have a chip on their shoulders.

Re: Our journey in dropping the ORM in Go

#155

Post by toast0 »

hparadiz wrote: post #101
toast0 wrote: post #60

Earlier quoted context omitted.

No real specifics, but I usually run into ORM in the context of I'm the person on the team with experience running MySQL and someone has a slow query. When they're hand built queries, I can usually provide a query or sequence of queries that provides the same data without knocking the server over and it gets put into production within the same day. When they're ORM, it takes days to find the query, and then more days…

You should be using an APM like NewRelic to find those slow queries. If that's not an option you should have a profiler that tracks how long each query took and have it print out the query to a log. Turn it on for a couple hours during peak time and analyze it for slow queries. Also use static table design when applicable and avoid running any query that locks the table up. If the query takes any longer than 500 ms y…

Finding the bad queries is easy --- mysql has a slow query log since forever. The problem is finding where the query was generated. And convincing the people who wrote the bad query to use a better one.

Even without that, show process_list or whatever it is tells you how long the query has been running, with mytop you can just look for anything that's been running for more than a couple screen refreshes and start yelling from there.

Usually, when I'm brought in, somebody else designed the tables, poorly, too. There's usually not a whole lot I can do about that, because I don't usually have enough authority to demand a data migration.

Re: Our journey in dropping the ORM in Go

#156

Post by laumars »

dagmx wrote: post #154
laumars wrote: post #153

Earlier quoted context omitted.

> Nowhere in any of my posts did I say juniors are too stupid to learn. Much like your previous comment, you are filling in your own blanks. Easy conclusion to make when every post is full of reasons why juniors can’t learn SQL ;) > is easier to learn than learning SQL from scratch, I very much doubt many juniors are learning SQL front scratch. They’d have learned some already at college. And if they don’t genuinely…

Nowhere have I said juniors can't learn SQL either. This is such a pointless argument anyway. You aren't bothering to read what I'm writing, you're twisting what I'm saying in what little you do read by filling in your own blanks, and you're insinuating that I won't use other best practices just because I won't agree with your subjective definition of best practices. I have read many arguments about this, and again,…

> Nowhere have I said juniors can't learn SQL either

Then why invest so much effort trying to convince me that they shouldn’t learn SQL or that ORMs are better than learning how the underlying database manages their relational data? ;)

If I come across arrogant it’s because I’ve spent meany years managing fixing a great many failing dev teams and the comments you’re making resonate heavily with those who are usually against the changes I make. They always come round to agreeing that those changes are for the better after living with them for a few months.

But I guess we’ll never prove that on a message board so I’m happy to agree to disagree for the sake of etiquette.

Re: Our journey in dropping the ORM in Go

#157

Post by hparadiz »

toast0 wrote: post #155
hparadiz wrote: post #101

Earlier quoted context omitted.

You should be using an APM like NewRelic to find those slow queries. If that's not an option you should have a profiler that tracks how long each query took and have it print out the query to a log. Turn it on for a couple hours during peak time and analyze it for slow queries. Also use static table design when applicable and avoid running any query that locks the table up. If the query takes any longer than 500 ms y…

Finding the bad queries is easy --- mysql has a slow query log since forever. The problem is finding where the query was generated. And convincing the people who wrote the bad query to use a better one. Even without that, show process_list or whatever it is tells you how long the query has been running, with mytop you can just look for anything that's been running for more than a couple screen refreshes and start yel…

That's very true and NewRelic has a built in trace route as well as a way to sort and view the slow queries. Even breaks down and does an analysis on the query. Coupled with logging down to the individual thread. It's very powerful.
Post reply on HN