Live data from Hacker News

What ORMs have taught me: just learn SQL (2014)

wozniak.ca

581–590 of 654 posts

Re: What ORMs have taught me: just learn SQL (2014)

#581
post #467
post #439

Earlier quoted context omitted.

There is a big difference between just writing helper functions to construct SQL and convert data types, and OO-style magical auto-persisted objects. The latter is what I don't like about ORMs but the former is fine. I feel that this is an important distinction to make. As an example, the sqlalchemy docs[0] make this very clear: there's an ORM, but there's also just a core expression library that simply helps you con…

Agreed. Helpers (and indeed types) can make working with SQL an actual pleasure. You do need to learn the SQL, though. (My TypeScript/Postgres solution, in this vein: https://github.com/jawj/mostly-ormless/blob/master/README.md ).

This looks great. It looks similar to JooQ.

Re: What ORMs have taught me: just learn SQL (2014)

#582

Earlier quoted context omitted.

> One day you might want to rewrite some of your service/s, split them into microservice/s, etc. At that point, there will be a minimum of two services talking to your datastore. You should not do this. It removes almost all of the benefits of extracting things into a separate service (services should own their data and the only means of accessing it should be via their APIs). That's not utopian; that's one of the ma…

Right, so let’s suppose you already segmented the data to two different backing datastores, and your monolith is now connecting to both of them instead of just the one. Now you can do the service migration, at which point you still run into the situation I’m discussing.

Cutovers are hard, to be sure. Ideally they should also be short (the time time a service undergoing mitosis spends talking to the old and new locations should be measured in days or hours or less).

Don't choose general data access patterns for the infrequent occurrence of cutover. Cutover is when you break a few rules and then immediately stop doing so. Build for everyday access patterns instead (which should be through the API of whatever owns the data--SQL is a powerful language and a really shitty API).

Re: What ORMs have taught me: just learn SQL (2014)

#583

This was my position for a while. ORMs introduce a layer of magic which obscures what's actually going on under the hood. I decided I would just make raw SQL queries and handle mapping data explicitly. I quickly ended up with a lot of duplicated code. So then I thought, "Well ok, I should add a bit of abstraction on top of this..." I started coding some simple functions to help map the tabular data to objects. One th…

I use jdbi.org in Java all the time because it does just that for me.

Re: What ORMs have taught me: just learn SQL (2014)

#584
It amazes me how much consensus there is now about ORMs being an anti-pattern. I had figured this out over a decade ago; back then ORMs were so common and popular that even the most senior developers thought I was a fool.

There was literally no choice; I had to work for several companies which were using ORMs. I dealt with the reality of the industry by specializing on the front end.

Those were the dark ages for back end development.

Nowadays front end development has taken the lead when it comes to insanity; React, Babel, GraphQL, Webpack, CoffeeScript and TypeScript... All tools/frameworks which add negative long term value.

I switched back again to back end development a few years ago to escape the front end madness... But now back end development is also starting to degrade; pure functional programming and languages which transpile-to-JavaScript (to run on Node.js) are the latest diseases. Thankfully back end development is more fragmented and there is more room for different tools.

If this continues, I will have to give up on software development and switch to consulting; I will be forced to sell complex (but popular) tools which create problems for businesses and then offer to sell them a solution to solve some of the problems which I created. It's not a joke though, business people really are becoming THAT retarded.

Re: What ORMs have taught me: just learn SQL (2014)

#585
post #378

Earlier quoted context omitted.

The simple solution to 1 is to never allow direct database access. Api only.

So you do all your analytics by running a series of service calls and then writing a script to collate them into the needed results? Seriously?

I'm not the GP, but yes, absolutely. There are plenty of things that make this less than awful:

- The existence of tools that allow structured access to multiple APIs (GraphQL is a nice middle ground between "YOLO any queries you want" and "you only get row-by-row access exposed by the web APIs").

- The existence of data on multiple internal data stores. Analytics folks usually are not prepared to engage with the complexity of data being stored across handfuls or more of different stores with different schemas. The owner of the application knows how to join that stuff better than they do.

- Building intermediate/denormalized stores isn't frowned upon just because analytics shouldn't run ad hoc queries on the main production DBs. Expose change streams or bulk ("too much" data) endpoints and make it easy to load their results into a reporting system, which can be raw SQL. It's not redundant; if you don't do this, the following conversation starts to happen often: Q: "I'm running raw analytics queries on production and it's not quite working, can we just make $substantial_schema_change so my report works/is fast?" A: "No, we explicitly chose not to structure the DB/index/whatever like that because it seriously fucks up a real user access pattern."

Re: What ORMs have taught me: just learn SQL (2014)

#586

Earlier quoted context omitted.

Hah... I’m probably falling for Poe’s law here, but anyways... there are certainly cases where in-house is better than external dependency - specifically when your team knows the tech domain better than anyone external can... but in general well-maintained (preferably open source with a community, or a well funded company) external dependencies are almost always better. They usually would have the years of fixing edg…

They’re also tailoring their solution to be as generic as possible. Having written OSS and also having written enterprise applications, it seems plainly obvious to me why a homegrown solution is preferred. Code developed internally is understood by the team (you may not understand the underlying implementation of a dependency), and can be tailored exactly to suit your needs (ignoring edge cases that aren’t relevant,…

To counter your arguments, I'm going to use a couple of typical examples of when an in-house versus open-source / external debate comes up. I'm not counting the infamous "leftpad" cases, those are usually trivial, and really don't matter in the grand scheme of things. If it's a one-liner, just implement it yourself.

1. A high level database or queue lib, or a custom / powerful serialization lib or, relevant to this topic) an ORM or other foundational/low-level part of your tech stack.

What you can expect to happen is a bunch of very good programmers early on build powerful abstractions using macros, metaprogramming, advanced type system concepts and build up a codebase adding up to a few thousands of lines. It just works, it's a good system - a few bugs are patched by the team every month, but that's fine. Fast forward a few years, the programmers have moved on, "onboarded" the rest of the team to the codebase during their respective last week, but given how complicated the codebase is no one is really capable of debugging it and fixing issues. And given that it's not open-source, it never got an opportunity to build a community of contributors. Your team is now SOL, and it's going to take _months_ to replace it with a more well-maintained open-source solution.

2. Building a A/B testing implementation in-house - again a couple of good programmers build a working, scalable, basic system in a weekend. It actually works and the code is good, simple, readable and well-tested. But then, your PM team or your Marketing team wants you do add graphs. Then export the data to RedShift. And then tweak the algorithms powering the backend. Then multi-arm bandit. And so on. Now, what was now a weekend project, turns into months of work - whereas there exist paid services that do this for you.

Sometimes, it's unavoidable, external alternatives are genuinely not good*. But I strongly think, you have to be very, very careful about building systems in-house when they are not your business.

> I don’t mean to sound crass but how on earth could you think this is an example of Poe’s law.

Sorry for this. I do feel quite strongly against your original comment (at least the way it was written without context), and I think it's the _opposite_ of being a "responsible developer" in all but edge cases, and think you are wrong. But calling it an example of Poe's law was not right on my part, and was harsh.

> But if you’re going to be working with that application for years and years to come then I highly recommend trying to write your own code instead of relying on libraries.

I've done this, and have done both in-house and oss code, but in-house, very reluctantly - for example - when there's just one or two maintainers committing code, and there's no alternative. But even then, I have usually forked the code and used that or parts of that as the base, rather than starting from scratch

Re: What ORMs have taught me: just learn SQL (2014)

#587

Earlier quoted context omitted.

> You can mix and match perfectly fine. Thank you! This seems like one of those topics where people often feel the need to pick a side for some reason. I've often heard criticism to the effect of, "people only use ORMs because they don't know SQL. Learn SQL!" It's seemingly impossible to convince these people that ORMs are fantastic for reducing boilerplate code and they can coexist right next to raw SQL for problems…

This. Earlier in my career I made a point to deep dive into SQL. Long story short: I realized that there are a lot of very good reasons to limit the amount of raw SQL in your application that have nothing to do with familiarity. SQL is just a bad language, and it’s unfortunate that we’re still stuck with it, basically unchanged, decades after its introduction. For me, it’s almost as anachronistic as COBOL.

YMMV, but for me it feels like magic every time I use it. I can get things done in declarative way that would take a while in any programming language I know (and I'm a Clojure guy, so I value simplicity and conciseness).

Re: What ORMs have taught me: just learn SQL (2014)

#588

Earlier quoted context omitted.

ORMs make the simple things simple, and the complicated things impossible.

ORMs let you drop into SQL whenever you need, usually in a way that is fully compatible with the model, so that's entirely false.

Hibernate promises that you use the database "as if" it was not there. You can drop into SQL whenever you need, but you cannot remove all the stuff you had to add just to make it work (saving, reloading, handling caching....). So instead of being "out of the way" it poisons your entire code-base.

Re: What ORMs have taught me: just learn SQL (2014)

#589
post #467
post #439

Earlier quoted context omitted.

There is a big difference between just writing helper functions to construct SQL and convert data types, and OO-style magical auto-persisted objects. The latter is what I don't like about ORMs but the former is fine. I feel that this is an important distinction to make. As an example, the sqlalchemy docs[0] make this very clear: there's an ORM, but there's also just a core expression library that simply helps you con…

Agreed. Helpers (and indeed types) can make working with SQL an actual pleasure. You do need to learn the SQL, though. (My TypeScript/Postgres solution, in this vein: https://github.com/jawj/mostly-ormless/blob/master/README.md ).

I worked for a bit on a code gen based typescript postgres builder, but haven't had time lately to build it out - https://github.com/Sammons/morbid

I really think typescript would benefit from a good solution to this.

Re: What ORMs have taught me: just learn SQL (2014)

#590
post #244
post #101

Earlier quoted context omitted.

Running raw user SQL isn't a prerequisite of an ORM needed to make it an "ORM", it's a useful feature that most ORMs try to include because the authors recognize the many shortcomings. Also, by writing raw engine-specific SQL, you automatically invalidate one of ORMs biggest selling points which is being SQL-database agnostic. And by "drop into", this typically means writing custom stitching code that stitches the SQ…

The PHP ORM Doctrine supports a custom SQL like language called DQL that integrates your defined model/relationships as well as converting it to the correct SQL dialect for your backing store. Queries look like: SELECT u FROM ForumUser u WHERE (u.username = :name OR u.username = :name2) AND u.id = :id Where ForumUser is your model. https://www.doctrine-project.org/projects/doctrine-orm/en/2....

If I remember right, it's hugely lacking in ability to interact with more complex data types in e.g. postgres, like jsonb
Post reply on HN