[deleted]
What ORMs have taught me: just learn SQL (2014)
621–630 of 654 posts
Re: What ORMs have taught me: just learn SQL (2014)
#622Earlier quoted context omitted.
Whereas I find doing too much business logic related data manipulation not performed by the database to be an anti-pattern that creates significant risks with testing and a source of data bugs. My model of thinking is that any copy of data that isn't currently resting in the database is potentially stale; avoid round trips like the plague; get new data into the database as soon as possible. For me and the way I work,…
> My model of thinking is that any copy of data that isn't currently resting in the database is potentially stale; avoid round trips like the plague; get new data into the database as soon as possible. Nothing about an ORM stops you from persisting data as soon as it is ready or updating the state from the DB to ensure consistency (or from using transactions). > rather more often a question of whether I even want my…
>> —and they change the way you think,
> I am going to pay more attention to this
Everyone thinks about things in their own way I suppose, but perhaps a way to parse it could be to think about whether you're approaching data from a "load and store" mentality or a "truth and snapshot" mentality.
In my mind, unless you wrap the entire programming round trip in a transaction, all data sitting in variables are a snapshot of the past and thus stale by definition.
Re: What ORMs have taught me: just learn SQL (2014)
#623Earlier quoted context omitted.
So then you’re mapping to whatever data structure your app uses instead of objects. In OOP languages like Python, everything is some type of object anyway.
That’s just a semantic game. If your language returns the result of a query as a generic array of generic dictionaries (or whatever), that isn’t mapping, nor is it object oriented in principle.
But then your generic array of generic dictionaries needs to be mapped to whatever data structures make sense for your application. ORMs save you that step.
Re: What ORMs have taught me: just learn SQL (2014)
#624Using ORM if you don't know SQL is a recipe for disaster. And after you learn SQL, you will likely decide to go back to ORMs, because now you know how to use it efficiently to save your time.
Re: What ORMs have taught me: just learn SQL (2014)
#625Earlier quoted context omitted.
> It is saying "it makes easy things easier". No it makes simple things easy. Simple things in raw SQL are bloody complicated. Even just getting data, manipulating it, and saving it is at least twice as difficult in maintainability and lines of code than using an ORM. > An ORM converts one paradigm into a completely different paradigm, which is why it fails and is a terrible idea. I'm not sure where people get the id…
> Even just getting data, manipulating it, and saving it is at least twice as difficult in maintainability and lines of code than using an ORM. That has not been my own experience outside of the most trivial queries. Once any amount of complexity is introduced, I find that ORM-based queries often make it difficult to really see whats going on with indexes and locking, I can’t just paste a query (eg to use EXPLAIN) wi…
If you want to see what query is hitting the database, put a trace on the database and watch it. It's trivial to copy and view the explain there. Your difficulties aren't the fault of the ORM, it's your refusal to want to change the way you work that is the issue. Query logs work fine as well, having to look in a log is hardly something any developer should find difficult.
Re: What ORMs have taught me: just learn SQL (2014)
#626Earlier quoted context omitted.
Eventually if you’re working with an object oriented language or if you have to convert the result to JSON, you’re going to have to convert one paradigm to another. Unless you’re programming in assembly, everything you do is being converted into another paradigm that is what every compiler and interpreter does.
You can only use a fraction of the features of a SQL database when you use an ORM because they don't translate to the new paradigm. When I am using a high level programming language, it is merely helping me do things like manage memory. I am not constantly wishing I could drop down and work with pointers and so on. It is a foundational paradigm that builds on the top of the one before it. ORMs just present a differen…
> You can only use a fraction of the features of a SQL database when you use an ORM because they don't translate to the new paradigm.
Not remotely true. ORM's are great at mapping views/procs, and you can use any feature of the db you want in a view or proc. Using an ORM does not mean not being able to your database to its fullest extent.
Re: What ORMs have taught me: just learn SQL (2014)
#627Earlier quoted context omitted.
There are plenty of scenarios in certain verticals where you need to control memory management like games and others where you need to program in assembly. Garbage collection is an incompatible paradigm where you need to control when memory is allocated and freed. Back in the day when I was doing C, there were times when we just couldn’t get the speed we needed from the compiler. I wrote inline assembly. Does that me…
An ORM is not a high-level version of SQL though. A more accurate metaphor for an ORM would be like a converter from one high level programming language that is object oriented, to another high level programming language that is functional.
In the case of C#, LINQ is a declarative built in part of the language. At runtime, it is converted into an AST and if you use Entity Framework, it is translated into SQL - another declarative language.
You can theoretically express any standard SQL query in LINQ even though outer joins can be obtuse at first. The translation may not be as optimal as hand written sql, but no compiler can translate code into assembly that would be as optimized as someone who could hand roll their own. We decided decades ago that high level languages were worth the trade off most of the time.
Re: What ORMs have taught me: just learn SQL (2014)
#628Earlier quoted context omitted.
Why do I have two problems if I use a regex? "Programming Perl" and "Mastering Regular Expressions" were what drew me into programming in the early 2000s. Whilst ORMs are optional regular expressions are not so I don't get your point.
Pretty sure regex is not the only answer to parsing problems. Maybe it is for someone who has preferred using it for the past 15 years.
Ok, what's your alternative for validating an email address?
Re: What ORMs have taught me: just learn SQL (2014)
#629Earlier quoted context omitted.
> It is saying "it makes easy things easier". No it makes simple things easy. Simple things in raw SQL are bloody complicated. Even just getting data, manipulating it, and saving it is at least twice as difficult in maintainability and lines of code than using an ORM. > An ORM converts one paradigm into a completely different paradigm, which is why it fails and is a terrible idea. I'm not sure where people get the id…
> Even just getting data, manipulating it, and saving it is at least twice as difficult in maintainability and lines of code than using an ORM. That has not been my own experience outside of the most trivial queries. Once any amount of complexity is introduced, I find that ORM-based queries often make it difficult to really see whats going on with indexes and locking, I can’t just paste a query (eg to use EXPLAIN) wi…
To gnaritas (comment dead):
Things don't usually start out bad, but they become bad only after you have a more complex codebase with many users (ie many queries). Its at this point that you need to know what a query is doing, yet its at this point that the query logs are full of queries, so finding the ones I want becomes hard. Putting a trace on the queries also becomes hard when you have a large codebase where the query logic is intermingled with application logic. I mentioned this in my comment.
You say it's my "refusal to want to change the way you work that is the issue", but I never chose to work that way. All of the codebases where I've had this issue were inherited: I was not the one to decide to work like this and I did not mix the query logic into the application logic. But that was my point: I've had the same experience across multiple teams in multiple companies, so blaming the developers seems like a cop out and not much of a solution. I'll happily adapt the way I work, but I can't force existing teams to change. Maybe discipline could fix it, but I have not experienced this discipline anywhere I've worked. But sure, its my fault somehow.
Re: What ORMs have taught me: just learn SQL (2014)
#630Earlier quoted context omitted.
I'm not the GP, but yes, absolutely. There are plenty of things that make this less than awful: - The existence of tools that allow structured access to multiple APIs (GraphQL is a nice middle ground between "YOLO any queries you want" and "you only get row-by-row access exposed by the web APIs"). - The existence of data on multiple internal data stores. Analytics folks usually are not prepared to engage with the com…
Forcing analytics to go through the API doesn’t actually reduce load on the production DB, it just increases load on the API itself. Step 1 should probably be a dedicated read replica and step 2 should probably be an ETL process.
Ding ding ding. Dedicated read replica and an ETL gets you to a point where queries don't bring down prod. If you have an analyst org running wild making bad decisions about data that they think says things it doesn't -- that's probably a good sign that it's time for a dedicated data engineering team, and potentially a BI flavored data science team as well.