Earlier quoted context omitted.
Is LINQ actually an ORM? It looks like a DSL to do relational stuff in C#, but I don't see where the 'O' part of ORM fits in especially not in your examples to show off LINQ's power and convenience.
LINQ is not an ORM. My examples were showing easy, lazy evaluation. My C# is rusty, but the examples mostly hold I think. The following gets executed on iteration (of expensiveOrders), the lambda is compiled into a Function and run on each item in orders: IEnumerable orders = ...; const expensiveOrders = orders.Where(o => o.total > 100) The following gets compiled into an expression tree, which an ORM can analyze and…
What ORMs have taught me: just learn SQL (2014)
81–90 of 360 posts
Re: What ORMs have taught me: just learn SQL (2014)
#82Earlier quoted context omitted.
Is LINQ actually an ORM? It looks like a DSL to do relational stuff in C#, but I don't see where the 'O' part of ORM fits in especially not in your examples to show off LINQ's power and convenience.
LINQ is not an ORM. My examples were showing easy, lazy evaluation. My C# is rusty, but the examples mostly hold I think. The following gets executed on iteration (of expensiveOrders), the lambda is compiled into a Function and run on each item in orders: IEnumerable orders = ...; const expensiveOrders = orders.Where(o => o.total > 100) The following gets compiled into an expression tree, which an ORM can analyze and…
(Logic programming might also be workable?)
Re: What ORMs have taught me: just learn SQL (2014)
#83On the other side you have juniors, just starting development and working with an ORM/ODM the first time. It's not uncommon for them to spend years in development while learning very little SQL at all, hardly understanding the database (apart from what they learn in school). I don't blame them. I hardly know TCP/IP at all and still I write technology for it everyday. But ORMs are a more leaky abstraction; a lot of complexity is made easier by just switching to SQL.
More principal points:
- Why do we want to abstract away further from the most important part of the business: data. Data usually outlives the applications built on top of it.
- Best practice is to keep backend API's stateless and requests short-lived. ORM's promote the use of state. In client-side applications state is a much more interesting long-lived thing, but a backend API these days? It has actually become a lot simpler there since I've started doing development; few modern backend servers render views these days.
I understand why the author (re-)embraced stored procedures but I never will. Blame the overzealous PL/SQL Oracle seniors I've met, god forbid any human bestows such complexity on it's fellow colleagues. It's also hard to automate tests for them and promote it through DTAP alongside an application, hence I keep putting all logic in the application.
I can imagine if the language and ecosystem are very strongly geared towards ORM you shouldn't try to do anything else. That's just painful. But in most languages other than Java and C# I'd make a good consideration if you really need/want an ORM (coming from Java, I never took non ORM design seriously there, but perhaps I shouldn't have).
Not sure where I'll stand in a few years time on this… but for now just happy with plain old SQL and query builders.
Re: What ORMs have taught me: just learn SQL (2014)
#84Ah yes - the proverbial "ORMs are bad, just learn SQL" post. This is analogous to saying "don't use frameworks". Sound ridiculous? Yes, yes it is. The law of leaky abstractions applies to many, many things, ORMs included. I would also argue they apply in different degrees, usually related to the design of the ORM (the post mentions SQLAlchemy vs. Hibernate, for example). But consider the following: 1) Why do people s…
ORMs are bad, because objects aren't particularly useful to deal with most data. And when you've already been through the effort to make your data relational for the database, might as well re-use that effort. That's not to say that SQL is the answer. Proper first class support for relations in your programming language / libraries is great. As for using a DSL, Datalog is worth a look.
Objects are containers of data; that's a crazy statement to make. The relational structure maps pretty cleanly to objects, properties, and collections.
However, objects are not good for reporting. And the author mentioned doing 14 joins and hundreds of columns - that smells like a reporting query.
Re: What ORMs have taught me: just learn SQL (2014)
#85Earlier quoted context omitted.
For 3: the proper abstraction for relational data is, surprise, a relation. I used to work in an environment where we had relations as full first class data structures. They are very pleasant to work with. We actually had 'relation-object-mappers', ie when we had to interact with other systems that didn't use relations, we often mapped them to relations internally to make them play nicer.
How were those relations represented? If I understand correctly, you were not just wrapping tables in a database, but using some other backing implementation. What were the most common operations, and what was the performance like?
It wasn't about speed of execution, but expressiveness when coding. Later on they even added proper type system support.
Common operations were things like map/project, extend, filter, join, collect-by-key / expand, etc.
Just as Codd pointed out in his original papers, relations allow you to not have to make a choice about a hierarchy for your data.
Using key-value store like a hash-table in your program, or the much vaunted has-a relationship between objects would force you to make these choices. Thus making interacting with the data awkward for all but one access pattern.
Relations work best when your program is written in a style that deals largely in immutable data. (What we call 'purely functional', but people in dysfunctional languages have also picked up on the advantages recently.)
Re: What ORMs have taught me: just learn SQL (2014)
#86> If you're using an RDBMS, bite the bullet and learn SQL. If this person spent all that time using Hibernate and then SQLAlchemy, and all that time did not know SQL, then their suffering and bad experiences make complete sense. You absolutely need to know SQL if you're going to use an ORM effectively. Good ORMs are there to automate the repetitive tasks of composing largely boilerplate DML statements, facilitating q…
Re: What ORMs have taught me: just learn SQL (2014)
#87- Are maintainable by a team. "Oh, because that seemed faster at the time."
- Are unit tested: eventually we end up creating at least structs or objects anyway, and then that needs to be the same everywhere, and then the abstraction is wrong because "everything should just be functional like SQL" until we need to decide what you called "the_initializer2".
- Can make it very easy to create maintainable test fixtures which raise exceptions when the schema has changed but the test data hasn't.
- Prevent SQL injection errors by consistently parametrizing queries and appropriately quoting for the target SQL dialect. (One of the Top 25 most frequent vulnerabilities). This is especially important because most apps GRANT both UPDATE and DELETE; if not CREATE TABLE and DROP TABLE to the sole app account.
- Make it much easier to port to a new database; or run tests with SQLite. With raw SQL, you need the table schema in your head and either comprehensive test coverage or to review every single query (and the whole function preceding db.execute(str, *params))
- May be the performance bottleneck for certain queries; which you can identify with code profiling and selectively rewrite by hand if adding an index and hinting a join or lazifying a relation aren't feasible with the non-SQLAlchemy ORM that you must use.
- Should provide a way to generate the query at dev or compile-time.
- Should make it easy to DESCRIBE the query plans that code profiling indicates are worth hand-optimizing (learning SQL is sometimes not the same as learning how a particular database plans a query over tables without indexes)
- Make managing db migrations pretty easy.
- SQLAlchemy really is great. SQLAlchemy has eager loading to solve the N+1 query problem. Django is often more than adequate; and has had prefetch_related() to solve the N+1 query problem since 1.4. Both have an easy way to execute raw queries (that all need to be reviewed for migrations). Both are much better at paging without allocating a ton of RAM for objects and object attributes that are irrelevant now.
- Make denormalizing things from a transactional database with referential integrity into JSON really easy; which webapps and APIs very often need to do.
Is there a good JS ORM? Maybe in TypeScript?
Re: What ORMs have taught me: just learn SQL (2014)
#88.NET has particularly nice support for developing typed ORM's by utilizing typed Expressions which lets you parse the syntax tree of the expression (instead of executing it) so you can generate the appropriate SQL that matches the intent of the expression. You can check out a live example of what that looks like for C# in:
http://gistlyn.com/?gist=84129042921da413661c96545a63e541&co...
Although the development experience is more productive using the rich intelli-sense inside any C# IDE. It's not just the Type Safety and producitivy that typed APIs offer, ORM's also provide built-in conventions for converting RDBMS types into the most appropriate language data type and their typed abstractions take care of generating the appropriate RDBMS specific SQL for each supported RDBMS.
A lot of the stigma of using ORMs is from "Heavy ORMs" which constantly fight the leaky abstraction of mapping a Relational Data Model into a Hierarchical Object Model which I've never seen an implementation I've liked, they're always inefficient and expose APIs that make it difficult to know what SQL is generated or have any ideas which APIs perform hidden perf-killing N+1 queries behind the scenes. Many Heavy ORMs want to maitain entire control over the source code used to interface with your RDBMS. They should be separated from "Micro ORMs" which are loosely coupled so it only needs a DB connection a Type definition that matches the RDBMS table or schema that's returned where they provide a clear 1:1 typed mapping of an RDBMS table to your programming languages Type.
The Types provide a contract your app logic can bind to and given they can map to clean disconnected POCOs/POJOs they can be reused to develop declarative, safe, typed Web Services that can be inferred from the Type's schema saving you the effort from having to implement it: http://docs.servicestack.net/autoquery-rdbms as well as automatically generating the UI to query it: https://github.com/ServiceStack/Admin
Disclaimer: I've developed the above.
If your ORM is causing you friction by all drop down to custom SQL, but don't use Stored Procedures unless you've identified situations where they provide clear benefits over their trade-offs. They're essentially free text commands without the support or capability of a proper programming language that splits your logic from your system making it harder to reason about it in isolation that doesn't benefit from the investments around maintaining source code, e.g. development environments, source control, CI, static analysis & compiler feedback, fast unit testing, REPLs, etc.
I wouldn't recommend using an ORM to save you from learning SQL, but rather to leverage ORM's to save the effort and boilerplate from interfacing your programming language with your RDBMS and provides an "in code" contract representation of your RDBMS Tables that your App logic can bind to.
Re: What ORMs have taught me: just learn SQL (2014)
#89> If you're using an RDBMS, bite the bullet and learn SQL. If this person spent all that time using Hibernate and then SQLAlchemy, and all that time did not know SQL, then their suffering and bad experiences make complete sense. You absolutely need to know SQL if you're going to use an ORM effectively. Good ORMs are there to automate the repetitive tasks of composing largely boilerplate DML statements, facilitating q…
cause I think its relevant -- note he's the creator of SQLAlchemy.
EDIT: Sorry, my mistake. The partent is talking about the parent comment, not the FTA
Re: What ORMs have taught me: just learn SQL (2014)
#90Earlier quoted context omitted.
cause I think its relevant -- note he's the creator of SQLAlchemy.
Wikipedia says SQLAlchemy was created by somebody named Michael Bayer : https://en.wikipedia.org/wiki/SQLAlchemy EDIT: Sorry, my mistake. The partent is talking about the parent comment, not the FTA