DenoDB • Hacker News Board

Re: DenoDB

#161

Post by smt88 » Fri, Jun 18, 2021, 3:37 PM UTC

SomeCallMeTim wrote: post #156

smt88 wrote: post #140

Earlier quoted context omitted.

I've used ORMs for 20 years in Java, PHP, C#, JavaScript, TypeScript, Python, and Kotlin. I have pushed code to production with at least 10 ORMs, possibly more. ORMs are a bad idea. They are a productivity killer. Do not use them. I also used to think I'd find a "good" ORM because the early magic is so exciting and efficient. But for anything beyond a toy or proof-of-concept, they are more cost than benefit. Some of…

Wow. See this other comment I wrote: https://news.ycombinator.com/item?id=27551288 TypeORM in particular is well designed and quite awesome. A huge benefit of TypeORM is that you get full static type support in all of your queries, including in the returned results. SQL results are effectively one-off dynamic type result piles that you need to validate by hand, and are insanely inefficient to code.

> A huge benefit of TypeORM is that you get full static type support in all of your queries, including in the returned results.

When did I say I wanted to give up static typing? You can have both[1][2]. I would never give up static typing and would not have used Node if TypeScript/Flow were unavailable.

> TypeORM in particular is well designed and quite awesome.

I've used TypeORM extensively in production on multiple projects for years, including with a fairly large team.

It has silent failures, and its API was unstable/unclear for a long time. It's been a very messy project for most of its lifetime. They couldn't even decide on an API for lazy-loading of relations, and there was some hidden API that was semi-supported and not type-checked.

Partly because it's built on Node, it's very difficult to use a debugger to see the relevant frames when you hit a breakpoint or an exception -- if you can even set a breakpoint where you want it to be before just running into an error at runtime.

There's also the same core issue that all ORMs suffer from, which is that the abstraction becomes leaky as soon as you need to do anything complicated with lots of joins or advanced SQL features.

1. https://github.com/adelsz/pgtyped

2. https://jawj.github.io/zapatos/

Re: DenoDB

#162

Post by SomeCallMeTim » Fri, Jun 18, 2021, 3:41 PM UTC

pionar wrote: post #152

brainless wrote: post #148

Earlier quoted context omitted.

I am also very happy reading, writing and understanding SQL. My own side-product depends on that ability: https://dwata.com . It uses reflection of the underlying SQL, using an ORM based reflect library, generates a vector of the relations and state of schema and creates SQL on the fly. But, would I recommend a team to start writing SQL by hand: No, absolutely not.

Why not? If you're going to use a tool (be it SQL, NoSQL, docker, kubernetes), your team needs to have/build some expertise in it. IMHO, using something like an ORM abstracts that layer. It also introduces a "black box" into your ecosystem. Going by my opinion in the first paragraph above, you'd have to build some expertise in this ORM. Why spend that time when you can just spend time getting to know your data storag…

Because "getting to know your data storage technology better" isn't the business goal. On the surface, that's a great goal to have, but taking an approach that requires you to do 100x as much work so you can learn SQL better isn't software engineering. It's over-billing.

I throw together 200 lines of schema in Prisma.io and it creates 20,000 lines of GraphQL handling code plus more ORM-specific code that allows me to create type-safe access to the database.

And "the database" right now is PostgreSQL, but one of the business requirements is that "the database" could also be SQL Server or MySQL.

Having learned the salient details of the ORM in ... hmm ... about three days? ... I feel that the advantages of getting a full GraphQL API that can be wired up to an arbitrary backend database for 99% of the queries vastly outweigh the fact that most of my database queries are entirely inside a "black box."

In six months I've encountered exactly one problem where the black box made my life more difficult. I think I wasted about an hour, maybe 90 minutes, on tracking down why it didn't work as expected. Given that "writing custom SQL" for all of the queries that I've created would have required 10s of thousands of lines of additional code plus system tests to verify that the queries were all working as expected? That's a profoundly huge win.

I'd still be writing SQL queries months from now if I had taken the approach you're suggesting. Maybe it's advice that's good for consulting firms that charge by the hour and love to find ways to make their developers work more hours, but in my book it's not even ethical to recommend.

Re: DenoDB

#163

Post by evanelias » Fri, Jun 18, 2021, 3:50 PM UTC

markstos wrote: post #146

So, pick a database, use DenoDB and then use the lowest-common-denominator feature set of all the databases it supports. There's a great parable from almost two decades ago. Movable Type was a popular open source blogging platform that used an ORM and supported multiple database backends. Wordpress only supported MySQL. Remember Movable Type? I hacked on both and preferred working with Perl and PostgreSQL but the ORM…

Initial disclosure: I worked for Six Apart (makers of Movable Type) a dozen years ago, so maybe I'm still inadvertently biased, but I certainly try not to be.

Wordpress won out for a number of reasons, mostly relating to licensing (Movable Type temporarily went non-FOSS at a critical moment in time), ease of hosting (PHP vs Perl), ease of dynamic content (MT generated static pages), and better company product focus (MT's parent company tried doing too many things at once and ran out of money).

I say this as someone who doesn't love ORMs, so don't get me wrong here, but I've never seen a serious analysis that ranked MT's ORM layer anywhere as a factor in WP's dominance.

The vast majority of MT users used MySQL anyway, and MT's ORM definitely had first-class support for MySQL. In 2010, MT 5 dropped support for Postgres and SQLite entirely.

I was in Six Apart's Services org, mostly developing MT plugins for larger MT users, but also occasionally WP plugins as well. Personally I quite liked MT's ORM at least circa MT 4.x, and found that developing MT plugins was far more enjoyable than WP plugins at the time. (fwiw I was equally fluent in Perl and PHP, so that wasn't a factor.)

MT had a very powerful plugin system that, combined with Perl's ability to "monkey patch", essentially allowed you to hook into pretty much any part of the CMS or page generator that you wanted. Back when MT was popular, it had a thriving extension ecosystem that very much rivaled Wordpress's, so it is simply not accurate to say MT bombed due to a poor extension ecosystem.

Yes, eventually WP's ecosystem was massively larger, but that also simply tracks with size of PHP developer community vs size of Perl developer community over time.

Re: DenoDB

#164

Post by brainless » Fri, Jun 18, 2021, 4:00 PM UTC

pionar wrote: post #152

brainless wrote: post #148

Earlier quoted context omitted.

I am also very happy reading, writing and understanding SQL. My own side-product depends on that ability: https://dwata.com . It uses reflection of the underlying SQL, using an ORM based reflect library, generates a vector of the relations and state of schema and creates SQL on the fly. But, would I recommend a team to start writing SQL by hand: No, absolutely not.

Why not? If you're going to use a tool (be it SQL, NoSQL, docker, kubernetes), your team needs to have/build some expertise in it. IMHO, using something like an ORM abstracts that layer. It also introduces a "black box" into your ecosystem. Going by my opinion in the first paragraph above, you'd have to build some expertise in this ORM. Why spend that time when you can just spend time getting to know your data storag…

By that logic, I should also know compilers, lexers, kernel, and the whole pile of protocols that run the Internet in a great amount of details. Where do you draw the line?

We need abstractions. Sure some abstractions are leaky, others are simply not good. But that is exactly why people should try to make better ones.

Re: DenoDB

#165

Post by andrewmcwatters » Fri, Jun 18, 2021, 4:16 PM UTC

SomeCallMeTim wrote: post #162

pionar wrote: post #152

Earlier quoted context omitted.

Why not? If you're going to use a tool (be it SQL, NoSQL, docker, kubernetes), your team needs to have/build some expertise in it. IMHO, using something like an ORM abstracts that layer. It also introduces a "black box" into your ecosystem. Going by my opinion in the first paragraph above, you'd have to build some expertise in this ORM. Why spend that time when you can just spend time getting to know your data storag…

Because "getting to know your data storage technology better" isn't the business goal. On the surface, that's a great goal to have, but taking an approach that requires you to do 100x as much work so you can learn SQL better isn't software engineering. It's over-billing. I throw together 200 lines of schema in Prisma.io and it creates 20,000 lines of GraphQL handling code plus more ORM-specific code that allows me to…

I don’t necessarily agree with the approach, but I completely agree with the thinking. Thank you for sharing.

Re: DenoDB

#166

Post by 1st1 » Fri, Jun 18, 2021, 4:19 PM UTC

phplovesong wrote: post #117

deergomoo wrote: post #111

Earlier quoted context omitted.

I've been keeping one eye on EdgeDB [0], which I believe could remove a lot of the desire for ORMs. In my experience a lot of the drive behind ORM use is that SQL kinda sucks at returning anything that looks vaguely like a tree. For example, with SQL if you wanna load all a list of Users, and all those User's Posts, and the Thread that post was made in, you're either doing joins and some awkward transposition of the…

Never seen edgedb before. Ill have a look, thanks for the tip. I usually find the docs always giving a super trivial example. Like a join. Joins are the bread and butter of database work, and design. You model your data and write queries that operate on it. When it comes to real-world things, i usually do complex reports that include joins from aggregated sets on some condition. Here is where the orms fail big time.…

> Edgedb seems to be build on postgres, so does it come with the same power

Our goal with EdgeQL (the query language of EdgeDB) is to make it more powerful than SQL. There are a few things that we're still missing, but we're getting there.

What makes EdgeQL interesting is that it's functional in its nature, making it fully composable. So both simple queries and complex queries (deeply nested subqueries, aggregates, etc) work just fine.

Re: DenoDB

#167

Post by madkat » Fri, Jun 18, 2021, 4:23 PM UTC

gmac wrote: post #67

bjt wrote: post #44

Earlier quoted context omitted.

> What's the alternative? Not sure the parent comment would like it, but there's a middle ground between ORM and raw SQL that I consider a sweet spot. It's more of a "query builder" library that gives you language-appropriate constructs for building any SQL you like, but also provides more correctness guarantees than just writing raw SQL strings. SQLAlchemy's "expression" layer, for example, does this really nicely.…

For TypeScript, there’s a very helpful list here: https://phiresky.github.io/blog/2020/sql-libs-for-typescript...

Is there something like this for MongoDB with Typescript?

Re: DenoDB

#168

Post by dimitrios1 » Fri, Jun 18, 2021, 4:25 PM UTC

brainless wrote: post #134

Every time we discuss an ORM project, someone is bound to complain about ORMs in general. I have a small suggestion: please try ORMs in different languages. A lot of the power of an ORM depends on the language. Your ORM may not be the same as my ORMs - Django ORM, SQLAlchemy, Pony ORM. Try them and you will know why. I will wait. From my limited understanding, the language needs to allow reflecting and manipulating y…

Used ORMs in many different languages, from ActiveRecord, Hibernate, EntityFramework, SQLAlchemy, Bookshelf.js, GORM, and Ecto (Ecto was the best, ActiveRecord was the most "feature rich", Hibernate makes me cry, and EntityFramework is OK so long as you give Microsoft a lot of money).

For dynamic languages the downsides are increased runtime cost over simply writing and executing sql and doing the mapping yourself, and tons of magic. Magic everywhere. Magic makes things harder to debug.

For static languagges the downsides are heavy impedence mismatch, obtuse to use APIs at times (to overcome type systems), and again, a performance hit.

What I have found is for the majority of the time, people who say they want an ORM, what they really want is a type-safe (or type-hinted) way to write queries and operate on returned data. That's probably 90% of the value add. That can be done without a heavy, complex ORM framework.

Re: DenoDB

#169

Post by akudha » Fri, Jun 18, 2021, 4:26 PM UTC

brainless wrote: post #134

Every time we discuss an ORM project, someone is bound to complain about ORMs in general. I have a small suggestion: please try ORMs in different languages. A lot of the power of an ORM depends on the language. Your ORM may not be the same as my ORMs - Django ORM, SQLAlchemy, Pony ORM. Try them and you will know why. I will wait. From my limited understanding, the language needs to allow reflecting and manipulating y…

I have tried ORMs in only 3 languages, they all sucked. It is not all fault of the language though - writing SQL in SQL feels natural, but writing SQL in PHP/Go etc feels just plain weird, no matter how well the ORM libraries are defined.

I also don't enjoy writing HTML in non-HTML languages - it just doesn't feel nice or natural.

This is before getting into the other problems (having to learn the libraries, the libraries generating inefficient SQL etc).

Maybe I am old and not hip, I dunno. Using ORMs is just annoying and unpleasant

Re: DenoDB

#170

Post by throwaway894345 » Fri, Jun 18, 2021, 4:39 PM UTC

brainless wrote: post #134

Every time we discuss an ORM project, someone is bound to complain about ORMs in general. I have a small suggestion: please try ORMs in different languages. A lot of the power of an ORM depends on the language. Your ORM may not be the same as my ORMs - Django ORM, SQLAlchemy, Pony ORM. Try them and you will know why. I will wait. From my limited understanding, the language needs to allow reflecting and manipulating y…

FWIW, some people rave about SQLAlchemy and for the life in me I can't understand why. It does too much behind the scenes, the high level interface overloads a lot of operators (IIRC `==` and/or `and`/`or`) are overloaded and return objects rather than booleans, and by default all objects are truthy--this has lead to bugs at least a few times. There is the "core" which is basically just a query builder and the only part worth using as far as I can tell, but searching for things in the core always turns up results from the ORM layer. The documentation is hard to read because it's all rendered on a single page and many method names and class names are overloaded, so if you ctrl+f for a method, you have to scroll up several pages to find the class heading. Further, because it's all dynamically typed, it's very difficult to tell precisely what types a method accepts or returns.

I'm not going to use that alone to write off all ORMs--I've heard the Ruby folks like their Active Record--but it seems like if such a thing as a helpful ORM exists, they are few and far between.