Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to make websites that will require lots of your time and energy

blog.jim-nielsen.com

Post reply on HN
Print view View original

111–120 of 247 posts

Re: How to make websites that will require lots of your time and energy

#111

Post by npteljes »

superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

This fully matches my experience, and my conclusions as well. I'd add that I often don't get to pick whether the logic will be more on the ORM side, or on the DB side. I end up not caring either - just pick a side. Either the DB be dumb and the code be smart, or the other way around. I don't like it when both are trying to be smart - that's just extra work, and usually one of them fighting the other.

Re: How to make websites that will require lots of your time and energy

#112

Post by sethammons »

Scarblac wrote: post #62
superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

I always see this sentiment here but I just havent experienced any of it in 14 years with the Django ORM.

My life is this in django. Querysets have been passed around everywhere and we've grown to 50 teams. Now, faced with ever slower dev velocity due to intertwined logic, and reduced system performance with often wildly non performant data access patterns, we have spent two years trying to untangle our knot of data access, leading to a six month push requiring disruption to 80% of team's roadmaps to refactor to get the ORM objects not passed around, but to use plain types or DTOs, which will only then allow us to migrate a core part of our database which is required for both product development and scaling needs.

Here's the thing. In five of six companies I have worked at, this story is the exact same. Python, Ruby, Elixir. Passing around ORM objects and getting boundaries mixed leading to more interdependencies and slower velocities and poor performance until a huge push is required to fix it all.

Querysets within a domain seems fine, but when you grow, domains get redefined. Defining good boundaries is important. And requires effort to maintain.

Re: How to make websites that will require lots of your time and energy

#113

Post by mattmanser »

Tade0 wrote: post #55
superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

A couple of years ago I had an opportunity to fill a fullstack role for the first time in several years. First thing I noticed was that I couldn't roll an SQL statement by hand even though I had a distinct memory of being able to do so in the past. I went with an ORM and eventually regretted it because it caused insurmountable performance issues. And that, to me, is the definition of a senior engineer: someone who re…

ORMs are absolutely fantastic at getting rid of the need for CRUD queries and then boilerplate code for translating a result set to a POCO and vide versa. They also allow you to essentially have a strongly typed database definition. It allows you to trivialise db migrations and versioning, though you must learn the idiosyncrasies.

What they are not for is crafting high performance query code.

It literally cannot result in insurmountable performance issues if you use it for CRUD. It's impossible because the resulting SQL is virtually identical to what you'd write natively.

If you try to create complex queries with ORMs then yes, you're in for a world of hurt and only have yourself to blame.

I don't really understand people who still write basic INSERT statements. To me, it's a complete waste of time and money. And why would you write such basic, fiddly, code yourself? It's a nightmare to maintain that sort of code too whenever you add more properties.

Re: How to make websites that will require lots of your time and energy

#114

Post by FridgeSeal »

superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

But think of how much time you’ll save needing to map entities to tables!!!! Better to reinvest that time trying to make the ORM do a worse job, automatically instead!!

Re: How to make websites that will require lots of your time and energy

#115

Post by skinkestek »

superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

People love to rant about ORMs.

But as someone who writes both raw SQL and uses ORMs regularly, I treat a business project that doesn’t use an ORM as a bit of a red flag.

Here’s what I often see in those setups (sometimes just one or two, but usually at least one):

- SQL queries strung together with user-controllable variables — wide open to SQL injection. (Not even surprised anymore when form fields go straight into the query.)

- No clear separation of concerns — data access logic scattered everywhere like confetti.

- Some homegrown “SQL helper” that saves you from writing SELECT *, but now makes it a puzzle to reconstruct a basic query in a database

- Bonus points if the half-baked data access layer is buried under layers of “magic” and is next to impossible to find.

In short: I’m not anti-SQL, but I am vary of people who think they need hand-write everything in every application including small ones with a 5 - 50 simultaneous users.

Re: How to make websites that will require lots of your time and energy

#116

Post by FridgeSeal »

mattmanser wrote: post #113
Tade0 wrote: post #55

Earlier quoted context omitted.

A couple of years ago I had an opportunity to fill a fullstack role for the first time in several years. First thing I noticed was that I couldn't roll an SQL statement by hand even though I had a distinct memory of being able to do so in the past. I went with an ORM and eventually regretted it because it caused insurmountable performance issues. And that, to me, is the definition of a senior engineer: someone who re…

ORMs are absolutely fantastic at getting rid of the need for CRUD queries and then boilerplate code for translating a result set to a POCO and vide versa. They also allow you to essentially have a strongly typed database definition. It allows you to trivialise db migrations and versioning, though you must learn the idiosyncrasies. What they are not for is crafting high performance query code. It literally cannot resu…

Plenty of tools out here doing plain sql migrations with zero issues.

At my day job everyone gave up on attempting to use the awkward ORM dsl to do migrations and just writes the sql. It’s easier, and faster, and about a dozen times clearer.

> I don't really understand people who still write basic INSERT statements

Because it’s literally 1 minute, and it’s refreshingly simple. It’s like a little treat! An after dinner mint!

I jest, I’m not out here hand rolling all my stuff. I do often have semi-involved table designs that uphold quite a few constraints and “plain inserts” aren’t super common. Doing it in sql is only marginally more complex than the plain-inserts, but doing them with the ORM was nightmarish.

Re: How to make websites that will require lots of your time and energy

#117

Post by vimto »

superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

This is a common sentiment because so many people use ORMs, and because people are using them so often they take the upsides for granted and emphasise the negatives.

I've worked with devs who hated on ORMs for performance issues and opted for custom queries that in time became just as much a maintenance and performance burden as the ORM code they replaced. My suspicion is the issues, like with most tools, are a case of devs not taking the time to understand the limits and inner workings of what they're using.

Re: How to make websites that will require lots of your time and energy

#118

Post by cluckindan »

superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

There is a solution engineered specifically for avoiding N+1 queries and overfetching: GraphQL.

More specifically a GraphQL-native columnar database such as Dgraph, which can leverage the query to optimize fetching and joining.

Or, you could simply use a CRUD model 1:1 with your database schema and optimize top-level resolvers yourself where actually needed.

Prisma can also work, but is more susceptible to N+1 if the db adapter layer does separate queries instead of joining.

Re: How to make websites that will require lots of your time and energy

#119

Post by tetha »

skinkestek wrote: post #115
superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

People love to rant about ORMs. But as someone who writes both raw SQL and uses ORMs regularly, I treat a business project that doesn’t use an ORM as a bit of a red flag. Here’s what I often see in those setups (sometimes just one or two, but usually at least one): - SQL queries strung together with user-controllable variables — wide open to SQL injection. (Not even surprised anymore when form fields go straight into…

I'd say, pure SQL gives you a higher performance ceiling and a lower performance and security floor. It's one of these features / design decisions that require diligence and discipline to use well. Which usually does not scale well beyond small team sizes.

Personally, from the database-ops side, I know how to read quite a few ORMs by now and what queries they result in. I'd rather point out a missing annotation in some Spring Data Repository or suggest a better access pattern (because I've seen a lot of those, and how those are fixed) than dig through what you describe.

Re: How to make websites that will require lots of your time and energy

#120

Post by strken »

skinkestek wrote: post #115
superasn wrote: post #38

Always use ORMs and then spend the next year debugging N+1 queries, bloated joins, and mysterious performance issues that only show up in prod. Migrations randomly fail, schema changes are a nightmare, and your team forgets how SQL works. ORMs promise to abstract the database but end up being just another layer you have to fight when things go wrong.

People love to rant about ORMs. But as someone who writes both raw SQL and uses ORMs regularly, I treat a business project that doesn’t use an ORM as a bit of a red flag. Here’s what I often see in those setups (sometimes just one or two, but usually at least one): - SQL queries strung together with user-controllable variables — wide open to SQL injection. (Not even surprised anymore when form fields go straight into…

I'm wary of people who are against query builders in addition to ORMs. I don't think it's possible to build complicated search (multiple joins, searching by aggregates, chaining conditions together) without a query builder of some sort, whether it's homegrown or imported. Better to pull in a tool when it's needed than to leave your junior devs blindly mashing SQL together by hand.

On the other hand, I agree that mapping SQL results to instances of shared models is not always desirable. Why do you need to load a whole user object when you want to display someone's initials and/or profile picture? And if you're not loading the whole thing, then why should this limited data be an instance of a class with methods that let you send a password reset email or request a GDPR deletion?

Post reply on HN