Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I learned after getting users

basementcommunity.bearblog.dev

Post reply on HN
Print view View original

101–110 of 155 posts

Re: Things I learned after getting users

#101

Post by brycelarkin »

ookblah wrote: post #99
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

Honestly, I started on the ORM hate train when I first started and wrote my own queries in an effort to be "performant", but my frankenstein of writing direct queries and abstracting code to be reusable became basically a super shitty ORM that was a nightmare to maintain for a large codebase lol. I don' think there's anything wrong with it and for 90% of CRUD actions it's what you should choose. By the time you hit p…

I felt the same way until I found sqlx (https://github.com/launchbadge/sqlx). The package lets you write sql in your app with compile-time checks. Was so much easier to get up and running than learning a new ORM.

Re: Things I learned after getting users

#102

Post by partly_cloudy »

layoric wrote: post #100
ookblah wrote: post #99

Earlier quoted context omitted.

Honestly, I started on the ORM hate train when I first started and wrote my own queries in an effort to be "performant", but my frankenstein of writing direct queries and abstracting code to be reusable became basically a super shitty ORM that was a nightmare to maintain for a large codebase lol. I don' think there's anything wrong with it and for 90% of CRUD actions it's what you should choose. By the time you hit p…

I am the same, the advice of "learn SQL" is still very important. Most ORMs have the escape hatch of just mapping results of a straight SQL query to an expected structure (thinking of typed languages like C#/Java here). 95% of the time I can just use the ORM to make simple queries in a easy to maintain way for your application. For the 5% of queries that are either complex or can't be easily represented via your ORM…

"95% of the time I can just use the ORM to make simple queries in a easy to maintain way for your application. For the 5% of queries that are either complex or can't be easily represented via your ORM of choice, declare the expected type response and use that escape hatch to write your performant SQL."

site creator here - yeah this is the approach i'm taking now. the ORM is useful for sure and there's still a benefit to using it, but anything that needs to read from a few different tables, i'm definitely going with raw SQL

Re: Things I learned after getting users

#103

Post by uoaei »

dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

Honestly, ORM is an unnecessary layer most of the time.

You can have database schemas, and you can have software schemas, and they don't necessarily need to coincide as long as your database schema allows efficient queries. The rest of your relational-manager-logic can be embedded in the post-processing methods you apply to the query results.

Re: Things I learned after getting users

#104

Post by selcuka »

manv1 wrote: post #97
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

It's hard to explain to developers why an ORM isn't really that great of a solution....but at a high level: * your database is the core of your app. Abstraction implies that you don't care about the details of your data store, which eventually will lead to two major problems: 1. When things are slow you won't be able to debug it, because you're abstracted away your database and don't understand what queries your ORM…

> I mean it knows what you're using to select data, so it should auto-create indexes for you, right?

Curious, how does it know what you're using to select data?

Re: Things I learned after getting users

#105

Post by monroewalker »

partly_cloudy wrote: post #83
scoofy wrote: post #70

I have a site that will likely need a denylist for usernames. Do you have any resources on implementing that? I mean, it sounds obvious how to do it, but if it's already been done, I'd rather just have a list to work from.

the denylist on the site is actually pretty quite simple. for now, it's just a list of chunks of words and the form validation will just compare the username against that list, where each chunk in the list if used as a regexp

You could also consider shadow banning these users. If you're immediately preventing them from creating accounts with certain usernames, they'll probably just get more creative with the profanity or save it for their posts.

Re: Things I learned after getting users

#107

Post by noduerme »

eyelidlessness wrote: post #74
jw1224 wrote: post #44

Earlier quoted context omitted.

> I'm talking differences of a thousand fold in query load depending on how one expresses the ORM calls That doesn’t sound like ORM… More like an N+1 problem. Eager-loading makes N+1 more likely with ORMs, but it’s easy to avoid when you know what to look for. ORMs are designed to reduce querying, not increase it a thousand-fold :)

N+1 is a common problem with many ORMs, and a classic example of the object-relation impedance mismatch. Even the notion of eager vs implied-lazy loading suggests N+1, just spread out over time. Granted that might be optimal for a whole lot of use cases! But it’s definitely not optimal for a use case where you need a join upfront and your ORM does it in memory. Also granted many ORMs can handle this in a lot of gener…

Upfront: I really dislike ORMs. They're an unnecessary abstraction and a performance headache as an extra layer of middleware to worry about keeping up to date. I'd never use one in production.

But we do always need a clean way to handle CRUD between apps and DBs, that ideally doesn't require custom calls for each and every data view. Here's the thing: Probably 90% of CRUD can be handled with generic updates and inserts. Raw JSON reads can be passed back to the client... let the client know how to cast or structure those to complex data types. The rest, that the server has to manipulate, can be cast / structured using the same classes the client uses, if in node. The actual meat of really complex reads or really optimized writes should always be exceptional and done by hand.

Re: Things I learned after getting users

#108

Post by noduerme »

xarope wrote: post #88
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

Better yet, learn stored procedures, then call them just like any API with a well established set of inputs and expected return typed outputs. You'll thank me later once you decide to split your DB from your app server.

It's been at least 12 years since I had a DB on the same server as the app or service that called it... I can't fathom why anyone would do that these days. But either way, the stored procedures would work just as well. The issue you're alluding to (I think) is the time to set up connections for a dozen calls between the backend and the DB if they're on separate servers. But that's actually less of a bottleneck now than it used to be, with persistent pooled connections and most things hosted in the same VPN zones. I'm not arguing for ORMs... but appside processing [edit: "joins"] can still be appropriate if you're dealing with a monolithic DB and don't want certain giant queries or procedures to block rapid reads. The point is more to understand your infrastructure, and ORMs obfuscate it to a point that makes you irresponsible as a developer in some cases.

Also, with lateral joins and windows in mysql 8 you can really have control over whether you prefer your execution plan to loop or scan. Which takes away most of the rest of the argument for appside processing.

Re: Things I learned after getting users

#109

Post by closeparen »

dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

I'm happy to write my own SELECT queries, but it sucks to be manually schlepping row[0], row[1], etc. into the fields of my domain entities, or making sure I provide the parameters to the parameterized INSERT statement in the right order. ORMs usually let you drop down to SQL as desired, and I do that when I have a nontrivial query to express, but CRUD plumbing is nice.

Re: Things I learned after getting users

#110

Post by BiteCode_dev »

jw1224 wrote: post #44
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

> I'm talking differences of a thousand fold in query load depending on how one expresses the ORM calls That doesn’t sound like ORM… More like an N+1 problem. Eager-loading makes N+1 more likely with ORMs, but it’s easy to avoid when you know what to look for. ORMs are designed to reduce querying, not increase it a thousand-fold :)

Right. In fact any ORM worth its salt will have methods specifically designed to let you address the N+1 problem easily.

E.G: django "select_related"

Not to mention they will will have ways to use raw SQL through the lib infrastructure, which is still better than querying SQL manually, and give you full control of the query.

Also wordpress shows that you can write manual SQL and still have terrible performances because your application is badly structured, which an ORM helps with.

In the end, I don't think there is less trap with raw SQL, the traps are just different. It's not a matter of "better".

Post reply on HN