Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I learned after getting users

basementcommunity.bearblog.dev

Post reply on HN
Print view View original

71–80 of 155 posts

Re: Things I learned after getting users

#71

Post by akprasad »

travisjungroth wrote: post #69
tetha wrote: post #50

Earlier quoted context omitted.

Sorry to butt heads there, but what ORM does automatically handle schema changes? I have so many teams with highly abstracted ORMs telling me that no-downtime schema changes are impossible, no matter how trivial the changes are, or would be in e.g. Hibernate. And the only team capable of zero-downtime schema changes uses a minimal DSL to SQL lib.

There are two parts to the schema change, the table changes (migrations) and the code changes. I think the person you were replying to was asking about the code part. Every ORM I've used or looked at[0] does that automatically in the sense that you make a change to a model and all queries generated by that model will now have that change. Depending on the change they could break, but the point is the changes propagat…

^yes, that's indeed what I had in mind, and you've articulated the position very clearly.

Re: Things I learned after getting users

#73

Post by OJFord »

hermitcrab wrote: post #8

>listen to your users. they might have better ideas than you! So true. My products have improved greatly from listening to (some!) user feedback.

And if you're going to do it, make it easy. Many sites have missed out on my thoughtful (not to say I'm 'right', but that even if they don't want to do it it's considered and reasoned) feedback simply because the process although offered requires a login (i.e. sign-up) or because the last time I tried I got some dumb email back thanking me for my question it's not really possible but I can work around it by (x y z that I already said would be easier if w) and do get in touch if I have any more (sic) questions.

If you're going to solicit feedback, just dump it in an ideas bucket, no need to reply, certainly don't funnel it through the support channel for bugs/questions.

Re: Things I learned after getting users

#74

Post by eyelidlessness »

jw1224 wrote: post #44
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

> I'm talking differences of a thousand fold in query load depending on how one expresses the ORM calls That doesn’t sound like ORM… More like an N+1 problem. Eager-loading makes N+1 more likely with ORMs, but it’s easy to avoid when you know what to look for. ORMs are designed to reduce querying, not increase it a thousand-fold :)

N+1 is a common problem with many ORMs, and a classic example of the object-relation impedance mismatch.

Even the notion of eager vs implied-lazy loading suggests N+1, just spread out over time. Granted that might be optimal for a whole lot of use cases! But it’s definitely not optimal for a use case where you need a join upfront and your ORM does it in memory.

Also granted many ORMs can handle this in a lot of general cases if you know how to use them, and know their limitations. But they’re inherently highly dynamic, and inevitably deoptimize for some cases where just querying the database directly will be much more effective. That’s not even an admonishment to “learn SQL” as is the common retort, it’s just the generic “abstractions are leaky and sometimes it’s better to bail out a layer or more”.

Re: Things I learned after getting users

#75

Post by superb-owl »

dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

ORMs are great for avoiding boilerplate and adding some typesafety. But once you start doing complicated things like JOINs, it's better to drop into raw SQL.

> Listen to this old man's advise: learn SQL properly. It's not that hard.

I couldn't agree more with the first half, or disagree more with the second :) Once you're operating at scale, it takes a lot of fine-tuning, know-how, experimentation, and reading docs to get things moving efficiently.

Re: Things I learned after getting users

#76

Post by eyelidlessness »

akprasad wrote: post #42
dahwolf wrote: post #30

"this is mostly because i relied on a SQL ORM which in short is a tool that makes writing SQL easier to pick up and faster to develop. the biggest downside is that it might execute 50 queries to your database to get a list of information, when it probably only needs 1, which will cause slowdown." I appreciate this honesty. Listen to this old man's advise: learn SQL properly. It's not that hard. Focus on it for a few…

If writing SQL directly, what process do you use to update your queries during schema changes? Do you rely on a test suite to catch errors then update queries by hand? Are you using compile-time checks through libraries like sqlx [1]? [1]: https://github.com/launchbadge/sqlx

Finally something that bucks my instinct that integration tests are a waste of time. If I’m deploying a SQL query, everything that gets in or out gets thorough automated testing. Regardless of your attitude on the subject, directly interfacing a database is business logic. Test it like you would anything else that interacts with an external dependency. If you made a schema change and it broke something, you’ll either get test failures, or learn where your coverage is incomplete, or learn where you don’t care about a particular failure mode.

Normally I’d prefer testing more isolated units, but if it’s your own data access, just write the integration tests. They’re at least implicitly part of your “unit” anyway.

Re: Things I learned after getting users

#77

Post by wahnfrieden »

econnors wrote: post #4

> when the site first got a surge of users from hacker news, there was one poster in particular who came to the site, registered a bunch of offensive, racist usernames and proceeded to post and create threads that were just full of dumb slurs. this was definitely a learning experience because i had to act quickly, so i tried a bunch of different methods to get rid of him. it's sad that people like this exist in the w…

i'm more sad to see founders repeatedly put out ugc-oriented software or communities without any design priority for safety.

safety is the hard part of ugc products but is left to figure out after scale and ossification.

Re: Things I learned after getting users

#78

Post by notfed »

jiayo wrote: post #54
droobles wrote: post #47

Earlier quoted context omitted.

What do you think of "micro ORMs" like knex.js, or Dapper for C#? I personally agree with you and my personal projects just use stored procedures, but these micro orms have always piqued my interest since they're just a SQL abstraction for building single queries.

I would consider those "query builders" rather than fully fledged ORMs.

Yeah no. A micro ORM still expects you to write the full, raw SQL.

The raison d'etre for a micro ORM is to capture the results of that SQL query into a simple, flat collection of objects in a way that has a better UX than your framework's default behavior.

Micro ORMs are the perfect middle ground for debates like these.

Re: Things I learned after getting users

#79

Post by KronisLV »

lmm wrote: post #62
jjoonathan wrote: post #53

Earlier quoted context omitted.

> bizarre aversion to applying the same learning effort to their ORM Oh, sure, instead of just learning SQL I'll just learn SQL, the ORM, the ORM's weird edge case features that actually support the SQL I want, the undocumented ORM internals that prevent the good query from actually being generated, and then I'll commit a patch to the open source project to fix the undocumented internals and shepherd a custom depende…

> the ORM, the ORM's weird edge case features that actually support the SQL I want, the undocumented ORM internals that prevent the good query from actually being generated ORMs are much less bad than SQL at that, IME. If the ORM generated a particular query there is usually documentation for why, often an option you can change. If the SQL engine decided not to use the right index for this query... tough, there's lit…

> If the SQL engine decided not to use the right index for this query... tough, there's literally nothing you can do.

MySQL Index Hints: https://dev.mysql.com/doc/refman/8.0/en/index-hints.html

MariaDB Index Hints: https://mariadb.com/kb/en/use-index/

PostgreSQL: No. More information: https://stackoverflow.com/questions/309786/how-do-i-force-po...

SQL Server INDEX hint: https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-tr...

Oracle INDEX Hint: https://docs.oracle.com/cd/B19306_01/server.102/b14200/sql_e...

Of course, knowing exactly what is wrong and what hint you need to use (whether the issue is the index in particular or something else) is also a bit of work to figure out.

I recall working on a project that had Oracle as the RDBMS of choice and there was this one query that took approx. 45 minutes to execute, which wasn't acceptable. Merely a SELECT that JOINed a few tables and checked the data with some EXISTS constructions and such. If memory serves me right, looking through the AWR reports and adding a single NO_EXPAND hint made the query execute in not 45 minutes but around 3 seconds.

Of course, ORMs don't necessarily solve the aforementioned types of problems either, since it's still SQL statements being executed under the hood and figuring out how to add hints to those might be a bit more challenging. I've had cases where you need to drop down to native query level when using ORMs for particular queries, because there were no elegant ways of doing it otherwise (apart from creating a DB view and mapping against that in some cases, for read only access).

I think both using ORMs and using SQL directly suck, just in their own unique ways. That said, I wouldn't outright dismiss either: depending on what you're doing, one or the other is going to be a good enough tool for the job. You occasionally also see some pretty interesting projects in the ORM space, which is nice.

For example, jOOQ allows you to use a type safe fluent API for constructing your queries: https://www.jooq.org/

Oh and MyBatis let's you generate your own SQL dynamically, which is an interesting approach: https://mybatis.org/mybatis-3/dynamic-sql.html

Re: Things I learned after getting users

#80

Post by travisjungroth »

akprasad wrote: post #71
travisjungroth wrote: post #69

Earlier quoted context omitted.

There are two parts to the schema change, the table changes (migrations) and the code changes. I think the person you were replying to was asking about the code part. Every ORM I've used or looked at[0] does that automatically in the sense that you make a change to a model and all queries generated by that model will now have that change. Depending on the change they could break, but the point is the changes propagat…

^yes, that's indeed what I had in mind, and you've articulated the position very clearly.

Thanks, I’m glad! It’s a topic that pops up for me a lot. Related: https://news.ycombinator.com/item?id=26905647
Post reply on HN