Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wrong ways to use the databases, when the pendulum swung too far

luu.io

Post reply on HN
Print view View original

91–100 of 110 posts

Re: Wrong ways to use the databases, when the pendulum swung too far

#91

Post by TZubiri »

docandrew wrote: post #67
biorach wrote: post #41

Earlier quoted context omitted.

> I find a lot of programmers don't know how to write an array of floats to disk what does that have to do with it?

I think his point is that ORMs (and maybe DBs in general) are used for data persistence by folks who just don’t know any alternative.

Yes that is my contention.

This Simpson's clip summarizes it in a more poetic style

https://www.youtube.com/watch?v=2BT7_owW2sU

I've seen a common problem for auto-didacts is that, since the advanced and modern concepts outnumber the fundamentals, they often find themselves learning advanced concepts before the basics.

This is especially common in programming with stackoverflow or AIs where the devs look for the quickest and easiest to use solution, pushing the code and complexity beneath the rug under the dependency layer, so that their implementing code looks nice and clean.

It's hard to figure out as a begginer that the simplest and most basic solution are 10 lines of POSIX function calls, instead of three lines of "import solution" "setup solution" "use solution".

Re: Wrong ways to use the databases, when the pendulum swung too far

#92

Post by Quarrelsome »

TZubiri wrote: post #90
Quarrelsome wrote: post #78

Earlier quoted context omitted.

I know both and ORMs are fine. They save a lot of time at the cost of some inefficiency. However some of the newer ones are like Entity Framework Core for dotnet are significantly smarter than older models.

Is that time saved initially but paid later as tech debt? Or time saved on the long run?

often the debt is fine and doesn't need to be paid back. Its only tech debt when its a hotspot. For example all the ORM code that handles admin functions that the vast majority of the user base don't use.

Re: Wrong ways to use the databases, when the pendulum swung too far

#93

Post by winrid »

Quarrelsome wrote: post #79
gfody wrote: post #16

Earlier quoted context omitted.

ORMs are basically the enemy of rdbms, by encouraging antipatterns and undermining language features they have easily done more harm than good. of course reasonable people can be trusted to use them reasonably but if you're arguing about it on the internet..

ye I think the guy I was arguing with was mostly a DBA so I get his perspective, but its just religious to be wholly on one or the other side of the argument.

Even as someone who is a DBA I say ORMs are great for most simple queries.

Re: Wrong ways to use the databases, when the pendulum swung too far

#94

Post by vrighter »

roenxi wrote: post #61
bytefish wrote: post #42

Earlier quoted context omitted.

I am not going to dismiss your experience here. Stored Procedures can turn into wild monsters. Pair it with Triggers and you are in for chasing a noodle. But it's also a reality, that relational databases often become the main integration point in companies. In those environments it’s hard (next to impossible) and dangerous to use something like ORMs. Often enough I don't "own the tables" and I don't "own the columns…

How many DBAs have you dealt with that only give you SELECT and EXECUTE? That seems somewhat crazy. It doesn't cut down the surface area for bugs at all (the same amount of SQL is still required and it'd be the same code as the app was going to execute anyway as a normal query). What scenarios are they worried about here?

We only give select and execute permissions. This has saved our asses a lot of times. Do not underestimate the damage a development team with a high turnover can do.

Re: Wrong ways to use the databases, when the pendulum swung too far

#95

Post by jrochkind1 »

luuio wrote: post #86
jrochkind1 wrote: post #82

Earlier quoted context omitted.

Hm, I misunderstood, I thought the checkpoint system was also concurrency control to make sure nobody else had changed it from underneath you between read and write, since you had to read and write the whole (mega) document even though you only wanted to change a part (sub-document). Doesn't the KV provide idempotency on it's own -- so long as you're checking that no changes have happened between read and write, why…

> why wouldn't doing the same write twice produce an idempotent result you can imagine this: ``` var thing = KVStore.Get (...); if (things.Checkpoints.Contains(myUuid) == false) { thing.Counter += 1; } KVStore.Update(thing); ``` having an etag doesn't help you with retries, where we expect that `thing` could be mutated by another flow between your retries.

If the thing was mutated between your retries, then wasn't the etag changed by that mutation? So if you know the etag you started with, your conditional update on etag fails due to changed etag. So you fetch it again and start over. Which is the general optimistic locking algorithm.

i may be missing something though?

Re: Wrong ways to use the databases, when the pendulum swung too far

#96

Post by paulddraper »

_ea1k wrote: post #19
Quarrelsome wrote: post #10

Religion and engineering do not make good bedfellows. I got into a pointless argument with someone on LinkedIn who was trashing on ORMs, strawmanning them by stating they would all pull the entire data into memory to just perform a count (some ORMs are much more sophisticated than this). It was some sort of weird religious argument because the chap thought every single piece of data processing should be written in a…

Yeah, I've had almost the same experience with people and ORMs. Often they'll make a good point about some weakness of some specific ORM, but it is almost always some use case that represents 1% (or less) of their application. Worse, there's nothing stopping them from just using sql for those couple of times that they need to bypass the ORM. After all, there's no reason to be religious about it and only use one tool.

99% of the time, I see ORMs pulling every single field for every query.

One of the things I appreciate about GraphQL is that it doesn’t even offer a way to ‘SELECT *’

Re: Wrong ways to use the databases, when the pendulum swung too far

#97

Post by paulddraper »

slooonz wrote: post #68
bob1029 wrote: post #56

Earlier quoted context omitted.

> But for processing (and validating) various data loads and incoming data streams, by looking up matching values in existing reference tables, stored procedures can increase performance/efficiency and reduce complexity Performing operations on data directly in the SQL provider is the peak of human enlightenment. It takes a lot of leadership or wisdom to push a modern team away from using crap like EF to process ever…

Operational concerns trumps raw performances most of the time. Stored procedures live in a different CI/CD environment, with a different testing framework (if there’s even one), on a different deployment lifecycle, using a different language than my main code. It is also essentially an un-pinnable dependency. Too much pain for the gain. Now, give me ephemeral, per-connection procedures (call them unstored procedures…

Why do they live in separate CI?

Re: Wrong ways to use the databases, when the pendulum swung too far

#98

Post by Capricorn2481 »

zabzonk wrote: post #31
freetime2 wrote: post #27

Anecdotally, the worst codebase I ever worked on made heavy use of stored procedures. Over the years people couldn’t be bothered or were afraid to update them - which really was the root of the problem. This led to all kinds of crazy patterns in the application code like calling things in a loop where a bulk operation was needed. Or stringing together several stored procedure calls to get the desired outcome, when re…

> you probably want to have the bulk of your business logic written in C# Perhaps, if the C# applications are the only ones accessing the database. But suppose multiple applications written in different languages (and for different purposes) need the database access? You can do this via stored procs or (better in my experience) by adding an intermediate server process which the applications use, via a publicly docume…

> You can do this via stored procs or (better in my experience) by adding an intermediate server process which the applications use, via a publicly documented API, and which then talks to the the database which may provide stored proc access to the intermediate server

I don't see how that is materially different from what the OP said, you're both advocating for a single server in front of the database. We can call it intermediate or not, but the only difference is whether the API is public, so it makes sense to keep business logic in the app code.

Re: Wrong ways to use the databases, when the pendulum swung too far

#99

Post by LorenPechtel »

awesome_dude wrote: post #39
zabzonk wrote: post #31

Earlier quoted context omitted.

> you probably want to have the bulk of your business logic written in C# Perhaps, if the C# applications are the only ones accessing the database. But suppose multiple applications written in different languages (and for different purposes) need the database access? You can do this via stored procs or (better in my experience) by adding an intermediate server process which the applications use, via a publicly docume…

> Perhaps, if the C# applications are the only ones accessing the database. But suppose multiple applications written in different languages (and for different purposes) need the database access? Without wishing this to sound like a personal attack, YUCK A database that's accessed by multiple applications, regardless of the number of languages, is a really bad smell. If I have a user db, and hundreds of other applica…

The important part is what changes it.

Read my files, fine. I always try to avoid breaking changes anyway. Just don't write them!!

Re: Wrong ways to use the databases, when the pendulum swung too far

#100

Post by mike_hearn »

roenxi wrote: post #72
mickeyp wrote: post #70

Earlier quoted context omitted.

Amending data obviously. Building a dashboard using payroll data and having the power to change it are two wildly different things, and no sane large corporation would allow a dashboard's user account to change that data . You're coming at this from the idea that one user has full access to everything when that is likely the biggest security lapse you'll find in modern apps that use databases: a Web app that has full…

Yeah, that was a very confused question from me. I thought for a moment EXECUTE permissions were a roundabout way of granting restricted write access. Let me try again: how is "How am I going to work here without Stored Procedures?" supposed to be interpreted in that comment? The application could just run the queries without using stored procedures. Stored procedures don't change the permission requirements; they're…

Stored procedures can execute with different permissions to the caller, like a setuid binary. That's a common way to use them.
Post reply on HN