Live data from Hacker News

Things I wished more developers knew about databases

medium.com

321–330 of 464 posts

Re: Things I wished more developers knew about databases

#321
post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

> The most senior developer (24yo or so) I see the problem there.

Shameless age discrimination. Very smart people exist at all ages. I've seen 21yo junior grad developers outperform 45yo 'senior' developers.

Re: Things I wished more developers knew about databases

#322
post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

I've heard about some database drivers/middleware being smart and optimizing/caching queries like "SELECT 1" to save the round trip, so now I do something like "SELECT now()" or similar to do health checks.

Re: Things I wished more developers knew about databases

#323
post #31

Earlier quoted context omitted.

add to that "do not want to declare/worry about types", "cant be bothered about RAM usage/cache coherency/etc", "concurrency" etc. etc. All of those are "premature optimizations".

> All of those are "premature optimizations". And they are, until they aren't. And I really do mean that in both directions: Much of the time, the simplest, dumbest, most naive solution is 100% fine for realistic load for the forseeable future. And then some of the time it breaks terribly and you do need to spend effort optimizing it, whatever that entails.

The question rarely asked is, "Can your team go from zero to sixty on this sort of thing when this happens?"

Technically optimizing as you go might be a 'waste' of time, until it isn't, because your team can handle a serious performance problem in days instead of months (or IME, never).

Re: Things I wished more developers knew about databases

#324
post #96
post #20

Learn about modelling. Database is more than just storing data. Drink less koolaid of NoSQL, any NoSQL. It is trading initial result with future development time. SQL has been battlefield tested. No amount of "convenience" is more convenient than learning the fundamentals.

I'm disappointed that there isn't more criticism of the SQL language . The whole NoSQL buzz got me excited, then turned out to actually mean NoRelational. It is wild that we are still using a language that looks and feels like COBOL, and any criticism is met with drive-by disapproval (downvotes and no comments) or an argument about why relational databases are important. SQL is a deeply flawed language by standards t…

> Wouldn't it help to have a language for working with it that isn't outright terrible?

There is a language like that, and it's downright elegant. It's called LINQ, and it's built into the C# language. It inverts FROM and SELECT, which lets it reliably support autocomplete. LINQ was built using concepts from category theory/relational theory and is widely used in C# to make code tighter and simpler (even though some eschew it because simple for loops are faster). It also transpiles to SQL via Linq-to-SQL. It looks like this:

  var results = from c in collection
                where c.Fld1 
It's the closest thing we have to SQL that Typescript is to Javascript.

That said, most people who access databases in C# still either write SQL or use an ORM like EF or Dapper.

Databases speak SQL natively for better or for worse. And the SQL language, despite its flaws, is actually mathematically very deep because it's been developed and extended over so many years. See [2] for some serious SQL-fu. To replicate all this functionality into a new language that transpiles into performant SQL is a huge undertaking.

SQL is one of those incumbents that is extremely hard to displace, partly because it's so ubiquitous and that in practice, not everyone finds outright terrible.

True, it's not good, but it's not terrible. I write it day in and day out. The analogy I can think of is the Erlang language -- it doesn't have a pretty syntax but it solves the concurrency problem really well. SQL for better or for worse solves the complex analytic query problem really well.

[1] https://en.wikipedia.org/wiki/Language_Integrated_Query

[2] https://modern-sql.com/slides

Re: Things I wished more developers knew about databases

#325
post #280
post #261

Earlier quoted context omitted.

Just because you saw a few exception does not mean the rule does not hold in general. Or are you saying most people don't learn with time (a corollary of your theory that age is not a big factor)?

I would say that I have seen no discernible pattern, so I have learnt that it is imprudent to judge a developer by their age. People do learn over time, but some people learn more "cogently" than others, i.e. some get more out of one year's worth of learning than others get in 10.

The best thing I've seen on the topic is: "Some people have 10 years experience, some people have 1 years experience 10 times".

Re: Things I wished more developers knew about databases

#326
post #225

Earlier quoted context omitted.

This is a very interesting comment! Two questions: Do you have any example code that shows how this works? I get what you’re saying intuitively but example code will help me bring it to table. What about cross cutting concerns? I’ve found stored procedures to be a performant solution here. By version controlling them, and limiting to pure functions, I found them quite maintainable. Would you instead just define a new…

Briefly, 1. Let me try with a simple example. Suppose you have a fact table A with fields (ItemID, Item, Amt) where Amt is in USD. Rule of thumb is: don't expose A to the consumer; instead write a SQL View V_A and expose that instead: CREATE VIEW V_A AS SELECT ItemID, Item, Amt FROM A Then suppose a European counterpart wants to use the same API but needs the amounts to be in Euros. You can write another view: (in pr…

There's another step that could be added there, too: After the ALTER VIEW, V could be slowly incrementally updated over however long you need to back-populate AmtGBP, and the views will continue to just work the whole time. Once done, V_A can be simplified to remove the ISNULL and Amt, then Amt dropped from V. That way you don't get build-up of cruft over the years, and the experience isn't interrupted for the migration.

(Possibly a bad idea for currency conversion for various reasons, but just wanted to mention it since this type of migration may be just right for other data)

Re: Things I wished more developers knew about databases

#327
post #261
post #228

Earlier quoted context omitted.

I've seen older developers that call themselves senior, but lack basic knowledge. I've seen younger developers, wise well beyond their years. Age simply isn't a big factor in how you judge a developer.

Just because you saw a few exception does not mean the rule does not hold in general. Or are you saying most people don't learn with time (a corollary of your theory that age is not a big factor)?

Most people don’t magically grow wiser with time. They need to be in an environment where they can grow, otherwise they’ll emerge just as stupid as before.

Re: Things I wished more developers knew about databases

#328
post #327
post #261

Earlier quoted context omitted.

Just because you saw a few exception does not mean the rule does not hold in general. Or are you saying most people don't learn with time (a corollary of your theory that age is not a big factor)?

Most people don’t magically grow wiser with time. They need to be in an environment where they can grow, otherwise they’ll emerge just as stupid as before.

The counterpoint is you need time to grow wise, and there's no way to get wise without many years of experience.

Re: Things I wished more developers knew about databases

#329
post #276

Earlier quoted context omitted.

Go code often reinvents/reimplements a lot of things from scratch, reintroducing problems that have been addressed long ago in other systems. It's like this new trend, let's rewrite everything in Go to be cool. Financially makes little to no sense.

Let’s not just target Go with that sentiment, it applies almost universally, just in varying degree. Counterpoint: how is anyone supposed to learn, if not from their mistakes? We might worry about the blast radius, but there’s no compression algorithm for experience.

Another option is to learn from the mistakes of others.

Re: Things I wished more developers knew about databases

#330
post #280

Earlier quoted context omitted.

I would say that I have seen no discernible pattern, so I have learnt that it is imprudent to judge a developer by their age. People do learn over time, but some people learn more "cogently" than others, i.e. some get more out of one year's worth of learning than others get in 10.

The best thing I've seen on the topic is: "Some people have 10 years experience, some people have 1 years experience 10 times".

That's misleading because it's too simplistic. A smart person could spend 10 years gaining real, legitimate experience and they could still be eclipsed by someone with little experience but much more talent.
Post reply on HN