Live data from Hacker News

Things I wished more developers knew about databases

medium.com

131–140 of 464 posts

Re: Things I wished more developers knew about databases

#131
post #89

Earlier quoted context omitted.

When you look at SQL from a logical/set-based perspective, it is by no means unintuitive. Basically, all you do is join all the tables you need and then filter out everything you don't need and maybe do an aggregation here and there.

SQL can do a lot more though. Triggers, functions, procedures, access control.....

Plus indices, views, synonyms, constraints, and the entire concept of data normalization...

Re: Things I wished more developers knew about databases

#132
post #89

Earlier quoted context omitted.

When you look at SQL from a logical/set-based perspective, it is by no means unintuitive. Basically, all you do is join all the tables you need and then filter out everything you don't need and maybe do an aggregation here and there.

How about the following: - When to use JOIN vs a subquery? - When is a subquery actually a correlated subquery? Will this destroy your performance? Or is it a critical feature? - Should you put constraints in the JOIN or in the WHERE? Will the distinction drastically affect performance? - When do you use WHERE vs HAVING? - Is the NULL from the join because no joined row was found, or because the joined row had a NULL…

Like everything else, write it the simple/elegant way then profile it and tweak if you have to.

Once you're at the point where you have to worry about these things, tuning the SQL is still probably much less complex than writing the query in your app language or figuring out how a NOSQL db can do these joins.

Re: Things I wished more developers knew about databases

#133

Earlier quoted context omitted.

They're not identical concepts, but both relational schemas and statically typed programming languages provide assurances about basic structure. Mongo, and Python, offer no such assurances, and leave it to the developer to get it right. Agree that SQL's syntax is rather bad.

MongoDB supports Schema Validation since MongoDB 3.2 (Dec. 2015) and JSON Schema since MongoDB 3.6 (Nov. 2017) so MongoDB can enforce types strictly if you want to at the collection level. NoSQL doesn't mean NoSchema or NoTypes. https://docs.mongodb.com/manual/core/schema-validation/#json...

So they bolted it on eventually then. The relational databases went the other way and bolted-on schemaless data [0] [1]. We can still compare the schema-based and schemaless approaches.

[0] https://www.postgresql.org/docs/10/datatype-json.html

[1] https://docs.microsoft.com/en-us/sql/t-sql/functions/json-qu...

Re: Things I wished more developers knew about databases

#134
post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

SQL definitely solves a lot of performance issues for typical users who think they can do better

but throwing strings over the wire, trying to get as much as possible without crossing cardinalities and ignoring the n query problem makes SQL at scale disastrous

Devs always get one of these things wrong and not being able to use my programming language in the query makes some things hard to express

Datalog and Datomic are definitely worth learning https://youtu.be/Pz_NvY1kw6I

Re: Things I wished more developers knew about databases

#135

I learned SQL working on a "database as a product". The database was filled with medical ontologies. It was a perfect environment for learning. We were always looking for obscure things in that database, and rarely changed data, just selects for days. In the end I once wrote a query spanning about 100 lines that used common table expressions and found it quite maintainable. Where can an SQL beginner find such a datab…

CTEs are great. It's such a nicer syntax than subqueries.

Re: Things I wished more developers knew about databases

#136
post #89

Earlier quoted context omitted.

When you look at SQL from a logical/set-based perspective, it is by no means unintuitive. Basically, all you do is join all the tables you need and then filter out everything you don't need and maybe do an aggregation here and there.

How about the following: - When to use JOIN vs a subquery? - When is a subquery actually a correlated subquery? Will this destroy your performance? Or is it a critical feature? - Should you put constraints in the JOIN or in the WHERE? Will the distinction drastically affect performance? - When do you use WHERE vs HAVING? - Is the NULL from the join because no joined row was found, or because the joined row had a NULL…

> The basic concepts are simple, but the implementation details quickly become very complex

True, but those kinds of questions come up in every language: Should I use an array or a dictionary? Should people have references to projects or should projects have references to people or both? Is money a float, an int, a decimal or should I write my own money class? Should I memoize the results? Is it thread-safe?

As you can see, this could go on forever, pretty much for any language.

Re: Things I wished more developers knew about databases

#137
post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

I firmly believe that every developer should spend 2-3 weeks early in their career working with nothing but SQL. It will pay huge dividends for the rest of it. IMO a lot of the issue is that developers for many years using Java or PHP, were using SQL to handle everything. The application language was a pass through later between the client and the database. Your goal was to accomplish as much as possible in a single…

With the whole shelter-at-home thing, I had a chance to work on a simple app for my kids. One of the corollaries of "simple" was avoiding an ORM.

Implementing logic is single SQL queries can definitely be a bit challenging, but I thought it was also quite rewarding and liberating - Raw SQL is incredibly powerful!

Re: Things I wished more developers knew about databases

#138
post #49

Earlier quoted context omitted.

If you're writing a CRUD application, an ORM saves a lot of headaches. If you're doing complex reporting queries, an ORM is strictly worse. And yes, I've seen developers, architects, and authors of ORMs that believed otherwise. They are wrong. As an example, very, very few ORMs can make the distinction between SELECT ... FROM foo LEFT JOIN bar ON foo.id = bar.foo_id AND bar.category_id = 5 LEFT JOIN baz ON bar.id = b…

Curious, what is the difference here?

The extra "AND expr" in the join clause makes it more strict, which means the outer join can produce more nulls in the joined tuple. In a sequence like A left join B left join C, moving the extra join filter between the two joins is the difference between getting tuples like (a..., null..., null...) vs (a..., b..., null...).

Re: Things I wished more developers knew about databases

#139
post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

I don't think it's as elaborate as you're saying -- the clearest road into a productive job for many years has been "join a fast growing tech company who uses to do the magic, so the tech team is always building more business logic". When you've been working with (eg:) ActiveRecord and Ruby on your database data, direct SQL just doesn't come up until you're trying to wrangle more efficiency in your queries or do more complicated joins.

It certainly makes sense for all developers to learn more SQL! But it just hasn't been part of the daily work requirement for many.

As for the attraction of noSQL databases, I think perhaps it's more of a new-hotness trend rather than a serious technical decision. NoSQL has its place, but like most of this kind of tooling, it's often applied incorrectly because of a lack of thinking through the business logic.

Re: Things I wished more developers knew about databases

#140
post #31

Earlier quoted context omitted.

add to that "do not want to declare/worry about types", "cant be bothered about RAM usage/cache coherency/etc", "concurrency" etc. etc. All of those are "premature optimizations".

> All of those are "premature optimizations". And they are, until they aren't. And I really do mean that in both directions: Much of the time, the simplest, dumbest, most naive solution is 100% fine for realistic load for the forseeable future. And then some of the time it breaks terribly and you do need to spend effort optimizing it, whatever that entails.

To be clear I am not advocating optimizing everything down to the last bit here. Problem here is when THERE IS a need many would not even have a slightest clue where to begin as all those "low level" concepts were relegated into oblivion.
Post reply on HN