Live data from Hacker News

Things I wished more developers knew about databases

medium.com

421–430 of 464 posts

Re: Things I wished more developers knew about databases

#421
post #353
post #318

Earlier quoted context omitted.

Because then it’s harder to deal with our own imposter syndromes if we can’t blame it on the youth and hold their heads in the toilet while giving them the professional-development equivalent of a wedgie. This was discussed at length in last week’s “Grey Beard Weekly” newsletter.

So the "old people" are the bad guys for you, hmm. Then again, it's mostly "old people" who are discriminated against when it comes to hiring.

So the "old people" are the bad guys for you, hmm.

Yea that was snark, bud. Doesn't always translate well over text, admittedly.

But no, not the "bad guys". That's an interesting conclusion you've arrived at.

Re: Things I wished more developers knew about databases

#422
post #353

Earlier quoted context omitted.

So the "old people" are the bad guys for you, hmm. Then again, it's mostly "old people" who are discriminated against when it comes to hiring.

It seems so, and the fact that people are accusing me of ageism as I pointed out that a company hires only people below 24 years of age is very telling...

You didn't point that out, you made a vague complaint about someones age, and the comment you replied to doesn't even give you a good basis to assume that, so it's no wonder nobody understood what you tried to hint at. Hint: in many companies, a project involves only a small part of the workforce.

Re: Things I wished more developers knew about databases

#423
post #353

Earlier quoted context omitted.

So the "old people" are the bad guys for you, hmm. Then again, it's mostly "old people" who are discriminated against when it comes to hiring.

It seems so, and the fact that people are accusing me of ageism as I pointed out that a company hires only people below 24 years of age is very telling...

[deleted]

Re: Things I wished more developers knew about databases

#424
post #318

Earlier quoted context omitted.

Because then it’s harder to deal with our own imposter syndromes if we can’t blame it on the youth and hold their heads in the toilet while giving them the professional-development equivalent of a wedgie. This was discussed at length in last week’s “Grey Beard Weekly” newsletter.

Now that you have advertised the newsletter. Maybe share a link too.

There isn't a newsletter. It was tongue in cheek.

Re: Things I wished more developers knew about databases

#425
post #355

Earlier quoted context omitted.

I don't see how JOIN is like flatMap. It's a subset of a cross-product. It's more like do-notation: x

> I don't see how JOIN is like flatMap This is explained in the blog post I linked, including an example.

It's really flatMap/map from the link:

  List s1 = Stream.of(1, 2);
  Supplier> s2 = ()->Stream.of("A", "B");

  s1.flatMap(v1 -> s2.get()
                   .map(v2 -> tuple(v1, v2)))
    .forEach(System.out::println);
That's just a nested loop.

Re: Things I wished more developers knew about databases

#426
post #397
post #12

Earlier quoted context omitted.

Many interests are pulling developers' attention in several different areas all the time. Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), language concerns, new things All of these like to say "if only the developer could do $MY_AREA better, they'd be better developers and we'd have better software". Each of them wants to pile on…

> Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), language concerns, new things Hammers, nails, tape measures, saws, levels, reading blueprints, adhesives... If you want to be a professional, you need to learn the tools of your trade. Being able to work directly with a SQL database is a foundational capability in the software dev…

>If you want to be a professional, you need to learn the tools of your trade.

The tools of the trade has constantly been expanding. It's not set in stone as you imply.

It's like me telling a carpenter that they now also must become an electrician, window installer, insulation installer, HVAC installer, steelworker, concrete worker, brick layer, and security system installer.

Your new job title is "Fullstack Building Developer" instead of "Carpenter". We can't afford to have 10 specialists that all do a great job, we can only afford 1 person doing a poor/mediocre job in 10 things.

Re: Things I wished more developers knew about databases

#427
post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

Developers do not want to deal with data in 2D tables all the time. The expressive power SQL offers for manipulating these 2D tables is pretty good, but it can't compare to the ease of working with data not confined to that shape, which is what devs are used to from "real" languages - whether imperative, pure functional, or whatever.

In my mind the best example of this is grouping.

Just like most languages now have "Map" and "Filter" functions in their standard libraries, many now also have some form of "GroupBy" function for working with lists and objects in memory. Invariably the meaning of this is that it takes:

  1. A list of Xs

  2. A key function from an X to a Y, where Ys can be tested for equality
And spits out:

A list of pairs (Y, Xs), each pair containing:

  1. A key value Y

  2. A list of Xs from the input list matching that key
That list-of-(keys and sublists) can then be fed into whatever .Map(), .Filter(), etc. you want to use next. It is perfectly good for simple aggregates like "give me the total salary we're paying each department", but it's also great for stuff like "give me the 3 highest-paid employees within each department". It just makes sense - map over your groups, sorting each sub-list by salary descending and taking the first 3. Nice clean pure functional stuff.

By contrast in SQL, the first problem is super easy - group by department, select sum(salary). The second problem trips a lot of people up because suddenly you aren't really using GROUP BY anymore, even though the thing you're doing is still a "grouping" task. I think the main way to do it is to use ROW_NUMBER() OVER (PARTITION BY DepartmentId ORDER BY Salary DESC) to get a ranking number for each employee then select only those with ranking 1. If you were to describe to a person what you're trying to accomplish, the first step would be the same for problems 1 and 2 - get the employees for each department. But the queries in SQL are not alike and do not appear to share the same first step.

2. There needed to be dedicated extra language syntax to do this, instead of having one function (not even special syntax) that gives you the essence of grouping - which other functions then aggregate/massage further into whatever you need. This makes it harder to learn and harder to pick the right tool for a given query job.

3. Locality is almost fundamental to programming. I like writing the inner dept.OrderByDescending(e => e.Salary).Take(3) in the real programming language, because I can think locally to "this department's employees" and even down to "this employee's salary". In SQL, you're a mile above the data tying things together with push pins and string by ID, row number, etc. You rarely get to think locally.

4. In the same vein as above, structured data maps better to how I think. When I write data code, the metaphors in my head involve places and containers - drawers, shelves, moving things around. I am organizing the objects into boxes, going into each box, and picking my top 3. I am not marking each object with a number then going over them all as a big set and tossing out the ones with numbers greater than 3.

5. When I get my result from this query, I will probably want to work with it in the nested-list form, even if only to display it with merged cells for the the department (so I don't repeat every department name 3 times). Funny enough, if not using an ORM, I'll end up using my language's GroupBy to massage the flat rows back into this form.

Despite all this I want even less to do with NoSQL. I don't want to chuck blobs of JSON into the DB and end up with duplicate data, orphan data, ill-defined schemas, data that's only efficiently queryable one way, etc. I really do want to store my data in a relational form, and a normalized one where possible. I am happy with my tables. I just would love to query them in a LINQ-like language that is capable of representing and manipulating nested data structures -- and returning them to my application as such.

Re: Things I wished more developers knew about databases

#428

Earlier quoted context omitted.

> The question was and is rhetorical. The fact that you ask a question not expecting a direct answer is not proof that a direct answer does not exist. > People don’t inwardly digest the mistakes of others. In my spare time, I'm a rock climber, and mistakes in my rope systems can kill me. The same is true in mycology, firearms, airplane piloting, civil engineering. If you really feel that you can only learn from your…

No, but it’s a trap. If someone answers an obviously rhetorical question, they’re inadvertently demonstrating a predilection for engaging the construction, not the substance, of a statement, and almost certainly missing the ironic subtext. I’d be happy to repeat my assertion though. People don’t inwardly digest the mistakes of others, which is why educators on safety-critical topics such as those mentioned must go to…

> No, but it’s a trap. If someone answers an obviously rhetorical question, they’re inadvertently demonstrating a predilection for engaging the construction, not the substance, of a statement, and almost certainly missing the ironic subtext.

That's a pretty self-aggrandizing analysis of the situation.

From my perspective, I got you to make the statement, "People don’t inwardly digest the mistakes of others", which sounds a lot more absurd when you actually say what you mean plainly instead of hiding it in rhetoric.

Maybe some people go through life that way, but that's a pretty poor life strategy and I personally make a pretty big effort to learn from other peoples' mistakes. Maybe I haven't been successful always, but I can point to lots of examples of where I have.

Re: Things I wished more developers knew about databases

#429

Earlier quoted context omitted.

And with CSS, basically all you do is tell the browser how things look.

Conceptually yes. The devil is in the details: tables, flexbox, grid, div-soup, inconsistent naming, etc.

Sure. Conceptually is fine. But I'm pointing out how ridiculous it is to water down CSS/SQL to "Basically just _____". Nothing is hard by that logic.

Re: Things I wished more developers knew about databases

#430
post #115

Earlier quoted context omitted.

No, left joins were intended here. They mean different things. With inner joins the semantics are the same. If the performance is different, the database query optimizer is broken.

I see, apologies. However, I'm reasonably certain we'd see poor performance with that query on MySQL 5.7, but I don't think it's got as good an optimiser as Postgres for example.

I can't say that MySQL doesn't perform poorly. But if so, then their optimizer is more broken than I though.

Even SQLite figures this one out and does it right.

Post reply on HN