Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I wished more developers knew about databases

medium.com

Post reply on HN
Print view View original

451–460 of 464 posts

Re: Things I wished more developers knew about databases

#451

Post by pjmlp »

AsyncAwait wrote: post #241
vsareto wrote: post #227

Earlier quoted context omitted.

> I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data. There's usually not a relationship that goes the other way though, for example, developers don't tell DBAs to pick up code so they can write the models in our language in addition to the underlying SQL. This highlights a trend of increasing responsibilities push…

Agree with this. There's the whole DevOps nowdays as well, which basically just shifts what used to be an entirely separate, full-time, role onto the developer. Adding DBA to that sounds like it would benefit noone, except perhaps business owners looking for short-term savings at the expense of productivity, ala open-office floor plans.

In most places I have worked, knowing the database well enough to manage it and at very least write stored procedures has always been expected from developers writing database related code.

I have been doing this for a little more than 30 years now, so it isn't "nowadays" thing.

Re: Things I wished more developers knew about databases

#452

Post by pjmlp »

wenc wrote: post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

It depends pretty much on the quality of one's degree.

Thankfully we had one semester course on databases, all the respective layers, got to implement B+Tree indexes in C, and a GUI application using Oracle's Forms their 90's Visual Basic variant, but using PL/SQL instead.

Re: Things I wished more developers knew about databases

#453

Post by pjmlp »

TremendousJudge wrote: post #82
brightball wrote: post #56

Earlier quoted context omitted.

I firmly believe that every developer should spend 2-3 weeks early in their career working with nothing but SQL. It will pay huge dividends for the rest of it. IMO a lot of the issue is that developers for many years using Java or PHP, were using SQL to handle everything. The application language was a pass through later between the client and the database. Your goal was to accomplish as much as possible in a single…

> The application language was a pass through later between the client and the database. This style of doing things resulted in spaghetti style unmanageable databases, filled with an unknowable number of triggers and procedures, all written in PL/SQL (which is much, much worse than either Java or PHP). The reason why ORMs started to become popular is that you can write your application without filling your DB with ar…

Given how strong typed PL/SQL is, based on Ada, graphical tooling like SQL Developer exists, I wonder why it is much worse than either Java or PHP.

High performance SQL code should stay on the database, instead of doing wire round trips.

Re: Things I wished more developers knew about databases

#454

Post by pjmlp »

LouisSayers wrote: post #297
breischl wrote: post #163

Earlier quoted context omitted.

But now you're filling your application with arcane and inscrutable logic, with an extra layer of abstraction via the ORM to make it even less scrutable. I think one should view a SQL DB like a microservice. Instead of REST endpoints (or gRPC or whatever), create stored procedures. These define a strong contract with your DB, the capabilities that it provides to your app(s). Now you know what the query and insert pat…

I remember people I used to work with arguing against stored procedures for two main reasons. 1) Version Control - I guess a lot of the stored procedures were being put straight into the DB without recording a history of the changes. These days you could easily do this using DB migrations I guess. 2) Testing - is unit testing a thing for Stored Procedures? I guess again, you might be able to do this from code as well…

Both Oracle and SQL Server do support versioning tools and unit tests for their stored procedures.

You also have graphical debuggers for them.

Re: Things I wished more developers knew about databases

#455

Post by Amanzel »

seanc wrote: post #392
hermitdev wrote: post #256

Earlier quoted context omitted.

At a previous employer, I was forced to use a certain 3rd party ODBC library. Under certain circumstances, it would just do `exit(1)`, with nothing even to stderr. Very frustrating and annoying to debug/fix. Had I physically known the developer responsible for that behavior, I'd probably have faced murder charges and plead temporary insanity.

"Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live. I usually maintain my own code, so the as-if is true. " http://wiki.c2.com/?CodeForTheMaintainer

Lol

Re: Things I wished more developers knew about databases

#456

Post by goto11 »

amznthrowaway5 wrote: post #434
goto11 wrote: post #376

Earlier quoted context omitted.

Talent is no substitute for experience.

I'd say it's mostly the other way around, experience can't substitute for high natural cognitive ability. Of course a person still needs experience, but people with high cognitive ability don't need nearly as much, and people with less cognitive ability will hit thresholds of capability much more quickly. A lot of people live in a bubble of people with similar ability so they don't grasp the true importance of abilit…

I mean talent is no substitute for experience in the sense that experience teach you things about how the world works which you cant figure out just by thinking, however smart you are. But sure, cognitive ability is a multiplier, and if you don't learn from experience it is wasted.

A have met people much smarter than me who wrote bad code because they were working from a theoretical framework which just didn't correspond to realty.

Re: Things I wished more developers knew about databases

#457

Post by aryamaan »

chaps wrote: post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

Tangential: any good sql libraries in go? gorm lacks in features like prepared statements, batch inserts and we end up knitting raw SQL queries.

Re: Things I wished more developers knew about databases

#458

Post by aryamaan »

debaserab2 wrote: post #214
vsareto wrote: post #12

Earlier quoted context omitted.

Many interests are pulling developers' attention in several different areas all the time. Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), language concerns, new things All of these like to say "if only the developer could do $MY_AREA better, they'd be better developers and we'd have better software". Each of them wants to pile on…

I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data. There's a lot of devs that think database design is the same as starting a new ORM class and generating a migration file. > Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), lan…

as someone with 6 yoe and who considers himself a decent developer, I am looking for ways which can help me to be so-called expert developer-- I don't mean to say it mockingly.

Genuinely interested in the advice which can help to gain more than I will on my own with my current trajectory.

Re: Things I wished more developers knew about databases

#459

Post by devonkim »

vb6sp6 wrote: post #447
collyw wrote: post #443

Earlier quoted context omitted.

But we aren't doing something simple like that, we are building monstrosities based around the theory of micro-services in the cloud. Kubernetes. It takes hours to get a development environment put together to try and reproduce / debug a problem. We are adding complexity instead of keeping things simple. And query tuning isn't that difficult. Spend a few hours on this site and you will be better than 90% of devs out…

> we are building monstrosities based around the theory of micro-services in the cloud. Kubernetes This isn't something I'm doing and none of the people I personally know are doing this (disclaimer: im not in SV or the "startup scene") > And query tuning isn't that difficult. Spend a few hours on this site and you will be better than 90% of devs out there: https://use-the-index-luke.com/ It used to be that companies…

The technology inept organizationally can’t even figure out how to use that 64GB of RAM properly is my point and more RAM won’t fix a full blown table scan on every other query coming in because people don’t have a way to stop queries without limits. However, you can add modern SSDs with millions of IOPS to paper over complete ineptitude at using a free query analyzer in your database. But to myself, inability to bring oneself to use that is about as incompetent as not being able to use a debugger or profiler. I can certainly understand a level of inability when hiring only junior devs or having one’s hands tied when dealing with production systems locked behind compliance and bureaucracy, but all these things are problems that hardware nor more people will fix honestly.

And TBF, I am mostly familiar with the guts of technology laggards and they’re adopting K8S at frighteningly fast rates compared to anything else I’ve seen in almost 2 decades. Very common to see pretty smart people familiar with the latest tech yet lacking some serious holes in fundamentals familiar to those with experience.

Re: Things I wished more developers knew about databases

#460

Post by chaps »

devonkim wrote: post #459
vb6sp6 wrote: post #447

Earlier quoted context omitted.

> we are building monstrosities based around the theory of micro-services in the cloud. Kubernetes This isn't something I'm doing and none of the people I personally know are doing this (disclaimer: im not in SV or the "startup scene") > And query tuning isn't that difficult. Spend a few hours on this site and you will be better than 90% of devs out there: https://use-the-index-luke.com/ It used to be that companies…

The technology inept organizationally can’t even figure out how to use that 64GB of RAM properly is my point and more RAM won’t fix a full blown table scan on every other query coming in because people don’t have a way to stop queries without limits. However, you can add modern SSDs with millions of IOPS to paper over complete ineptitude at using a free query analyzer in your database. But to myself, inability to bri…

So, FWIW. The database I mentioned in my original post had 2TB of RAM. Food for thought!
Post reply on HN