Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things I wished more developers knew about databases

medium.com

Post reply on HN
Print view View original

251–260 of 464 posts

Re: Things I wished more developers knew about databases

#251

Post by debaserab2 »

AsyncAwait wrote: post #241
vsareto wrote: post #227

Earlier quoted context omitted.

> I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data. There's usually not a relationship that goes the other way though, for example, developers don't tell DBAs to pick up code so they can write the models in our language in addition to the underlying SQL. This highlights a trend of increasing responsibilities push…

Agree with this. There's the whole DevOps nowdays as well, which basically just shifts what used to be an entirely separate, full-time, role onto the developer. Adding DBA to that sounds like it would benefit noone, except perhaps business owners looking for short-term savings at the expense of productivity, ala open-office floor plans.

Keep in mind that full time role used to be a hell of a lot of work. Take the project I'm working on right now: https://gist.github.com/ajbdev/11f494906ebb6aa3f6bdf35f2437e...

In ~50 lines of code I can erect a load balanced set of web servers with my app binary preloaded, secured in a private VPN subnet with an auto-scaling policy attached to it.

15 years ago this project would have meant:

- Racking new hardware somewhere on premise

- Configuring several routers and networking equipment

- Tweaking the different software responsible for the different app layers (load balancing, web servers, etc)

- Hoping we forecasted demand correctly and whatever hardware we racked can handle a traffic spike or growth surge

You needed a lot more expertise back then.

And really you can trade all of what I'm doing for a higher level of abstraction that requires even less knowledge of the underlying technology (netlify, heroku, amplify, etc) if you're willing to pay a little more.

Re: Things I wished more developers knew about databases

#252

Post by cookiecaper »

msla wrote: post #224
cookiecaper wrote: post #216

Earlier quoted context omitted.

Unless you got yourself into a situation where you were expected to set up a whole office with the same speed and expertise as a full-time network engineer based on some gross misrepresentation of your skillset, I don't see how this story is particularly relevant. Maybe it's a good cautionary tale about presuming that SoHo is the limit of networking? Technical topics are indeed both very deep and very broad. The leve…

> Unless you got yourself into a situation where you were expected to set up a whole office with the same speed and expertise as a full-time network engineer based on some gross misrepresentation of your skillset, I don't see how this story is particularly relevant. Taking a job as one of a business's "computer people" puts you in the path of a whole lot of interesting tasks, even if your main job is programming. > M…

> For example, I can make programs and package them in Docker containers and run them that way, but I don't know how I'd play with Kubernetes in a realistic fashion.

Minikube: https://kubernetes.io/docs/tasks/tools/install-minikube/

Alternately, you'd take advantage of $CLOUD_PROVIDER's initial signup credit and start cloud instance equivalents.

Nowadays most things have good virtual environments floating around (you can even download virtualized mainframes if you want). A little bit of time tinkering with such environments will take you surprisingly far -- especially in fields like network engineering, where even most professionals don't know how to experiment.

Re: Things I wished more developers knew about databases

#253

Post by hermitdev »

> You are lucky if 99.999% of the time network is not a problem.

This reminds me of one time I was having networking issues (around a CRUD GUI). Oddly, I identified the issue the (business) day before my users did. I was working on a rather large change (OS updates, 32->64 bit) and noticed that multicast updates broke. The timing was funny. I was working towards a minimal reproducible example already, and I had a small test app that showed that multicast was broke on my PC. So, I ran down and tried the app on their computers, and it worked... between the two of them, but not the company et large.

That's when I vaguely remembered an email for Network Ops about a router change on my user's floor over a weekend. I went by the Ops team (they didn't like email for some reason) and told them what I was seeing.

The short of it was, yeah, the router change screwed my 2 users on the floor (they got put on the printer VLAN by accident, which didn't receive multicast). Separately, the issue I saw was a bug in how we built a 3rd party lib on Windows that provided 64-bit multicast support.

Re: Things I wished more developers knew about databases

#255

Post by nuggien »

chaps wrote: post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

Nothing about go is magical, and you probably solved the problem blindly.

Re: Things I wished more developers knew about databases

#256

Post by hermitdev »

harikb wrote: post #200
chaps wrote: post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

Go sql/database uses its own connection pool. But still that shouldn't create any problems. I have seen the reverse where apps that assumed temporary tables stick around from statement to statement without an explict `txn` (which regular postgres connections don't need) clearly failed. But I have not seen the issue you talk about. My wild guess would be that the Go code never closed the result/rows which caused eithe…

At a previous employer, I was forced to use a certain 3rd party ODBC library. Under certain circumstances, it would just do `exit(1)`, with nothing even to stderr. Very frustrating and annoying to debug/fix. Had I physically known the developer responsible for that behavior, I'd probably have faced murder charges and plead temporary insanity.

Re: Things I wished more developers knew about databases

#257

Post by chaps »

nuggien wrote: post #255
chaps wrote: post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

Nothing about go is magical, and you probably solved the problem blindly.

Wow, okay. You alright over there?

Edit, because meh: I'm making no claims about go itself. No idea what makes you think that's what I'm saying, since I'm clearly talking about a library, and not even any stdlibs. "Magic" is just a term useful for describing systems that sweep much of their abstractions under the carpet in a way that probably has gotchas. Granted, the term itself is magical.

In terms of fixing the problem, I knew for a fact that the keepalives that I was seeing were nothing like what I've seen in the past, at many companies, across Oracle, postgres, and MySQL, all who've implemented "SELECT 1" for the sake keep alives, by devs who've been in the field for much longer than me. The suggestion was by no means blind, unless you consider implementing a widely used method for this exact purpose, "blind". Had I gone a different route in fixing the problem within the stable, existing system, it would have likely broken many of the other database connections by many teams. I'll pass on that, since frankly, even ignoring the risk of such a change, the dev should have done the investigation themselves.

Your post was unnecessarily aggressive and seems to come from me having struck a nerve somehow. Genuinely hoping you're doing alright. Peace.

Re: Things I wished more developers knew about databases

#258

Post by castorp »

homami wrote: post #231
wenc wrote: post #16

I never realized this before but many excellent developers struggle with SQL beyond simple SELECT statements. I have a colleague who is by all accounts a deeply technical person but one day he confessed to me that he didn't really grok SQL and that he'd rather work with a "real" procedural programming language to just store and retrieve data. Part of it may be due to the fact SQL isn't really a programming language b…

I find SQL INSERT statement not intuitive. I can understand why SQL requires me to declare the field names and then the values of a new row that I am inserting; but it would've been a huge time saver if SQL had a key-value dictionary-like syntax: INSERT INTO "my_table" "col1": value1, "col2": value2, ...

And when you need to insert more than one row, you repeat the column name over and over again for each row - not a good idea either.

Re: Things I wished more developers knew about databases

#259

Post by castorp »

PZ81JUXJE7uJ wrote: post #238
homami wrote: post #231

Earlier quoted context omitted.

I find SQL INSERT statement not intuitive. I can understand why SQL requires me to declare the field names and then the values of a new row that I am inserting; but it would've been a huge time saver if SQL had a key-value dictionary-like syntax: INSERT INTO "my_table" "col1": value1, "col2": value2, ...

MySQL supports a insert/set statement: INSERT INTO tbl SET a = 1, b = 2;

How does that look like with a multi-row insert?

Re: Things I wished more developers knew about databases

#260

Post by vsareto »

debaserab2 wrote: post #247
vsareto wrote: post #227

Earlier quoted context omitted.

> I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data. There's usually not a relationship that goes the other way though, for example, developers don't tell DBAs to pick up code so they can write the models in our language in addition to the underlying SQL. This highlights a trend of increasing responsibilities push…

> There's usually not a relationship that goes the other way though, for example, developers don't tell DBAs to pick up code so they can write the models in our language in addition to the underlying SQL. This highlights a trend of increasing responsibilities pushed onto the developer. Yes, because as a developer you're the one who has the responsibility of implementing the business requirements of the app. There's n…

>Often times when the DBA tries to teach the developer it's because he's sick of getting those 3AM wake-up calls.

And if he just learned to code, he could write all the queries himself, check in his own code, and never have 3AM calls. I see no reason why a DBA couldn't manage to write queries in source, especially with some hand-holding w.r.t. source control and check-in rituals. If they are already doing too much, just hire or train more of them. DBAs are equally as smart as devs. Loop DBAs into the business requirements.

>That's a strange mindset to have. Different jobs have different technologies, and as a developer you learn how to work with them. The specialists aren't designating anything, the job you're responsible for is.

This is very push/pull about who should own what and I personally think devs are often responsible for too much. In silo'd places, make no mistake that input is gathered about who owns what. In more collaborative environments, you see the other sides willing to step in and share the responsibilities; blurred lines, but they are still there. What we've adopted is an over-reliance that a dev will just learn many things and we hope he's good enough at all of them that we don't really need deep knowledge or need to delegate work.

Post reply on HN