Live data from Hacker News

Things I wished more developers knew about databases

medium.com

221–230 of 464 posts

Re: Things I wished more developers knew about databases

#221
post #7

(The 80/20 rule applies below, some developers do care) Developers... just don't care. They want to spin up an ORM, point it at a URI, and forget about it. I've fought this for over a decade now as a DBA, SRE, DevOps, and architect. Most of the developers don't want to deal with anything infrastructure-wise; they want to spend all the time they can just focusing on the problem they're writing software to solve. Obser…

I don’t care and that’s why I use raw sql lol. I don’t want to learn some orm bs

Re: Things I wished more developers knew about databases

#223
post #118

Earlier quoted context omitted.

A database is also exceptionally good at doing transactional stuff like atomically incrementing something. I'd even say: This is something that belongs in the database and not in some brittle application logic.

Generally, I agree. But in many RDBMSs, auto-increment features explicitly do not guarantee gap-less sequential ordering. If anything throws, it's easy to end up with discarded numbers, in which case you need to catch all errors, inspect the state of the sequence and/or the entity you're populating, and re-seed the sequence. At this point, I'd argue you've already left the pristine gardens of set theory and wandered…

ah! TIL:

MYSQL - https://stackoverflow.com/questions/449346/mysql-auto-increm...

POSTGRES - https://stackoverflow.com/questions/2095917/sequences-not-af...

thanks!

Re: Things I wished more developers knew about databases

#224
post #204

Earlier quoted context omitted.

The phrases "functional understanding" and "working knowledge" are gigantic sucking tarpits. Before I got my current job, I felt confident in my knowledge of computer networking at the LAN level. I knew I wasn't going into the telecom world and I knew I didn't have the knowledge to debug BGP or ensure a CO was doing everything right, but LANs? Sure. No problem. I knew DHCP, Ethernet, TCP/IP, even stuff like PPP which…

Unless you got yourself into a situation where you were expected to set up a whole office with the same speed and expertise as a full-time network engineer based on some gross misrepresentation of your skillset, I don't see how this story is particularly relevant. Maybe it's a good cautionary tale about presuming that SoHo is the limit of networking? Technical topics are indeed both very deep and very broad. The leve…

> Unless you got yourself into a situation where you were expected to set up a whole office with the same speed and expertise as a full-time network engineer based on some gross misrepresentation of your skillset, I don't see how this story is particularly relevant.

Taking a job as one of a business's "computer people" puts you in the path of a whole lot of interesting tasks, even if your main job is programming.

> Maybe it's a good cautionary tale about presuming that SoHo is the limit of networking?

It's certainly that, but I want to expand on this a bit: SoHo is the only stuff most people can play with. For example, I can make programs and package them in Docker containers and run them that way, but I don't know how I'd play with Kubernetes in a realistic fashion. There's whole genres of technology most people can't get realistic access to without some institutional support. It's an effective ceiling on some kinds of knowledge.

As far as learning how to learn, I agree with you. I think a lot of it comes down to vocabulary: Once you know the terms the experts use, you can bootstrap effectively and learn more terms and bootstrap even more effectively. Plus, words have a way of coming back to you at odd intervals, effectively dropping you hints when you see something you vaguely recognize.

Maybe we should all have Word Of The Day calendars.

Re: Things I wished more developers knew about databases

#225
post #198

Earlier quoted context omitted.

There's a middle way which is very powerful: SQL views (just SQL queries; no triggers or procedures) Here's a powerful mindset trick: think of SQL views as an sort of a REST API , but whose access language is SQL and not HTTP, and that returns data in a table rather than JSON (hierarchical). I once tried to build a REST API to a database, and someone told me I already had a battle-tested and highly performant API tha…

This is a very interesting comment! Two questions: Do you have any example code that shows how this works? I get what you’re saying intuitively but example code will help me bring it to table. What about cross cutting concerns? I’ve found stored procedures to be a performant solution here. By version controlling them, and limiting to pure functions, I found them quite maintainable. Would you instead just define a new…

Briefly,

1. Let me try with a simple example. Suppose you have a fact table A with fields (ItemID, Item, Amt) where Amt is in USD. Rule of thumb is: don't expose A to the consumer; instead write a SQL View V_A and expose that instead:

  CREATE VIEW V_A AS SELECT ItemID, Item, Amt FROM A
Then suppose a European counterpart wants to use the same API but needs the amounts to be in Euros. You can write another view: (in practice the conversion 0.92 shouldn't be a static number, this is just for illustration)

  CREATE VIEW V_A_EURO AS SELECT ItemID, Item, Amt * 0.92 AS AmtEUR FROM V_A
Expose this to the Europeans. You can keep stacking views on top of other views. Your U.S. consumers will always see the data through the lens of V_A and your European consumers will always see it through V_A_Euro.

Suppose the underlying table A now changes. There's been a merger and the company now stops reporting currencies in USD, and everything is now in British Pounds so your DBA adds a field AmtGBP and starts populating that field instead. Amt still contains historical data, but moving forward the data in Amt will be NULLs; AmtGBP is the new internal baseline currency. From a VIEW perspective, all you have to do is:

  ALTER VIEW V_A AS SELECT ItemID, Item, ISNULL(Amt, AmtGBP * 1.23) AS Amt FROM A
Your V_A and V_A_EURO consumers (could be Tableau, Excel, other SQL views, etc.) will still happily receive data per usual, unaware of the internal changes (the British are coming!) that have occurred. Contract kept.

  Table A 
2. Cross cutting concerns come in many forms so not sure if I can address. Stored Procedures are definitely an acceptable abstraction -- they accept parameters and can return tabular results just like VIEWs. They do however work in a procedural manner (like subroutines) and can produce side effects, which is sometimes necessary to accomplish very specific tasks. VIEWs on the other hand are more similar to pure functions (unless random number generation is involved) with no side effects. Because views are dynamic, they flex with your data and VIEW definitions.

Re: Things I wished more developers knew about databases

#226

Earlier quoted context omitted.

Too bad there isn't an equivalent on-call punishment for management and others around the dev process who also make mistakes and bad decisions. As far as I can tell, they get paid more for doing less.

I'd guess that problem might go away if you make their bonuses contingent on a sleep numbers (along with the SLAs) for whoever has to run the application or it's downstreams whether that's a dev or an ops type. Base the number on pagerduty calls or something like so they can't game the number.

That is a wonderful idea. Managers are responsible for the performance of their teams. NOBODY wants to sandbag sleep numbers, so it'd be a decent comp metric.

Re: Things I wished more developers knew about databases

#227
post #12

Earlier quoted context omitted.

Many interests are pulling developers' attention in several different areas all the time. Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), language concerns, new things All of these like to say "if only the developer could do $MY_AREA better, they'd be better developers and we'd have better software". Each of them wants to pile on…

I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data. There's a lot of devs that think database design is the same as starting a new ORM class and generating a migration file. > Database, security, accessibility, performance, infrastructure, tooling and productivity, business concerns, workflow processes (agile), lan…

> I don't agree. If you're designing data structures in a code base you shoulder some of the responsibility for the persistence characteristics of that data.

There's usually not a relationship that goes the other way though, for example, developers don't tell DBAs to pick up code so they can write the models in our language in addition to the underlying SQL. This highlights a trend of increasing responsibilities pushed onto the developer.

>Yes, these are all things that devs should strive to know as much about as possible. Software isn't easy. It takes a long time to become an expert. 10 years sounds about right.

But in some cases it's specialists designating what an expert developer should know. It's giving away control in some respects. This turns into new job requirements and a higher barrier for entry. The growth will need to stop at some point.

Re: Things I wished more developers knew about databases

#228
post #149

Here's a fun bug I had a few years ago - Had a postgres database which was using pgbouncer for connection pooling. The most senior developer (24yo or so) we had on the project was using Go to connect to the database to write some simple reports, but each report took hours to run, and often had to sleep for 30+ minutes. So, after a while, pgbouncer would kill their connection, and their report would die. No other appl…

> The most senior developer (24yo or so) I see the problem there.

I've seen older developers that call themselves senior, but lack basic knowledge. I've seen younger developers, wise well beyond their years. Age simply isn't a big factor in how you judge a developer.

Re: Things I wished more developers knew about databases

#229
post #34

Earlier quoted context omitted.

I wish I could slap anyone who gives a hoot about tabs vs spaces. Fortunately modern languages like go are removing the version control problem that not caring about style and using auto-formatting IDEs produces.

> using auto-formatting IDEs produces. I strongly suspect a lot of the remainder of the tabs vs spaces reminder is actually about how it either constrains editor / IDE choices OR requires and investment of time to deal with whichever choice someone else made. "Why do you care, my IDE just handles this" is pretty close to saying "use my IDE," on top of "use my convention."

That's why people are asking to push this problem down to the language level the way Go has done. Go defines both the "correct" format of the code as well as a standard way for any/every editor to enforce it (gofmt etc). That eliminates the double-headed subjectivity of both "use my IDE" AND "use my convention" down to just "use the standard convention defined by the language". And people love it because finally we can stop arguing about the stupid color of paint for the bikeshed and just bloody build the damn thing.

Re: Things I wished more developers knew about databases

#230
An odd thing that happened to me yesterday with the PostgreSQL ODBC driver (psqlODBC) v9.3.400. It wouldn't let me insert a string longer than 255 characters long into a character varying field into a local v9.4 database on windows using a recordset update. I didn't have a problem pasting it in via pgadmin. Altered the field to text and the problem went away. I've a suspicion that there is a limit on text in the tens of thousands of characters length too though despite both those fields being essentially the same thing and limitless.
Post reply on HN