The Database Administrator is dead

__DBAs and Sysadmins (real ones, not these clowns who use nothing but chief or puppet and doesn't know how ./configure && make works) are becoming extinct purely from economical reasons__

As long as we're insulting people ... it's going to be 2014 in three weeks. Who compiles anything for PHP4? Boasting about an ability to handle other people's messy scripts sounds like an anti-pattern to me, and your recommendation to your org should be to develop a plan to refactor anything dependent on a library that has been deprecated for 6 years and actively out of development for 5.

I also feel like the vast majority of folks claiming merit badges as DBAs in 2013 are the product of failed technology roadmaps and technical debt with no plans to pay it off. I'm sure there are still places where true, full-time DBAs are worth their paycheck, but in my experience these are far and few between with the current status of OSS database options and hardware performance.

I couldn't agree more. The most apt analogy to explain this to non-tech folks I can come up with is how the automobile went from a hobby toy for the mechanically inclined to a tool people use everyday.

There will always be a place in the world for engineers who understand how to get the most of out an internal combustion engine and ways to improve it, just like for database engineers. But we don't all need our car in the shop twice a month having parts tuned. Nor do the vast majority of databases hitting performance limits that would require full-time DBAs.

Consulting is a great business decision when you reach hundreds of GB in data. In-house when your tossing around Tera or Peta-btyes. By the way, I can whole-heartedly recommend http://www.pgexperts.com if your having Postgres issues. After speaking with a few of their core engineers at DjangoCon 2013 I would consult with them anytime I had PG performance issues.

Look, you're doing the equivalent of looking at a car and saying that every single one needs to be hand tuned.

Just because you're a mechanic.

I said there is a bar when you will need a DBA. It is much, much, much higher than you're making out.

And the things you're talking about, most of the people mucking around with them probably shouldn't be. They've probably made it worse. Any the tiny few who actually need to? They're a tiny few.

That's too bad since this is one operation that incurs a very significant local drop in performance. The best optimizations I've ever done in website servers is to introduce local caching where reasonable, ignoring updates. User records, item records, comments, sessions, all would be cached locally. You wouldn't believe the speedup.

Having your data on a different server as your website introduces, effectively, a much higher minimum latency. It adds 5-6 ms to every single page load. If I were writing the site that would be a factor 5 drop at least. This is extra time that your webserver has to keep state for a request, and so effectively can mean a drop of a factor 5 or more.

The DBA said then that this would undermine consistency. And, of course, he's right. However for most things websites don't actually need consistency (plus the web request response model makes it impossible, as the data client-side is not included in transactions). The prices at checkout, sure, there you want consistency. Everything else ... do you really care if it takes 1-2 minutes for your webfarm to be entirely in sync ? (and if it's important enough you can implement "clear this cache item now" requests too, which I did for user records after a complaint)

Databases are so ridiculously slow it's not funny. If your data fits in 64-bit address space, an mmap'ed serialized data structure will blow it's socks off, and for everything that doesn't absolutely require synced data it's fast. If your critical data is small enough, keeping it in memory with a background thread getting signaled to dump it to disk after every update may also work.

I wrote a network-as-a-service infrastructure for an isp once (something to do with data on cell phones), where each machine would happily do 1Gbit of web traffic, directed by it's 5Gb database (meaning it would need to look up user account balance, potentially update it, for every request), consistently and quickly (99.9% of requests was Databases are a language + a B+tree data structure for those who can't, or won't implement it themselves. Generalized databases have zero hope of competing with specialized ones.