Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Database Administrator is dead

thenextweb.com

Post reply on HN
Print view View original

51–60 of 78 posts

Re: The Database Administrator is dead

#51

Post by teddyh »

falcolas wrote: post #41
belorn wrote: post #35

Earlier quoted context omitted.

I normally associate DBA with efficiency and performance of data storage. However, you comment describes a blend between that, and a security administrator who's job include the creation and maintenance of high availability plans, disaster recovery plans (like backups), and monitoring. Do you think the overlapping speaks for the decrement of either profession specialization?

No - DB software is highly dependent on the system it's on. So much of a DB's underlying performance is going to depend on matching the settings you create for your DB to the hardware you're on (and optimizing the system to run a DB). In other words, efficiency and performance both depend heavily on the machine your DB is running on. As such, a good DBA needs to be able to do sysadmin tasks. The business won't care t…

What happens if you also have a “Web Server Administrator” with a similar holistic view? Won’t they step on each others’ toes? Is this not why you have separate roles, like the Dev/Ops division?

Re: The Database Administrator is dead

#52

Post by falcolas »

csmuk wrote: post #44
falcolas wrote: post #38

Earlier quoted context omitted.

Often the act of throwing "bigger kit" at the problem requires specialized tuning of the DB to be able to take advantage of it. More memory? Increase the buffer pool size. Faster HDD? Tweak the settings that determine how many disk operations are attempted every second. Bigger CPU? Figure out the point of diminishing returns on the number of CPU cores for your DB, and start sharding onto multiple DBs to make sure you…

This is all trivial knowledge under the "application performance" banner. All of our development staff are capable of diagnosing these issues and providing suggestions.

> All of our development staff are capable of diagnosing these issues and providing suggestions.

I hope you're paying them well, because if they're all crack DBAs on top of being crack developers, they're the proverbial unicorn, and are probably worth $2-3 hundred thousand apiece.

Re: The Database Administrator is dead

#53

Post by csmuk »

Keyframe wrote: post #46
csmuk wrote: post #28

Earlier quoted context omitted.

I disagree. dedicated DBAs are on their way out. Databases are becoming pretty good at managing themselves and the marginal performance gains from tuning usually are easily offset by throwing bigger kit at the problem or throwing more cash at the plan you are on.

I disagree. dedicated DBAs are on their way out. Is ETL considered a DBA task?

Not for us. Our developers built an API and tool chain to self service.

Re: The Database Administrator is dead

#54

Post by [deleted] »

jbooth wrote: post #34
falcolas wrote: post #32

Earlier quoted context omitted.

Because at a certain size of business, paying developers to make patches against your favorite OSS database doesn't scale. It's cheaper to pay for a, let's be honest, damned fine piece of software. Oracle, despite being the very definition of evil, has one hell of a software product. It is performant, has built-in features that makes Postgres look like BerkelyDB, HA solutions, and perhaps most importantly, the backin…

Eh. Different strokes for different folks, I guess. Most of those built-in features are anti-features IMO, and none of them make up for having to maintain long-ass PL/SQL functions as opposed to a reasonable programming language. And as far as performance goes, SSDs are cheaper than Oracle licenses, by a lot. Oracle doesn't have any magic pixie dust that changes your underlying hardware. If you want raw performance,…

[deleted]

Re: The Database Administrator is dead

#55

Post by falcolas »

twic wrote: post #42
falcolas wrote: post #38

Earlier quoted context omitted.

Often the act of throwing "bigger kit" at the problem requires specialized tuning of the DB to be able to take advantage of it. More memory? Increase the buffer pool size. Faster HDD? Tweak the settings that determine how many disk operations are attempted every second. Bigger CPU? Figure out the point of diminishing returns on the number of CPU cores for your DB, and start sharding onto multiple DBs to make sure you…

Databases definitely require knowledgeable tuning. But so do most other complicated moving parts in a modern infrastructure - web servers, app servers, kernels, cache servers, etc. Sysadmins manage those quite successfully. In my experience, they also manage databases quite successfully. The idea that a database is a special beast that requires special keepers is a holdover from a dark age.

> The idea that a database is a special beast that requires special keepers is a holdover from a dark age.

As are most databases (which are 10-20 years old).

Compared to a webserver, a DB is significantly more complicated, and significantly more important to your average business. Nginx takes a nosedive or performs poorly, and nobody really cares. Your DB takes seconds to respond to basic queries, and your entire business suffers.

I'm guessing you've yet to be bitten by DB performance issues that couldn't be resolve by adding indexes or basic query profiling. I'm glad to hear it, because it's not fun to have to set aside your day job and dive beneath that.

Re: The Database Administrator is dead

#56

Post by tbrownaw »

baudehlo wrote: post #39
tbrownaw wrote: post #17

Earlier quoted context omitted.

No, train them to use the database properly. A DBA is the person who keeps the databse running, much like a sysadmin is the person who keeps the server running.

That's a naive view of both DBAs and sysadmins these days. They both do much more than just keep things running. Sysadmin has mostly turned into opsdev. Same with DBA.

Sysadmin has mostly turned into opsdev.

Not in places large enough to value loose coupling (e.g., if keeping your servers running comes out of a different budget than improving your applications).

Re: The Database Administrator is dead

#57

Post by csmuk »

falcolas wrote: post #52
csmuk wrote: post #44

Earlier quoted context omitted.

This is all trivial knowledge under the "application performance" banner. All of our development staff are capable of diagnosing these issues and providing suggestions.

> All of our development staff are capable of diagnosing these issues and providing suggestions. I hope you're paying them well, because if they're all crack DBAs on top of being crack developers, they're the proverbial unicorn, and are probably worth $2-3 hundred thousand apiece.

No they just know how not to fuck up their queries to start with so we don't need a DBA to optimise 300 join statements...

Re: The Database Administrator is dead

#58

Post by falcolas »

teddyh wrote: post #51
falcolas wrote: post #41

Earlier quoted context omitted.

No - DB software is highly dependent on the system it's on. So much of a DB's underlying performance is going to depend on matching the settings you create for your DB to the hardware you're on (and optimizing the system to run a DB). In other words, efficiency and performance both depend heavily on the machine your DB is running on. As such, a good DBA needs to be able to do sysadmin tasks. The business won't care t…

What happens if you also have a “Web Server Administrator” with a similar holistic view? Won’t they step on each others’ toes? Is this not why you have separate roles, like the Dev/Ops division?

Well, your web server is not your database server. If it is, just know that your first scaling task will be to separate the two onto separate systems.

I was in a hurry when I wrote my earlier reply - system monitoring is not the only monitoring we perform. We watch the database itself for health and performance. We watch for abnormally long queries, abnormal locking, excessive deadlock resolution, replication health, and exploits.

Backups themselves require a bit of specialization - there are at least four different methods I'm aware of off the top of my head, and the proper choice for which to use is going to depend on the business (we typically set up 3 of the four - logical dumps, rolling binlogs, and binary dumps).

HA plans, particularly live failover, requires a deep knowledge of the database, its replication solution, and your business needs. Safely failing over a live MySQL database in a way that results in no data loss is hard to do.

Quite a bit of specialized knowledge is required to work with databases, even for normal sysadmin tasks.

Re: The Database Administrator is dead

#59

Post by mattzito »

jbooth wrote: post #21
ghshephard wrote: post #15

Earlier quoted context omitted.

"Most people don't need dedicated DBAs" That's a pretty easy statement to defend. But, I'll respond by saying that most Companies running Oracle 11g with more than a couple terabytes of databases, require a competent DBA, particularly if Disaster Recovery/Transaction Rollback is important. " the DBA often never has sufficient domain knowledge of the problem " Of the half dozen or so truly high level DBAs I've worked…

Why would someone run Oracle in this day and age? "We have 100k lines of established PL/SQL running our business" is a reason, of course, but you could make a similar argument that COBOL development is alive. PL/SQL pretty much always leads to an unmaintainable mess, and if you need BIG DATA, you can go much bigger without Oracle than you can with it, and cheaper to boot.

Guh - every few months this comes up, and it's fairly hilarious to me how much anti-oracle dogma there is here.

Oracle is the most feature-rich database in the industry, and the most expensive, and the most complicated. This comes with benefits and curses.

It is freakishly powerful. If you want, you can roll back just your view of the database to three hours ago. You can show different versions of the same schemas to different clients. You can dynamically scale out your database to hundreds of servers without having to manually shard. You can do multi-master replication, master-to-slave, master-to-slave-to-master, any combination you can think of.

Your database clients can automatically detect when a primary has gone down and fail over to a standby, which will automatically be brought online, no admin intervention necessary.

If you give Oracle raw disks, you can tell it to automatically use the faster parts of the spindles for more commonly used data. Or give it some SSDs, and it'll use it for cache. Or, if you buy Oracle storage servers, it'll actually offload basic query execution to the storage.

With that power comes great cost and complexity, which is why many web companies don't bother with it - when you get to google or facebook scale you build these types of capabilities into your application tier.

But I know of a ton of multi-petabyte Oracle implementations at big traditional companies, and they love it. Because they don't want to have to build all of that functionality at the application tier, and they trust Oracle's reliability.

Re: The Database Administrator is dead

#60

Post by kennygorman »

joosters wrote: post #14

TL;DR: Cloud-based DB co-founder says that most of his customers don't employ a DB admin. Um, it would only be surprising if they did , that would surely indicate that his company wasn't doing a good job.

True.

But most of these folks didn't have DBA's or even OPS engineers before they became customers.

Mostly they are teams of developers, they have developed the application and have (I think correctly) worried about the customer experience, code base, IP, performance, etc.

They do want the database driver API (in this case MongoDB) to just work beyond the interface. Backups, scaling, filesystems, etc they want to be part of the service, and just handled.

This is true for new business and it's true for businesses that are developing new apps or new projects or even projects that can't scale out on other infrastructure.

Full disclosure: I wrote the article in question ;-).

Post reply on HN