Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DevOps Isn't Dead, but It's Not in Great Health Either

thenewstack.io

Post reply on HN
Print view View original

61–70 of 74 posts

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#61

Post by jiggawatts »

KronisLV wrote: post #39
jiggawatts wrote: post #37

Earlier quoted context omitted.

Why even use SFTP? Just have them upload blobs to S3 or Azure Storage. The security is better and both are serverless solutions that cost basically nothing at normal scale. I seriously don’t understand why people in 2024 insist in reproducing these core services but badly and at great expense. PS: Azure Storage is multi-protocol and supports SFTP! https://learn.microsoft.com/en-us/azure/storage/blobs/secure...

> Just have them upload blobs to S3 or Azure Storage. Not everyone can store their data in cloud services, most likely. That said, self-hosted S3 compatible solutions like MinIO might be a good choice: https://github.com/minio/minio or maybe SeaweedFS for something with a more permissive license: https://github.com/seaweedfs/seaweedfs

> Not everyone can store their data in cloud

This is a nearly verbatim conversation I’ve had three or four times in the last twelve months:

“Why not just use blob storage for the data ingest?”

“We have security standards for sensitive data; we have to use private links and validated protocols and systems.”

“Okay, what’s step two? Where does the data end up? Where do end users consume it from?”

“Power BI!”

“The local app?”

“No, the online version.”

“You just said you can’t use cloud services like blob storage!”

“We can’t!”

“You’re using Power BI!”

“That’s different!”

“How?”

“Err…”

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#62

Post by Izkata »

digitalnomd wrote: post #52

What I’ve found is that pure “DevOps” roles have become a way for recruiters to sell people on being the ops person without them realizing it. I once worked at a place that hired me in as a software dev working in DevOps and the position quickly became consumed by the on-call workload. The system we were managing constantly had fires that needed to be put out but we could never fix the underlying problems because som…

> What I’ve found is that pure “DevOps” roles have become a way for recruiters to sell people on being the ops person without them realizing it.

Way waaaay back when I first heard of DevOps, it was described along the lines of "bringing developer best practices like automation and testing to ops". What you're seeing could be a remnant of that.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#63

Post by dspillett »

fifilura wrote: post #31
dspillett wrote: post #30

Earlier quoted context omitted.

> ops are an almost extinct species. This is infuriating at times. We ingest a lot of data from 3rd party systems of our clients for things like KPI analysis, the devs¹ provided various APIs for submitting data and we can read from various others, but good old reliable dumping-CSV-files-to-us-via-SFTP is still what most clients want to do. I have no problem with this², BUT the amount of times I end up talking to clie…

Would it be possible for you to take a step back and describe the problem you are infuriated by again?

Sitting on the phone with people who's job it is to know these things for their organisation, having to explain to them that no, they shouldn't send me the private key, not even (especially not even) if they send the passphrase too, and many other problems.

There seems to be a dearth of people for the ops jobs that need doing, hence OP's “ops are an almost extinct species”, so the tasks are thrown around and have to be done by the first person who doesn't duck fast enough. Those of us on the other side of the company interaction boundary end up having to help them with what should be the basics. I'm not their infrastructure team, I don't have access to be able to answer some of the questions they ask about their own infrastructure. I shouldn't even be client facing never mind teaching them how to do their jobs.

The infuriating part (perhaps “frightening” is a better word here) is that there are people managing the secure transmission of data (sometimes sensitive personal data) who do not understand what is going on.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#64

Post by dspillett »

jiggawatts wrote: post #37
dspillett wrote: post #30

Earlier quoted context omitted.

> ops are an almost extinct species. This is infuriating at times. We ingest a lot of data from 3rd party systems of our clients for things like KPI analysis, the devs¹ provided various APIs for submitting data and we can read from various others, but good old reliable dumping-CSV-files-to-us-via-SFTP is still what most clients want to do. I have no problem with this², BUT the amount of times I end up talking to clie…

Why even use SFTP? Just have them upload blobs to S3 or Azure Storage. The security is better and both are serverless solutions that cost basically nothing at normal scale. I seriously don’t understand why people in 2024 insist in reproducing these core services but badly and at great expense. PS: Azure Storage is multi-protocol and supports SFTP! https://learn.microsoft.com/en-us/azure/storage/blobs/secure...

> Why even use SFTP?

Because that is the infrastructure they have, what their current arrangements support. We can offer a number of other methods, but if their other systems don't support than and they don't want to pay for upgrades, SFTP it is. Not that I have anything against gobs of data arriving via SFTP for cleaning and importing¹ – it is tried & tested, reliable, and if both sides have a clue² it is easy & pain free. Much the same set of reasons they still want to send us CSV formatted data, often Win1252 encoded rather than UTF8, not JSON or something else better defined.

> Just have them upload blobs to S3 or Azure Storage.

The data is ending up in Azure storage in our case. But any transfer method still has identification and authentication matters to deal with, and they need to understand and their other software needs to support, the method chosen. This is why SFTP often wins by default: it is a relatively proven technique that people see as secure, and it is commonly supported.

> The security is better and both are serverless solutions that cost basically nothing at normal scale.

The cost to the client is only zero if their other systems already support it. They are very much against anything that implies extra development costs or the cost & effort of assessing and switching to newer systems.

> PS: Azure Storage is multi-protocol and supports SFTP!

SFTP support in Azure is a pretty recent addition, some time in this last year IIRC. We (well, I) had to put together our own relay arrangement some years ago: OpenSSH, appropriately locked down, with blob containers in storage accounts mounted via blobfuse). It works well, in nice cheap VMs/containers.

SFTP support in Azure works out rather expensive for our use case, or any reasonable use case IMO. It needs the hierarchical namespace enabled though that is “only” an additional $27/month per storage account, but then you need to pay for SFTP support by the hour!³⁴ Unless we constrain the clients to sending data at specific times that comes to >$200 per month, per account. We can't get away with charging our smaller clients, or the bigger ones for that matter, an extra $2,400/yr nor can we afford to eat that ourselves. See https://azure.microsoft.com/en-gb/pricing/calculator/ for reference.

We could work around the per-account part by stuffing all the data into multiple containers in one account – but that would not fly with some of the requirements financial sector companies understandably have for data at rest, so we would never pass an audit if we did that. Many even want their live and UAT/training/other environments to have completely separate storage accounts so that is potentially more than one lot of up to $2,400 per year per client needing SFTP. And even on one account, $2,400/year to enable 24/7 availability of SFTP transfers is absolutely ridiculous.

----

[1] I'd rather it be cleaned first, but that is a separate issue

[2] This part is, of course, the problem…

[3] $0.30/hour: https://azure.microsoft.com/en-gb/pricing/details/storage/bl...

[4] No shit… https://learn.microsoft.com/en-gb/azure/storage/blobs/secure....

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#65

Post by dspillett »

jiggawatts wrote: post #61
KronisLV wrote: post #39

Earlier quoted context omitted.

> Just have them upload blobs to S3 or Azure Storage. Not everyone can store their data in cloud services, most likely. That said, self-hosted S3 compatible solutions like MinIO might be a good choice: https://github.com/minio/minio or maybe SeaweedFS for something with a more permissive license: https://github.com/seaweedfs/seaweedfs

> Not everyone can store their data in cloud This is a nearly verbatim conversation I’ve had three or four times in the last twelve months: “Why not just use blob storage for the data ingest?” “We have security standards for sensitive data; we have to use private links and validated protocols and systems.” “Okay, what’s step two? Where does the data end up? Where do end users consume it from?” “Power BI!” “The local…

Similar, though only for one client ATM, and our clients are fine with cloud storage these days as long as everything is in an account properly locked down (bring your own key, etc.) so only their application instances can access it.

At least the data doesn't rest (as far as we know…) in PowerBI.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#66

Post by DarkNova6 »

threatofrain wrote: post #40
surfingdino wrote: post #6

I like DevOps, I don't like being on call 24x7 so I tell my clients I am a software developer not an ops. DevOps was sold as a way to avoid dev and ops teams blaming each other for not shipping software. It was at the time true that devs did not understand hardware and networking and ops did not particularly understand software. We got nice things like Docker and k8s, but the core issues have not gone away, devs don'…

DevOps is dying? As in the job market for DevOps is actually shrinking?

I'd rather think that the need for Ops is growing much faster than there is supply.

It is a primarily knowledge-intense profession where you can't just hire some untrained professionals and let them hack together an app. Or cannibaliize data-analystis and make them build some backend.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#67

Post by more_corn »

marcus0x62 wrote: post #45
KronisLV wrote: post #39

Earlier quoted context omitted.

> Just have them upload blobs to S3 or Azure Storage. Not everyone can store their data in cloud services, most likely. That said, self-hosted S3 compatible solutions like MinIO might be a good choice: https://github.com/minio/minio or maybe SeaweedFS for something with a more permissive license: https://github.com/seaweedfs/seaweedfs

> That said, self-hosted S3 compatible solutions like MinIO might be a good choice But then you would have people who can't figure out SSH trying to figure out private S3-compatible infrastructure.

Having walked people through both I’m going to say setting up a client for an S3 clone is easier.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#68

Post by threatofrain »

DarkNova6 wrote: post #66
threatofrain wrote: post #40

Earlier quoted context omitted.

DevOps is dying? As in the job market for DevOps is actually shrinking?

I'd rather think that the need for Ops is growing much faster than there is supply. It is a primarily knowledge-intense profession where you can't just hire some untrained professionals and let them hack together an app. Or cannibaliize data-analystis and make them build some backend.

That implies the reverse! That DevOps is hotter than ever.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#69

Post by surfingdino »

threatofrain wrote: post #68
DarkNova6 wrote: post #66

Earlier quoted context omitted.

I'd rather think that the need for Ops is growing much faster than there is supply. It is a primarily knowledge-intense profession where you can't just hire some untrained professionals and let them hack together an app. Or cannibaliize data-analystis and make them build some backend.

That implies the reverse! That DevOps is hotter than ever.

No. Ops is back. DevOps has just not worked in practice over the last decade as clients had let go of ops and dumped all work onto devs. We will be seeing an increase in demand for ops + cloud + k8s skills, which is the right direction to go into. We will also be seeing a significant move away from the cloud and you need old-school ops skills for that. Not necessarily for deploying, but definitely for configuring server and networks and debugging performance issues.

Re: DevOps Isn't Dead, but It's Not in Great Health Either

#70

Post by DarkNova6 »

surfingdino wrote: post #69
threatofrain wrote: post #68

Earlier quoted context omitted.

That implies the reverse! That DevOps is hotter than ever.

No. Ops is back. DevOps has just not worked in practice over the last decade as clients had let go of ops and dumped all work onto devs. We will be seeing an increase in demand for ops + cloud + k8s skills, which is the right direction to go into. We will also be seeing a significant move away from the cloud and you need old-school ops skills for that. Not necessarily for deploying, but definitely for configuring ser…

Wait. Not every company needs to work like Google Or Netflix? (sorry, couldn't resist). Ops-knowledge is so much in demand that anybody with an actual "DevOps" role has done Ops nearly exclusively.

In all seriousness, this is what I am seeing already. Often I see a platform team providing an Openshift for developers to rule as they want. There is cooperation on some levels and a technical architect who is setting standards.

And most companies don't want to pay the price tag that comes with an actually good DevOps guy anyway.

Post reply on HN