Live data from Hacker News

2023 DevOps Is Terrible

abidmoon.hashnode.dev

61–70 of 126 posts

Re: 2023 DevOps Is Terrible

#61
post #38

To me, DevOps culture just feels like a way for businesses to save money by not staffing a dedicated infrastructure team and pushing all those responsibilities onto application developers. The amount of time I've spent fiddling with Terraform, Ansible, Kubernetes manifests, Helm charts, Jenkins configuration, GitHub Actions configuration, AWS IAM, and so on over the past few years is absurd, probably more than the ti…

We have separate devops teams, though, so I do zero fiddling with all those technologies.

Re: 2023 DevOps Is Terrible

#62
post #27
post #10

Earlier quoted context omitted.

Because that is the software developer/systems engineer division that already existed 20 years ago. The idea behind devops was to be a single team.

> The idea behind devops was to be a single team. I assume you are pointing out that split-teams is the insane part, in which case I agree. The philosophy was supposed to define the blurred line between developers and operations, using tooling to help facilitate the shared responsibilities where sensible and make hand-off a breeze. As you pointed out, the division existed 20 years ago, and it still exists now at many…

[dead]

Re: 2023 DevOps Is Terrible

#63

Yuck, Portainer... when I read that I get nightmares. The problem is: when you give developers access to Kubernetes, to ECS, to Portainer, to whatever they. will. not. care. about literally anything. You'll find a hotpot of cobbled together Dockerfiles with zero provenance, with base images pinned to stuff years old, and completely inefficient layer ordering. Pipelines won't use caching or parallelism (okay, sometime…

The developers are literally paid to do the least amount of work that could possibly do towards delivering some abstract MVP "this sprint".

Actually making something that is simple and maintainable is the 7th or 8th priority of business, if it is a priority at all.

Artificial "obstacles" to delivering the artifacts in time in fact, wanted, a sort of corporation Deus ex machina, externalities to explain why they did not meet delivery deadlines last time.

Incentives are not aligned. Things will remain broken until incentives are aligned.

Re: 2023 DevOps Is Terrible

#64

Since the in house server days, technology solutions have only become more and more time consuming to implement, ID & PII obsessed, and bulky (data footprint-wise). I am thankful I remember much more simple times, when apps were tiny, run on regular computers in my office, and you could symply restore from a local backup to bring a service back online. Now we have companies paying thousands of dollars a month to host…

> Simpler solutions, conveyed and implemented in human language, will be king in 2024.

I would love this to be the case, but it seems that complex and expensive systems are too lucrative to everyone involved: CTOs get to manage big budgets and bit teams, devops people get paid large salaries to justify the CTOs salary and large service budgets, etc.

Re: 2023 DevOps Is Terrible

#65
What drives me nuts is the number of places that expect you to have deep experience with whatever CI/CD solution they're using before they'll even consider you for a position. How have we gotten to the point where deploying an application is so fucking arcane of a process that you need to have 5 years of experience with whatever acronym soup some CTO bodged together or you are considered worthless?

Re: 2023 DevOps Is Terrible

#66
post #31

Earlier quoted context omitted.

To be more blunt about it: if the person writing the code is not the one on the pager at 3am, they have no serious incentive to make it correct or resilient. This is why "you write it, you run it" is so important - the most reliable software is the software never written, but if something must be written, it's important that all incentives are aligned for it to work right. Of course, this is only at the operational e…

as a software developer, every place I've tried to work that's "you write it, you run it" turns into me just spending all my time running the code as it already existed, with actually trying to improve reliability as kind of a bonus 10% task that no one ever talks about or asks after.

It always depends on aligned incentives, for some very specific lines of work and very specific people, they will totally intentionally upkeep known bad software without improving it too much because it's somewhat a guarantee of very profitable overtime pay.

Re: 2023 DevOps Is Terrible

#67

I think the biggest issue with devops is that it originally meant "socialising" your sysadmins, by getting them to sit with your devs, so that shit didn't get lost because nobody thought to talk to the right team. But then it morphed into "oh lets innovate with infrastructure" but the innovation turned into "lol lets just restart from scratch and ignore history" Anybody who used early k8s can attest to how un product…

More a general question, but I’ve gotten pings in the past to be an SRE at a FAANG despite not doing much cloud stuff. Is it the type of role people take for a tour of duty and then transfer out of?

I imagine there are SREs who enjoy it and stay there, but curious why they’d invite someone heavy on the dev side to interview.

Re: 2023 DevOps Is Terrible

#68
post #6

Yep, bang on. As for the PE being the “solution” it is incredibly important and sorely needed but it won’t solve your enterprise culture.

Regarding enterprise culture I’m curious who will be in this “platform engineering” team and what expertise they’ll have. In (enterprise) reality it will more than likely be a bunch of sysadmins and former “network/storage guys” who saw which way the wind was blowing and got a cloud cert. They really have no business forcing a particular infra pattern on application teams because they have no application level expert…

> In (enterprise) reality it will more than likely be a bunch of sysadmins and former “network/storage guys” who saw which way the wind was blowing and got a cloud cert.

This sounds like a tale of woe. but. The fault is the culture, not the people doing the grunt work. Like devs, if you're used to nails, then you'll want to use a hammer for everything.

> gives you expertise in application design/architecture.

a sysadmin should know much better how your application runs in real life, and more importantly, how it behaves when it goes to shit.

They _should_ know more about all the different types of services (ie kafka vs NATS etc) because they will have had to set way more of them up than a typical dev. If thats not the case, you've had bad or poorly mentored juniors running your stuff.

The reason they should know, is because they have the pager to make it work at 4am.

Look, if I had it my way, and I was at a medium sized company, I would buy in 3 mainframes, in three disparate locations, set the lock step replication to "very yes" and make sure each dev used the local DB and use the job description lang, to define scale.

Pay the $1million a year to outsource reliability of DB and state. Because frankly most people can't program for either scale, redundancy or recoverability. (yes that includes k8s) let the hardware do all of that and save a bunch on staffing costs.

I work at a HUGE scale place, and yes there is a need for special sauce there. but even then not that much, and not enough to justify the massive churn in systems. Most PE is just cargo culting busy work. Stick to ECS and PSQL and just get on with making actual buisness features.

Re: 2023 DevOps Is Terrible

#69
post #6

Yep, bang on. As for the PE being the “solution” it is incredibly important and sorely needed but it won’t solve your enterprise culture.

Regarding enterprise culture I’m curious who will be in this “platform engineering” team and what expertise they’ll have. In (enterprise) reality it will more than likely be a bunch of sysadmins and former “network/storage guys” who saw which way the wind was blowing and got a cloud cert. They really have no business forcing a particular infra pattern on application teams because they have no application level expert…

[dead]

Re: 2023 DevOps Is Terrible

#70

Nobody knows anything The difficult paradoxes have not been resolved No-one can say this loudly because there is no safe space

Nobody Knows Anything

For a given level of complexity in an organisation, talk to any 10 experts on the subject in the organisation (ie how does the foobar process work) and you will find blind men describing an elephant. All technically correct but all missing important parts

The paradoxes

At some point an organisation acquired or grew an antogonistic part. Both cannot really exist but both have not been removed. Think either Google search vs Google ad sales (if you return perfect search terms each time, you get less page views hence less ad impressions. Make search worse get more ads served). Or simply other non core businesses. Pretty much every non-tech mega corp is really dozens and dozens of smaller businesses with a shared treasury so they can ride out each others ups and downs. But one or two businesses are really carrying oat of the load.

Anyway, you can find paradoxes or conflicts of interest in all businesses and badly aligned incentives hurt everyone.

Someone should fix that. Not me I don't have the power (indeed usually only the orignal centralmfounder has power to)

Safe spaces

Yeah these don't exist. If you ain't upbeat publically you are seen as trying to pull everyone else under. So how are problems acknowledged let alone fixed? Dunno.

What would really help would be free speech, open conversations and discussions based on data that can inform policy and provide impetus to chnage. we (kinda) have this in western journalism / media (including social media). But absolutely no large corporation anywhere has a open discussion area where different factions can raise their concerns. there are of course political infighting amoungst the powerful elites of the corporation but that's not the same thing - they might even be accurate about the problems but no one else knows.

Democracy matters

Sorry this was supposed to be about devops - and it kinda is. Most large corps have some devops thing going on where "this time we will fix it all for everyone" - but has there been open conversations with data supporting it from all parts of the org? oh come on.

Post reply on HN