Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Big DevOps Misunderstanding

wolfoliver.medium.com

Post reply on HN
Print view View original

81–90 of 321 posts

Re: The Big DevOps Misunderstanding

#81

Post by chucky_z »

manish_gill wrote: post #66
lukevp wrote: post #64

Earlier quoted context omitted.

If you haven’t heard about cdktf, check it out! It lets you do infrastructure as code but in full programming languages like Python, C# and TypeScript. It’s by hashicorp, and under the covers it runs Terraform, so you have the whole set of terraform providers at your disposal.

Ah, CDKTF seems to be the Pulumi equivalent for Terraform. I think I remember seeing it. We're far too invested into Terraform at this point to learn yet another tool. Oh well. Hopefully the next time I have to implement something like this, K8s ecosystem would also have similar tools (or maybe I just missed something)

There’s a straight up conversion tool iirc, and you can do things like include hcl as well I believe. Cdktf is just terraform at the bottom.

Re: The Big DevOps Misunderstanding

#82

Post by ludamad »

manish_gill wrote: post #54

I had to basically give up my full time role as a Data Engineer who is supposed to solve "big data" problems (Storage, Streaming, Machine Learning pipelines) etc, and had to solely focus on DevOps because there was no other expert in the team. We (a colleague and I) basically learned Terraform, then Kubernetes, and then Helm in an extremely stressful environment. I wouldn't recommend that to most people. (Also, I can…

I think the YAML issues center around a simple fact, few are taking e.g. 1000+ line YAML (or YAML for X) dev seriously. Complex YAML becomes code, and requires as much e.g. refactoring, idioms, reuse, static checking. At least it's not terribly hard to make a script output it as a format

Re: The Big DevOps Misunderstanding

#83

Post by coolgoose »

I am so tired of devops as a term. It's a stupid bandaid solution.

Infrastructure, regardless if you use a paas or not, is a tough cookie to crack and deserves a normal sysadmin role.

Monitoring, resource utilization, network management, library updates, costs among a few are things you need to master, observe and proactively manage, not keep it as an afterthought since it magically just works.

Re: The Big DevOps Misunderstanding

#84

Post by mschuster91 »

darepublic wrote: post #11
mschuster91 wrote: post #6

Can we please get rid of the DevOps hype train? What gets completely ignored is that there is a reason that sysops has been a specialized role for decades. How often do we get threads here that people get their AWS accounts hijacked because of some credential leak (e.g. access keys in frontend builds), extreme expenses due to stupid autoscaling settings combined with a DoS attack or content leaks out of insecure s3 b…

Access keys on frontend is the fault of the frontend developer or culture

There will always be irresponsible or junior people, the problem is that AWS allows extremely unsafe things in the first place.

Normal hosters wouldn't even open internal infra like DB instances, mail relays or storage up to the Internet, AWS by default doesn't even firewall their internal stuff off and thinks, as long as you have the right credentials, you can't be wrong, even if the IP is not from eu-central-1 but some dial-up in Mexico any more. S3 and SES have mind-bogglingly insecure defaults, and it is hard to wall off your SES credentials enough that they're useless to an attacker who has managed to exfiltrate them (and that one is easy enough, all an attacker needs is a vulnerable log4j and the dubious "best practice" of passing credentials via environment variables).

It is so incredibly easy to mess up on AWS it hurts. AWS is a gun shop that not just sells handguns but nuclear armed bomber jets to anyone presenting a credit card. No knowledge checks, training, nothing required, and you have the firepower of all of AWS on your hands.

Re: The Big DevOps Misunderstanding

#85

Post by StreamBright »

manish_gill wrote: post #75
StreamBright wrote: post #70

Earlier quoted context omitted.

You are on the right track. I have exactly the same story, data engineer, working on automation a lot. YAML, JSON and HCL should be purged from IaC asap. Luckily Dhall can be integrated everywhere we need YAML otherwise we would go nuts. The bigges failure of k8s is to understand that YAML does not scale to teams and beyond 100 lines, especially not good to nest it (hello ArgoCD). A type safe not accidentally Turing…

Dhall looks interesting. Thanks! I went into the rabbithole of terraform/HCL and came up with Terragrunt to do dynamic code generation. Combining it together with generous use of `make` keeps me sane, but it's not a long term solution. Even now, there are things within the system where the boundaries should be separate, but they aren't. We literally got exhausted and burned out writing our automation, and decided to…

> But Reading/Debugging kubernetes world is a dreadful experience.

Exactly. It is abstraction, over abstraction, over abstraction for no obvious benefits.

Re: The Big DevOps Misunderstanding

#86

Post by jedberg »

Devops is a philosophy not a role. The title DevOps engineer is meaningless.

There are software engineers, who write software and then deploy it onto self serve infrastructure, and there are operations engineers and data engineers who build and maintain the systems that the software engineers use to deploy and operate their software.

Usually when someone calls themselves a DevOps engineer they are an operations engineer, but sometimes they are a software engineer at a small company who also has to maintain their own infrastructure.

Re: The Big DevOps Misunderstanding

#87

Post by laumars »

manish_gill wrote: post #54

I had to basically give up my full time role as a Data Engineer who is supposed to solve "big data" problems (Storage, Streaming, Machine Learning pipelines) etc, and had to solely focus on DevOps because there was no other expert in the team. We (a colleague and I) basically learned Terraform, then Kubernetes, and then Helm in an extremely stressful environment. I wouldn't recommend that to most people. (Also, I can…

DevOps specialist here and I agree with your complaints about IaC meaning JSON and YAML documents. sigh

Re: The Big DevOps Misunderstanding

#88

Post by cortesoft »

shrimpx wrote: post #62
cortesoft wrote: post #17

As someone who has lead a devops team, and still has a devops team reporting to me, I have also struggled with the idea of having a team called devops, since it does go against the devops philosophy of devs owning their software in production. However, in our case (and probably many others), our devops team is responsible for creating and maintaining the tools and processes that allow the dev teams to manage their sy…

Devops is having an identity crisis, but what you describe is what I call devops. It's the subset of engineering that supports engineering by defining/inferring workflows and building systems and tools to codify them. But I've seen it called ops, sysops, internal platform, and even infrastructure -- in some companies, devops and infra are the same people.

Yeah, I am not sure how typical this is, but my company has DC (data center) ops, NetOps, SysOps, and DevOps…

DC ops is in charge of the physical system, from racking and stacking to hardware replacements.

NetOps handles the networking stack, managing the routers and switches, vlans, announcements, IP block management, network access requests, etc.

SysOps handles provisioning systems, working with dc ops to get systems repaired, laying down the OS, managing the inventory system, and anything else the systems need to be ready to run our software.

Then DevOps is responsible for all the software that our company writes, and managing how developers get their software onto the machines, how they configure the software, and how they interact with their software in production.

The lines are obviously not that clean cut, but that is the basic idea.

Re: The Big DevOps Misunderstanding

#89

Post by 988747 »

sharken wrote: post #19

Developers typically care about developing the business side of the application and have little interest in the rest. Actually, it can even be split in three, where developers are done when the Pull Request is done. Deployment, tooling and IaC is then handled by developers who specialize in this area. A third group is then responsible for audit and security on the chosen platform, e.g. Azure.

>> Developers typically care about developing the business side of the application and have little interest in the rest.

Correction: developers care about using cool new programming languages and frameworks. "Business side of the application" has to be forced on them by business analysts and scrum masters :P

Re: The Big DevOps Misunderstanding

#90

Post by topochico3 »

lukevp wrote: post #64
manish_gill wrote: post #54

I had to basically give up my full time role as a Data Engineer who is supposed to solve "big data" problems (Storage, Streaming, Machine Learning pipelines) etc, and had to solely focus on DevOps because there was no other expert in the team. We (a colleague and I) basically learned Terraform, then Kubernetes, and then Helm in an extremely stressful environment. I wouldn't recommend that to most people. (Also, I can…

If you haven’t heard about cdktf, check it out! It lets you do infrastructure as code but in full programming languages like Python, C# and TypeScript. It’s by hashicorp, and under the covers it runs Terraform, so you have the whole set of terraform providers at your disposal.

I don’t understand the appeal of CDK, it’s the most difficult to understand overly complex part of IaaC I have ever come across.

I want simple yaml/hcl that is easy to understand and write. I don’t need to feel smart writing TF, I need it to be easy to understand to reduce MTTR when things go wrong.

Post reply on HN