Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

System Initiative: Second Wave DevOps

systeminit.com

Post reply on HN
Print view View original

61–70 of 80 posts

Re: System Initiative: Second Wave DevOps

#61

Post by JohnMakin »

jedberg wrote: post #42
JohnMakin wrote: post #32

Earlier quoted context omitted.

Well said. Click ops is the root of all evil, IME, unless you're a very small shop.

Click ops is great for most shops, as long as it has advanced configs available and you have at least one expert on staff for when those configs are needed. At Netflix our goal was always to build tools where the majority of devs just check into source control and click a few buttons, but could go as far as configuring kernel tunables if necessary (but also making that as unnecessary as possible).

But the infrastructure underneath the stuff the devs use was almost certainly not click ops’d. I’m fine clicking a deployment into prod, I am not fine making advanced cloud or build configurations via a UI

Re: System Initiative: Second Wave DevOps

#62

Post by erilbeth »

It is never about the tools. it is the people who creates problems. examples in my case: * A manager whose dev team cannot meet any schedule is starting to blame devops. * A dev architect implements a brand new bleeding edge tech stack and ranaway, devops people have to wake up every night to fix poor-designed software. * A boss who doesn't hire a sec team, wants pci-dss passed, guess who's gonna do. * A team lead who abuses you, calls the company who hired you and tells them don't hire it, it is useless.

I've been working as a devops engineer for 6 years. I'm done. I quit and never gonna do this job again. Good luck with creating more complexity with tools.

Re: System Initiative: Second Wave DevOps

#63

Post by azanar »

azanar wrote: post #60
chologrande wrote: post #7

After reading this post, I've browsed the site. I'm not sure how this is anything but significantly worse than the current model? I've been around long enough to know that any "no code" style interface or GUI are typically the _problem_ not the solution. Regardless of the code they export, you end up with fat fingers, misclicks, forgotten UI paths to follow... Taking a software eng approach to shipping infra is a sta…

I don't see the current model as a single model, but as several models interrelating. In software, there are at least three models I can think of immediately: code, configuration and user data. Why do I separate configuration? Isn't configuration just code or data? I don't think it is. It is data _about_ a particular system, as opposed to a particular user. Why the distinction here? The code of a system can be design…

To make this concrete: terraform for me has been part code, part configuration.

I define a resource, and provide a whole set of knobs on that resource. That's the code part. I test that code against a variety of configurations, the same way I might unit test application code against a variety of app configurations. I also verify that changing knobs from one setting to another behaves. With automated testing, this actually isn't all that hard to do. Once I've verified things work right, I deploy.

At this point, I will default to trusting that things will work. This is the configuration part. Set these knobs to whatever permitted value you want, and the system will update behavior based on those new values. Most of the time, things like this work. That is good enough for me.

Re: System Initiative: Second Wave DevOps

#64

Post by chologrande »

dipperdottydoo wrote: post #41
chologrande wrote: post #19

Earlier quoted context omitted.

I'm definitely not google scale, but we're global, in over 300 cities spanning ~30 countries. On an avg day we process well over 25k rps on multiple services. Simple architecture and IaC like terraform is exactly how we manage the dependencies. It's the solution, not the problem.

You think you have simple architecture when you’ve introduced Terraform to what is, based on your statistics, a two server use case. A PlayStation is capable of 25 kRPS and probably its data iops, too. Buy another one and you’re HA. You’re trapped in the complexity of the method and think you’ve achieved nirvana. This comment reminds me of those demos when Hadoop was the rage, where people would do a $4 million Hadoo…

You're assuming you know the use case. We do more than serve LAMP stack. It takes more than a few playstations I can assure you.

Re: System Initiative: Second Wave DevOps

#65

Post by thephyber »

midasuni wrote: post #45
chologrande wrote: post #19

Earlier quoted context omitted.

I'm definitely not google scale, but we're global, in over 300 cities spanning ~30 countries. On an avg day we process well over 25k rps on multiple services. Simple architecture and IaC like terraform is exactly how we manage the dependencies. It's the solution, not the problem.

25krps? As in requests per second? I.e one request every 3 seconds?

Did you confuse “requests per second” (rps) with requests per day?

Re: System Initiative: Second Wave DevOps

#66

Post by dipperdootydoo »

chologrande wrote: post #64
dipperdottydoo wrote: post #41

Earlier quoted context omitted.

You think you have simple architecture when you’ve introduced Terraform to what is, based on your statistics, a two server use case. A PlayStation is capable of 25 kRPS and probably its data iops, too. Buy another one and you’re HA. You’re trapped in the complexity of the method and think you’ve achieved nirvana. This comment reminds me of those demos when Hadoop was the rage, where people would do a $4 million Hadoo…

You're assuming you know the use case. We do more than serve LAMP stack. It takes more than a few playstations I can assure you.

No, I don’t need to know the use case. There are a vanishingly small number of use cases that cannot be performed 25,000 times per second on hardware from 25 years ago. If you indeed work on one of those few cases, you wouldn’t simultaneously call Terraform and public utility cloud simple architecture for any use case relevant to Hacker News discussion; that’s just plainly false beyond a certain level of computing depth, i.e., after you’ve written a process scheduler in an operating system or a supercomputer. (Note that I’m not calling you inexperienced. I’m talking about exposure to diverse types of computing, or, more realistically, the papers those communities develop.)

Those two ideas, that 25k is hard and Terraform is easy, are incongruous positions to hold from my perspective and basically prove the point I made. I understand if that’s not as obvious to you. The Web and cloud trap people into believing the world you’re living in is computing, and that the computers you’re working with go a certain speed on the road. There’s a lot of infrastructure in between you and computing in the model you’re working in, and it’s not apparent to you as unnecessary to compute. Computers are capable of far, far, far more than the entire industry thinks. That’s why those Hadoop takedown demos made me smile back in the day, and why I can’t wait to demo against $10 million of Kubernetes eating companies of the future alive.

Or yeah, blow my mind with your workload that can’t be tackled in a few shakes of a PlayStation’s tail with strong vector units nearby (the reason I specifically mentioned a PlayStation).

Re: System Initiative: Second Wave DevOps

#68

Post by nathants »

what makes infra hard is the large delta between what you think exists and what actually exists. the larger the delta, the worse everything gets.

the only mitigation to this is less. less tooling, less infra, less abstractions. you want that delta approaching zero as uptime goes to infinity.

i’m not sure how replacing walls of code with walls of yaml of walls of gui graphs change anything at all.

i suppose it’s possible some paradigm leap in infra understandability is hiding in crazy nextgen ui/ux, but i’m not holding my breath.

the last leap i encountered was moving from the python sdk to the go sdk for manipulating aws. this was significant, but still more of a qol improvement than something fundamental to the solution space.

Re: System Initiative: Second Wave DevOps

#69

Post by holoway »

trabant00 wrote: post #56

Another attempt by Dev to make Ops a commodity? I was there when Microsoft announced ~20 years ago they're doing away with sysadmins with a GUI. Then Suse tried as well if I remember correctly. The conferences erupted with anger and booing, I was shaking my head and laughing. Then came the great YAML plague and we had to give up our title and general purpose languages in favor of silly names, templates and DSLs. But…

Nothing about SI is intended to make ops a commodity. This is a power tool built by Ops people.

Re: System Initiative: Second Wave DevOps

#70

Post by 5Qn8mNbc2FNCiVV »

dipperdootydoo wrote: post #66
chologrande wrote: post #64

Earlier quoted context omitted.

You're assuming you know the use case. We do more than serve LAMP stack. It takes more than a few playstations I can assure you.

No, I don’t need to know the use case. There are a vanishingly small number of use cases that cannot be performed 25,000 times per second on hardware from 25 years ago. If you indeed work on one of those few cases, you wouldn’t simultaneously call Terraform and public utility cloud simple architecture for any use case relevant to Hacker News discussion; that’s just plainly false beyond a certain level of computing de…

I didn't know I could do 25k database requests per second one a PlayStation.
Post reply on HN