Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We might want to regularly keep track of how important each server is

utcc.utoronto.ca

Post reply on HN
Print view View original

31–40 of 147 posts

Re: We might want to regularly keep track of how important each server is

#31

Post by jsjohnst »

tonyarkles wrote: post #18
cowsandmilk wrote: post #5

It is interesting to contrast with where the wider industry has gone. Industry: don’t treat your systems like pets. Author: proudly declares himself Unix herder, wants to keep track of which systems are important.

I’m absolutely loving the term Unix herder and will probably adopt it :) I’m generally with you and the wider industry on the cattle-not-pets thing but there are a few things to keep in mind in the context of a university IT department that are different than what we regularly talk about here: - budgets often work differently. You have a capex budget and your institution will exist long enough to fully depreciate the…

> moved everything to a big vSphere cluster, and backed it by a giant RAID-5 array

I’m with sibling commenter, if said IT department genuinely thought that the core point in “cattle-not-pets” was met by their single SuperMegaCow, then they missed the point entirely.

Re: We might want to regularly keep track of how important each server is

#32

Post by marcosdumay »

b112 wrote: post #16
luma wrote: post #12

Earlier quoted context omitted.

When you are responsible for the full infrastructure, sequencing power down and power on in coordination with your UPS is a common solution. Network gear needs a few minutes to light up ports, core services like DNS and identity services might need to light up next, then storage, then hypervisors and container hosts, then you can actually start working on app dependencies. This sort of sequencing leads itself natural…

Double? Try 100x!!

Exactly. If it was double, it would be a non-brainier.

Re: We might want to regularly keep track of how important each server is

#34

Post by cookiemonster9 »

throw0101a wrote: post #7
cowsandmilk wrote: post #5

It is interesting to contrast with where the wider industry has gone. Industry: don’t treat your systems like pets. Author: proudly declares himself Unix herder, wants to keep track of which systems are important.

> Author: proudly declares himself Unix herder, wants to keep track of which systems are important. Because not all environments are webapps which dozens or hundreds of systems configured in a cookie cutter matter. Plenty of IT environments have pets because plenty of IT environments are not Web Scale™. And plenty of IT environments have staff churn and legacy systems where knowledge can become murky (see reference a…

So much this. Not everything fits cattle/chicken/etc models. Even in cases where those models could fit, they are not necessarily the right choice, given staffing, expertise, budgets, and other factors.

Re: We might want to regularly keep track of how important each server is

#35

Post by ethbr1 »

spydum wrote: post #11

Asset management is definitely a thing. Tag your environments, tag your apps, and provide your apps criticality ratings based on how important they are to running the business. Then it's a matter of a query to know which servers can be shut, and which absolutely must remain.

> provide your apps criticality ratings based on how important they are to running the business

In a decentralized, self-service model, you can add "deal with convincing a stakeholder their app is anything less than most-critical."

Although it usually works itself out if higher-criticality imposes ongoing time commitments on them as well (aka stick).

Re: We might want to regularly keep track of how important each server is

#36

Post by datadrivenangel »

logifail wrote: post #30
jodrellblank wrote: post #28

Earlier quoted context omitted.

At the risk of no true Scotsman, that doesn’t sound like “cattle not pets“; when the cattle are sent to the slaughterhouse there isn’t any blast radius, there’s just more cattle taking over. You explicitly don’t have to replace them with exact clones of the original cattle from tape very slowly, you spin up a herd of more cattle in moments.

> when the cattle are sent to the slaughterhouse Data isn't "sent to the slaughterhouse". Ever. Data can be annoying that way.

The true problem with kubernetes and modern cloud. Not insurmountable, but painful when your data is large compared to your processing needs.

Re: We might want to regularly keep track of how important each server is

#37

Post by LeonB »

throw0101a wrote: post #7
cowsandmilk wrote: post #5

It is interesting to contrast with where the wider industry has gone. Industry: don’t treat your systems like pets. Author: proudly declares himself Unix herder, wants to keep track of which systems are important.

> Author: proudly declares himself Unix herder, wants to keep track of which systems are important. Because not all environments are webapps which dozens or hundreds of systems configured in a cookie cutter matter. Plenty of IT environments have pets because plenty of IT environments are not Web Scale™. And plenty of IT environments have staff churn and legacy systems where knowledge can become murky (see reference a…

I work with one system where the main entity is a “sale” which is processed beginning to end in some fraction of a second.

A different system I work with, the main “entity” is, conceptually, more like a murder investigation. Less than 200 of the main entity are created in a year. Many different pieces of information are painstakingly gathered and tracked over a long period of time, with input from many people and oversight from legal experts and strong auditing requirements.

Trying to apply the best lessons and principles from one system to the other is rarely a good idea.

These kind of characteristics of different systems make a lot of difference to their care and feeding.

Re: We might want to regularly keep track of how important each server is

#39

Post by throw0101b »

tonyarkles wrote: post #18
cowsandmilk wrote: post #5

It is interesting to contrast with where the wider industry has gone. Industry: don’t treat your systems like pets. Author: proudly declares himself Unix herder, wants to keep track of which systems are important.

I’m absolutely loving the term Unix herder and will probably adopt it :) I’m generally with you and the wider industry on the cattle-not-pets thing but there are a few things to keep in mind in the context of a university IT department that are different than what we regularly talk about here: - budgets often work differently. You have a capex budget and your institution will exist long enough to fully depreciate the…

> You have a capex budget […]

As someone who has worked IT in academia: no, you do not. :)

Re: We might want to regularly keep track of how important each server is

#40

Post by chris_wot »

jodrellblank wrote: post #28
tonyarkles wrote: post #18

Earlier quoted context omitted.

I’m absolutely loving the term Unix herder and will probably adopt it :) I’m generally with you and the wider industry on the cattle-not-pets thing but there are a few things to keep in mind in the context of a university IT department that are different than what we regularly talk about here: - budgets often work differently. You have a capex budget and your institution will exist long enough to fully depreciate the…

At the risk of no true Scotsman, that doesn’t sound like “cattle not pets“; when the cattle are sent to the slaughterhouse there isn’t any blast radius, there’s just more cattle taking over. You explicitly don’t have to replace them with exact clones of the original cattle from tape very slowly, you spin up a herd of more cattle in moments.

Analogies always break down under scrutiny. Any cattle farmer would find “spinning up a herd of cattle” to be hilarious.
Post reply on HN