Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Server BMCs can need to be rebooted every so often

utcc.utoronto.ca

Post reply on HN
Print view View original

71–80 of 96 posts

Re: Server BMCs can need to be rebooted every so often

#71

Post by andrewjf »

p_l wrote: post #43
dveeden2 wrote: post #6

There are too many names for the BMCs, even within a single vendor. BMC, ILO, LOM, DRAC, iDRAC, etc And the worst are those that use java applets or webstart and require a ancient java version.

This is one of the reasons I like to architect networks for netbooting (so no remote media needed) plus force every physical server to boot UEFI-only - because UEFI supports serial console properly, unlike BIOS, so I can just use IPMI Serial-over-LAN support. Combination of those two generally removes the need for any of the advanced features that required custom clients or even a Web browser

For sure - we have some networks that when a host netboots it always goes to something like http://netboot.xyz with serial console by default.

My favorite is some vendors using COM1 and some using COM2 so you have no idea which it is ahead of time.

Re: Server BMCs can need to be rebooted every so often

#72

Post by acranox »

digitallyfree wrote: post #59
acranox wrote: post #56

Earlier quoted context omitted.

Even worse was one of mine last year that needed Flash. Apparently we neglected to update it. I can handle ancient Java, but trying to get Flash setup was going to be futile, so I just went to the data center.

That was an old Cisco server, right? I think those models still require Flash even if they're fully updated, and people have to use VMs with Flash installed to access the BMC.

Yep. I think you may be right, it’s EOL and probably doesn’t have any more updates available. I have a VM for when I need old Java, but I was going to need an older VM to run Flash, and that just wasn’t how I wanted to spend my time. :D

Re: Server BMCs can need to be rebooted every so often

#73

Post by sys_64738 »

> Server BMCs are little computers running ancient versions of Linux with software that's probably terribly written and they stay running forever, which means all sorts of opportunities for slow bugs. Reboot away!

Such a silly comment. If your BMC is updated then it will use a recent version of Linux.

Re: Server BMCs can need to be rebooted every so often

#75

Post by gnufx »

ilyt wrote: post #54
gnufx wrote: post #41

Earlier quoted context omitted.

That's common, and something to avoid. You want a physically separate BMC network for security and reliability.

One example: on some servers turning off NIC will... turn off the port completely, including shared BMC connection hanging off it. Which happens during reboot meaning you won't catch the boot screen on KVM if you use shared nic

What I remember was losing it just at a crucial point in the PXE boot process. You're also dependent on the same switches as the main net, and have to rely on fragile firewalling or VLANs to restrict access to the BMCs.

Re: Server BMCs can need to be rebooted every so often

#76

Post by sys_64738 »

somat wrote: post #35

As a big openbsd aficionado I always wish I could just run bog standard openbsd on my bmc's. ipmi does not do much for me so I would be fine if it were missing a good opensource ipmi stack. Most people would just want to run linux, but the point is, the bmc is just a small computer in charge of monitoring your big computer, however the bmc usually is unable to run an off the shelf operating system, so it is running a…

BMCs generally run Linux and monitor the SoC functionality that the hardware is designed around. You need a vendor specific software stack for the hardware monitoring. The Rasp Pi is a toy.

Re: Server BMCs can need to be rebooted every so often

#77

Post by fulafel »

sys_64738 wrote: post #74
fulafel wrote: post #45

People are surprisingly chill about BMCs as attack surface, eg Intel management features get a lot more scrutiny.

You should never keep the BMC plugged into a network. Security 101.

That view is what I meant by my comment.

IMO it's wrong - It's terrible product design that servers have network ports that will cause catastrophic failures unless carefully only connected to a special expertly secured un-network for fragile things.

Re: Server BMCs can need to be rebooted every so often

#78

Post by lapinot »

klysm wrote: post #63
amluto wrote: post #30

Earlier quoted context omitted.

Can a mere mortal usefully deploy openbmc on a small number of servers?

I was looking into this recently. It doesn’t seem like you can easily get your hands on BMC hardware. There is one project I found where they’re using an FPGA and everything is open source but it still looked far from easy.

Afaik the problem is typically not hardware but software, since most server motherboards come with a bmc chip on it. Are you designing a motherboard? All i would ask for a bmc is root access. But now everyone (but the big players, who design their own motherboards) is basically stuck with a stupid embedded linux with shitty software that has half-backed features they don't even need, but now need to care about. When all you'd need would be a way to access a host serial console, read sensors, control boot sequence and perhaps re-flash the bios, ie most likely just basic interactions on serial interfaces.

Re: Server BMCs can need to be rebooted every so often

#79

Post by rbanffy »

lapinot wrote: post #78
klysm wrote: post #63

Earlier quoted context omitted.

I was looking into this recently. It doesn’t seem like you can easily get your hands on BMC hardware. There is one project I found where they’re using an FPGA and everything is open source but it still looked far from easy.

Afaik the problem is typically not hardware but software, since most server motherboards come with a bmc chip on it. Are you designing a motherboard? All i would ask for a bmc is root access. But now everyone (but the big players, who design their own motherboards) is basically stuck with a stupid embedded linux with shitty software that has half-backed features they don't even need, but now need to care about. When…

We got into that when we started expecting servers to have VGA and PS/2 ports because an RS-232 console would not be very useful with a GUI.

Is there a modular standard for BMCs?

Re: Server BMCs can need to be rebooted every so often

#80

Post by rbanffy »

yellow_lead wrote: post #39
userbinator wrote: post #2

This is brought to you by the BMC with a KVM-over-IP that wouldn't accept '2' entered on the (virtual) keyboard in any way or form. This is one of those things that I'd probably be willing to spend a ton of time analysing to find the root cause if I could. Dump the memory image, debug the code, and figure out why it's just that (and possibly other) keys. My guess just from the description is that a bitflip happened e…

Some people love the chase.

What's better than chasing white rabbits?
Post reply on HN