Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Server BMCs can need to be rebooted every so often

utcc.utoronto.ca

Post reply on HN
Print view View original

51–60 of 96 posts

Re: Server BMCs can need to be rebooted every so often

#51

Post by ilyt »

dijit wrote: post #22
arcticgeek wrote: post #15

Earlier quoted context omitted.

We have various Supermicro boards in production at work with BMCs from 2018 or so. The ATEN iKVM on them works just fine with a recent OpenJDK 11 and OpenWebStart. I’ve found that all the features work including mounting ISOs and doing remote upgrades. No need to whine about installing anything ancient or spreading ridiculous Java FUD.

Happy for you. Sysadmin for 15 years here though, Java was always a problem. The version wasn't always the worst bit, but it was always an exercise in frustration. Mostly security controls to blame, but I have had so many issues across so many systems that I cannot stand by and let you claim this is FUD about Java. the .NET applets are also a problem (because who has a compatible IE version?), but they worked more co…

I keep VM with old java/ff precisley for some old shitty servers kvms

Re: Server BMCs can need to be rebooted every so often

#52

Post by dsr_ »

sweetjuly wrote: post #32
iforgotpassword wrote: post #20

A colleague from a different department was managing a fleet of servers doing a lot of computing. Their BMCs would just stop working after a few weeks of uptime, every time. He was annoyed by it but mostly ignored it as it was mostly just used for additional monitoring. The power outlets in the racks were remote controllable, so you could still hard reset a server if this was required. However, as this fleet was used…

I wonder what the BMC was doing to hurt performance. Maybe it was questionable DVFS that stopped when the BMC died? Poor fan management that caused thermal throttling?

That's a pretty good guess: lots of BMCs have fan control, and turning it off should make all fans stay at max power.

In a datacenter the odds of anyone noticing that the fans are always on high is practically nil, unless you are specifically monitoring fan RPM. Most folks don't bother, as what you are actually interested in is temperature.

Re: Server BMCs can need to be rebooted every so often

#53

Post by ilyt »

chenxiaolong wrote: post #16

BMCs have to be some of the most unreliable devices that I've worked with. Some of the issues I encountered at my last job: * [ASRock BMC] The BMC firmware updater sometimes causes the NICs to get stuck in a bad state where every NIC has the same MAC address. This can be resolved via a proprietary UEFI application for reflashing the correct MAC address. * [Dell iDRAC] Local authentication randomly stops working due t…

> * [Supermicro BMC] Various IPMI commands just lie about successful execution. For example, setting the asset tag via the FRU succeeds, but has no effect. Those commands require toggling a write lock bit via an OEM command, which I only found by reverse engineering. Other commands, like the set asset tag DCMI command, leave the data in a corrupted state until a BMC reboot.

we have Supermicro server with serial flashed to be something like 12345678 which means it probably doesn't work reliably on their production line either lmao

Re: Server BMCs can need to be rebooted every so often

#54

Post by ilyt »

gnufx wrote: post #41
isopede wrote: post #18

A few of my recent systems have come with a built-in combo BMC on the motherboard NICs. I haven't seen this before, is it a new trend? I'm imagining they put in a switch in there with the NICs? How does this even work? I'm doing a few timing sensitive projects involving hardware timestamping in the NICs. Does this mess with say timing variability? I've disabled the BMCs out of paranoia but I don't know the topology i…

That's common, and something to avoid. You want a physically separate BMC network for security and reliability.

One example: on some servers turning off NIC will... turn off the port completely, including shared BMC connection hanging off it.

Which happens during reboot meaning you won't catch the boot screen on KVM if you use shared nic

Re: Server BMCs can need to be rebooted every so often

#55

Post by tinus_hn »

fulafel wrote: post #45

People are surprisingly chill about BMCs as attack surface, eg Intel management features get a lot more scrutiny.

An important distinction is that one is a device you choose to add to the system, and the other is an offer you can’t refuse.

Re: Server BMCs can need to be rebooted every so often

#56

Post by acranox »

dveeden2 wrote: post #6

There are too many names for the BMCs, even within a single vendor. BMC, ILO, LOM, DRAC, iDRAC, etc And the worst are those that use java applets or webstart and require a ancient java version.

Even worse was one of mine last year that needed Flash. Apparently we neglected to update it. I can handle ancient Java, but trying to get Flash setup was going to be futile, so I just went to the data center.

Re: Server BMCs can need to be rebooted every so often

#58

Post by rtp4me »

kube-system wrote: post #8
dveeden2 wrote: post #6

There are too many names for the BMCs, even within a single vendor. BMC, ILO, LOM, DRAC, iDRAC, etc And the worst are those that use java applets or webstart and require a ancient java version.

I am honestly surprised how bad many of these are, and in production, no less. I recently set up a supermicro system and spent a whole day just trying to figure out what to install to get the stupid ancient Java crap to load so I could mount an ISO.

I keep an old Windows XP VM around with Java 6 just for this exact reason.

Re: Server BMCs can need to be rebooted every so often

#59

Post by digitallyfree »

acranox wrote: post #56
dveeden2 wrote: post #6

There are too many names for the BMCs, even within a single vendor. BMC, ILO, LOM, DRAC, iDRAC, etc And the worst are those that use java applets or webstart and require a ancient java version.

Even worse was one of mine last year that needed Flash. Apparently we neglected to update it. I can handle ancient Java, but trying to get Flash setup was going to be futile, so I just went to the data center.

That was an old Cisco server, right? I think those models still require Flash even if they're fully updated, and people have to use VMs with Flash installed to access the BMC.
Post reply on HN