Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bpftune uses BPF to auto-tune Linux systems

github.com

Post reply on HN
Print view View original

21–30 of 78 posts

Re: Bpftune uses BPF to auto-tune Linux systems

#21

Post by admax88qqq »

pbhjpbhj wrote: post #20
gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

>"bpftune logs to syslog so /var/log/messages will contain details of any tuning carried out." (from OP GitHub readme) The rmem example seems to allay fears that it will make changes one can't reverse.

It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.

Re: Bpftune uses BPF to auto-tune Linux systems

#22

Post by sgarland »

gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

Yes, it is. IMO, except for learning (which should not be done in prod), you shouldn’t make changes that you don’t understand.

The tools seems to mostly tweak various networking settings. You could set up a test instance with monitoring, throw load at it, and change the parameters the tool modifies (one at a time!) to see how it reacts.

Re: Bpftune uses BPF to auto-tune Linux systems

#24

Post by sgarland »

usr1106 wrote: post #3

Interesting. But if tuning parameters to their best values were easy, shouldn't the kernel just do that in the first place?

I’d rather the kernel present a good-enough but extremely stable set of configs. If I’m using a distro like Arch or Gentoo, then sure, maybe run wild (though both of those would probably assume I’m tuning them anyway), but CentOS, Debian, et al.? Stable and boring. If you change something, you’d better know what it is, and why you’re doing it.

Re: Bpftune uses BPF to auto-tune Linux systems

#25

Post by sgarland »

mrbluecoat wrote: post #17

> bpftune is designed to be zero configuration; there are no options On behalf of every junior administrator, overworked IT admin, and security-concerned "cattle" wrangler, thank you. Having to learn a thousand+ knobs & dials means most will never be touched. I for one welcome automated assistance in this area, even if the results are imperfect.

I think it’s still important to know what those dials and knobs do, otherwise (as the currently top-voted comment says) when things break, you’ll be lost.

Re: Bpftune uses BPF to auto-tune Linux systems

#26

Post by nine_k »

sgarland wrote: post #22
gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

Yes, it is. IMO, except for learning (which should not be done in prod), you shouldn’t make changes that you don’t understand. The tools seems to mostly tweak various networking settings. You could set up a test instance with monitoring, throw load at it, and change the parameters the tool modifies (one at a time!) to see how it reacts.

I'd run such a tool on prod in "advice mode". It should suggest the tweaks, explaining the reasoning behind them, and listing the actions necessary to implement them.

Then humans would decide if they want to implement that as is, partly, modified, or not at all.

Re: Bpftune uses BPF to auto-tune Linux systems

#27

Post by sgarland »

nine_k wrote: post #26
sgarland wrote: post #22

Earlier quoted context omitted.

Yes, it is. IMO, except for learning (which should not be done in prod), you shouldn’t make changes that you don’t understand. The tools seems to mostly tweak various networking settings. You could set up a test instance with monitoring, throw load at it, and change the parameters the tool modifies (one at a time!) to see how it reacts.

I'd run such a tool on prod in "advice mode". It should suggest the tweaks, explaining the reasoning behind them, and listing the actions necessary to implement them. Then humans would decide if they want to implement that as is, partly, modified, or not at all.

Fair point, though I didn’t see any such option with this tool.

Re: Bpftune uses BPF to auto-tune Linux systems

#28

Post by nine_k »

sgarland wrote: post #27
nine_k wrote: post #26

Earlier quoted context omitted.

I'd run such a tool on prod in "advice mode". It should suggest the tweaks, explaining the reasoning behind them, and listing the actions necessary to implement them. Then humans would decide if they want to implement that as is, partly, modified, or not at all.

Fair point, though I didn’t see any such option with this tool.

It's developed in the open; we can create Github issue.

Actually https://github.com/oracle/bpftune/issues/99

Re: Bpftune uses BPF to auto-tune Linux systems

#29

Post by trelliscoded »

gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

If your staging doesn’t do capacity checks in excess of what production sees, yes.

Re: Bpftune uses BPF to auto-tune Linux systems

#30

Post by Twirrim »

gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

Disclaimer: I work for Oracle, who publish this tool, though I have nothing to do with the org or engineers that created it

I've been running this for a while on my laptop. So far yet to see any particular weirdness, but also I don't know that I can state with any confidence it has a positive impact either. I've not carried out any benchmarks in either direction.

It logs all changes that it's going to make including what they were on before. Here's an example from my logs:

    bpftune[1852994]: Scenario 'need to increase TCP buffer size(s)' occurred for tunable 'net.ipv4.tcp_rmem' in global ns. Need to increase buffer size(s) to maximize throughput
    bpftune[1852994]: Due to need to increase max buffer size to maximize throughput change net.ipv4.tcp_rmem(min default max) from (4096 131072 7864320) -> (4096 131072 9830400)
Post reply on HN