Earlier quoted context omitted.
>"bpftune logs to syslog so /var/log/messages will contain details of any tuning carried out." (from OP GitHub readme) The rmem example seems to allay fears that it will make changes one can't reverse.
It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.
Bpftune uses BPF to auto-tune Linux systems
31–40 of 78 posts
Re: Bpftune uses BPF to auto-tune Linux systems
#32Earlier quoted context omitted.
It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.
If they can be reversed individually you can simply deduce by rolling back changes one by one, no?
Re: Bpftune uses BPF to auto-tune Linux systems
#33Re: Bpftune uses BPF to auto-tune Linux systems
#34Earlier quoted context omitted.
It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.
If they can be reversed individually you can simply deduce by rolling back changes one by one, no?
You notice that your aggregate error rate been drifting upwards since using bpftune. It turns out, in reality, there is some complex interaction between the tuning and your routers, or your TOR switches, or whatever - there is feedback that causes oscillations in a tuned value, swinging between too high and too low.
Can you see how this is not a matter of simple deduction and rollbacks?
This scenario is plausible. Autotuning generally has issues with feedback, since the overall system lacks control theoretic structure. And the premise here is that you use this to tune a large number of machines where individual admin is infeasible.
Re: Bpftune uses BPF to auto-tune Linux systems
#35Re: Bpftune uses BPF to auto-tune Linux systems
#36Earlier quoted context omitted.
Yes, it is. IMO, except for learning (which should not be done in prod), you shouldn’t make changes that you don’t understand. The tools seems to mostly tweak various networking settings. You could set up a test instance with monitoring, throw load at it, and change the parameters the tool modifies (one at a time!) to see how it reacts.
I'd run such a tool on prod in "advice mode". It should suggest the tweaks, explaining the reasoning behind them, and listing the actions necessary to implement them. Then humans would decide if they want to implement that as is, partly, modified, or not at all.
Re: Bpftune uses BPF to auto-tune Linux systems
#37Earlier quoted context omitted.
>"bpftune logs to syslog so /var/log/messages will contain details of any tuning carried out." (from OP GitHub readme) The rmem example seems to allay fears that it will make changes one can't reverse.
It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.
Your issue appears to be true for any system change. Although, risk will of course vary.
Re: Bpftune uses BPF to auto-tune Linux systems
#38It's great how it grew out of simple packet filtering into tracing and monitoring. It's one of those great tools most should know. Been using it for years.
Re: Bpftune uses BPF to auto-tune Linux systems
#39Earlier quoted context omitted.
If they can be reversed individually you can simply deduce by rolling back changes one by one, no?
Suppose you run a fleet of a thousand machines. They all autotune. They are, lets say, serving cached video, or something. You notice that your aggregate error rate been drifting upwards since using bpftune. It turns out, in reality, there is some complex interaction between the tuning and your routers, or your TOR switches, or whatever - there is feedback that causes oscillations in a tuned value, swinging between t…
Does sound like a potential way to implement literal chaos.
Surely it's like anything else, you do pre-release testing and balance the benefits for you against the risks?
Re: Bpftune uses BPF to auto-tune Linux systems
#40Earlier quoted context omitted.
If they can be reversed individually you can simply deduce by rolling back changes one by one, no?
Suppose you run a fleet of a thousand machines. They all autotune. They are, lets say, serving cached video, or something. You notice that your aggregate error rate been drifting upwards since using bpftune. It turns out, in reality, there is some complex interaction between the tuning and your routers, or your TOR switches, or whatever - there is feedback that causes oscillations in a tuned value, swinging between t…