Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bpftune uses BPF to auto-tune Linux systems

github.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Bpftune uses BPF to auto-tune Linux systems

#61

Post by fragmede »

gerdesj wrote: post #55
6SixTy wrote: post #35

As an insane Gentoo user, sign me the F*** up

You are not insane. You are running a distro that enables you to patch both the distro package definitions and the source code that is compiled by those definitions. Then there is the day to day stuff such as USE ... I have a VM that was rather unloved that I ended up using git to go back in time to gradually move it forwards in time and update. It took quite a while. It started off life in the noughties and now spor…

Back in the day when Gentoo was in favor, USE flags and recompiling the world could make a huge difference in performance on much slower computers with far more limited RAM, and you cloths really see the difference. After waiting week for the world to recompile, that it. These days, unless you're, a hyperscaler and need to eke that last 10% of performance our of your hardware, ain't nobody got time for that.

    emerge --update --deep --with-bdeps=y --newuse @world

Re: Bpftune uses BPF to auto-tune Linux systems

#63

Post by gerdesj »

cortesoft wrote: post #58
gerdesj wrote: post #57

Earlier quoted context omitted.

We (in IT security) are expected to abrogate responsibility to funky AI or whatevs anti virus and other stuff. Buy and install a security package from ... whoever ... and all will be well. This is an expert system/advice run by real people (at a reasonably well respected firm) not an AI wankery thingie. It is literally expert advice and it is being given away and in code form which you can read. What on earth is wron…

If the alternative is those proprietary anti virus products, sure this is better. The original comment was comparing to doing nothing and just using the standard distro, I believe.

I do hope I haven't offended anyone but I also hope I will leave no one in any doubt that IT security is important.

The world is now very highly interconnected. When I was a child, I would have rubbish conversations over the blower to an aunt in Australia - that latency was well over one second - satellite links. Nowadays we have direct fibre connections.

So, does you does ?

Re: Bpftune uses BPF to auto-tune Linux systems

#64

Post by gerdesj »

fragmede wrote: post #61
gerdesj wrote: post #55

Earlier quoted context omitted.

You are not insane. You are running a distro that enables you to patch both the distro package definitions and the source code that is compiled by those definitions. Then there is the day to day stuff such as USE ... I have a VM that was rather unloved that I ended up using git to go back in time to gradually move it forwards in time and update. It took quite a while. It started off life in the noughties and now spor…

Back in the day when Gentoo was in favor, USE flags and recompiling the world could make a huge difference in performance on much slower computers with far more limited RAM, and you cloths really see the difference. After waiting week for the world to recompile, that it. These days, unless you're, a hyperscaler and need to eke that last 10% of performance our of your hardware, ain't nobody got time for that. emerge -…

I was a Gentoo aficionado for quite a while (20+ years). I'm going back too (rather slowly)

There is a lot of very decent technology in Gentoo - it literally describes what all other distros need to do, in some cases.

You mention a week of compilation - I remember the gcc 4 thing too!

Re: Bpftune uses BPF to auto-tune Linux systems

#65

Post by yourapostasy »

admax88qqq wrote: post #21
pbhjpbhj wrote: post #20

Earlier quoted context omitted.

>"bpftune logs to syslog so /var/log/messages will contain details of any tuning carried out." (from OP GitHub readme) The rmem example seems to allay fears that it will make changes one can't reverse.

It’s not a questions of being able to reverse. It’s a question of being able to diagnose that one of these changes even was the problem and if so which one.

Record changes in git and then git bisect issues, maybe?

Without change capture, solid regression testing, or observability, it seems difficult to manage these changes. I’d like to how others are managing these kinds of changes to readily troubleshoot them, without lots of regression testing or observability, if anyone has successes to share.

Re: Bpftune uses BPF to auto-tune Linux systems

#67

Post by withinboredom »

sgarland wrote: post #24
usr1106 wrote: post #3

Interesting. But if tuning parameters to their best values were easy, shouldn't the kernel just do that in the first place?

I’d rather the kernel present a good-enough but extremely stable set of configs. If I’m using a distro like Arch or Gentoo, then sure, maybe run wild (though both of those would probably assume I’m tuning them anyway), but CentOS, Debian, et al.? Stable and boring. If you change something, you’d better know what it is, and why you’re doing it.

These parameters affect things at the hardware level, and in the kernel. There’s “good enough” defaults already, but if you want to take full advantage of your specific hardware, you need to tune it. This is especially important the more different your hardware is from kernel developers’ computers (like 10g nics).

Re: Bpftune uses BPF to auto-tune Linux systems

#68

Post by tw04 »

gerdesj wrote: post #57
cortesoft wrote: post #56

Earlier quoted context omitted.

> You will obviously have a change management system which describes all the changes you have made to your putative standard distro configs. You will also be monitoring those changes. Which is now a list you will have to check for every issue. I don't think they are complaining they don't trust the writers of the code, just that it adds confounding variables to your system

We (in IT security) are expected to abrogate responsibility to funky AI or whatevs anti virus and other stuff. Buy and install a security package from ... whoever ... and all will be well. This is an expert system/advice run by real people (at a reasonably well respected firm) not an AI wankery thingie. It is literally expert advice and it is being given away and in code form which you can read. What on earth is wron…

Well two points: Oracle isn’t really and hasn’t really been respected in several decades.

Oracle exists for one sole purpose, which is to make Larry money. Anything they “give away for free” almost always includes a non-obvious catch which you only discover during some future audit.

In this case it appears to be gpl and thus most likely harmless. But I’d expect either the license to change once people are hooked, or some glaring lack of functionality that’s not immediately obvious, that can only be remediated by purchasing a license of some sort.

Re: Bpftune uses BPF to auto-tune Linux systems

#69

Post by zorked »

sgarland wrote: post #22
gausswho wrote: post #18

With this tool I am wary that I'll encounter system issues that are dramatically more difficult to diagnose and troubleshoot because I'll have drifted from a standard distro configuration. And in ways I'm unaware of. Is this a reasonable hesitation?

Yes, it is. IMO, except for learning (which should not be done in prod), you shouldn’t make changes that you don’t understand. The tools seems to mostly tweak various networking settings. You could set up a test instance with monitoring, throw load at it, and change the parameters the tool modifies (one at a time!) to see how it reacts.

Learning and self-tuning happens everywhere. Cache sizes that adjust to load, pools that grow and shrink, etc. This is just adding autotuning to something that doesn't have it. With presumably better algorithms than the rules-of-thumb approach that is more common.

It's a great idea.

Re: Bpftune uses BPF to auto-tune Linux systems

#70

Post by lupusreal »

gerdesj wrote: post #57
cortesoft wrote: post #56

Earlier quoted context omitted.

> You will obviously have a change management system which describes all the changes you have made to your putative standard distro configs. You will also be monitoring those changes. Which is now a list you will have to check for every issue. I don't think they are complaining they don't trust the writers of the code, just that it adds confounding variables to your system

We (in IT security) are expected to abrogate responsibility to funky AI or whatevs anti virus and other stuff. Buy and install a security package from ... whoever ... and all will be well. This is an expert system/advice run by real people (at a reasonably well respected firm) not an AI wankery thingie. It is literally expert advice and it is being given away and in code form which you can read. What on earth is wron…

Being able to dodge responsibility for something going wrong is great, but it's always better for you if you aren't in a position where you have to dodge responsibility in the first place.

So if this tool makes a well reasoned and ostensibly sensible tweak which happens to expose some flaw in your system and takes it down, being able to say "those experts Oracle made the mistake, not me" might get you out of the hot seat. But it's better to never be in the hot seat.

Post reply on HN