Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The ~200 Line Linux Kernel Patch That Does Wonders

phoronix.com

Post reply on HN
Print view View original

31–40 of 98 posts

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#31

Post by trotsky »

krschultz wrote: post #9
hippich wrote: post #7

I am newbie when it comes to compiling kernel. Is it a pain to do with stock ubuntu 10.10? Sometimes I run something heavy on my laptop and desktop freezes annoy me. If this patch will allow me to get around it - I would be glad to try it out. Anyone having url of some niuce tutorial to compile new kernel for ubuntu 10.10?

It isn't hard to do, but a painless way to learn is to install a ubuntu 10.10 instance in a virtualbox and try it all in the box. If you screw up, who cares. After you have been through the process once it won't be intimidating to do it for your real OS.

One thing to consider in terms of building a new kernel is that repos tend to customize their release kernels quite a bit, by adding non-mainline patches, extra drivers, etc.

Recompiling a vanilla kernel (from the linux-next git) is a pretty easy process, but you may find that when running it you've lost some nice features of the release you run or odd things have stopped working.

One might get better results from using a vendor supplied release kernel source tree (installing the kernel sources package for the repo) and then applying a patch to add the new scheduling groups. Making a patch like that is probably too hard for a newbie, but I'd be surprised if someone on the ubuntu forums doesn't end up providing one sometime soon.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#32

Post by zemanel »

rufo wrote: post #13
zemanel wrote: post #11

i whish something similar could be ported to BSD/Darwin, OSX. I have a MBP 6,2 (i5) with 4GB mem/5400 rpm disk and it's quite easy to hog it down, to almost unbearable sometimes.

What sort of tasks? FWIW, I had a similar configuration to yours (just an older MBP) and installing an SSD helped immensely . I can hit 200% CPU load and not even realize it until the fans kick in…

I know the subject is pretty much Apple's and oranges but i'm currently running: - Terminal with 2 tabs - firefox with 2 tabs - chrome with about 9 - gaim and skype - iTunes streaming soma.fm - Netbeans with an opened project - jEdit - Colloquy - Postgres instance

and as soon as i booted Windows XP in VMware, well, took me a while to be able to reply to this post (after the vm settled).

I also that you might be saying "d'oh" but i've had Gentoo running on this metal and a "similar" environment AND compiling stuff with -j4 doesn't freeze away my UI.

My user experience with "OSX" is that it's, way more prone to unresponsiveness due to load, but hey, who cares :P clicks Time Machine

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#33

Post by trotsky »

jws wrote: post #21

I don't think I've managed to piece all this together, perhaps someone can fill in the blanks. • The patch automatically creates a task group for each TTY. • The patch automatically assigns each new process to the task group for its controlling TTY. • In the case where there are large (>cores) numbers of cpu bound jobs, the latency of interactive jobs is vastly improved. I think the piece I'm missing is the behavior…

Good questions. As far as the "why is this an improvement over just nicing" I found this link elsewhere in this thread:

http://marc.info/?l=linux-kernel&m=128991621119292&w...

It includes this discussion:

No, it won't. The target audience is those folks who don't _do_ the configuration they _could_ do, folks who don't use SCHED_IDLE or nice, or the power available through userspace cgroup tools.. folks who expect their box to "just work", out of the box.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#34

Post by tbrownaw »

jws wrote: post #21

I don't think I've managed to piece all this together, perhaps someone can fill in the blanks. • The patch automatically creates a task group for each TTY. • The patch automatically assigns each new process to the task group for its controlling TTY. • In the case where there are large (>cores) numbers of cpu bound jobs, the latency of interactive jobs is vastly improved. I think the piece I'm missing is the behavior…

> I think the piece I'm missing is the behavior of the scheduler. Does it now make its decisions based on task group cpu consumption instead of process? I saw options to that effect back around 2.6.25.

Yes, and yes. Previously you could set things like this up explicitly, this makes it (optionally) automatic.

> Why is this an improvement over just nicing the "make -j64" into the basement

That gives you... I think 10% less CPU weight per level, so you can get down to 13% of base. So your 64 processes will still weigh about 8x one base process.

This lets you consider everything spawned from your terminal as one group, and everything from your X session as another, so your compile processes collectively (no matter how many you have) weigh as much as your GUI processes collectively weigh.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#35

Post by AngryParsley »

zemanel wrote: post #11

i whish something similar could be ported to BSD/Darwin, OSX. I have a MBP 6,2 (i5) with 4GB mem/5400 rpm disk and it's quite easy to hog it down, to almost unbearable sometimes.

5400 rpm disk

Even with a perfect scheduler you're going to have to wait on I/O. Disk speeds are the limiting factor on most machines, and this goes double for laptops. I highly recommend getting an SSD.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#36

Post by zemanel »

AngryParsley wrote: post #35
zemanel wrote: post #11

i whish something similar could be ported to BSD/Darwin, OSX. I have a MBP 6,2 (i5) with 4GB mem/5400 rpm disk and it's quite easy to hog it down, to almost unbearable sometimes.

5400 rpm disk Even with a perfect scheduler you're going to have to wait on I/O. Disk speeds are the limiting factor on most machines, and this goes double for laptops. I highly recommend getting an SSD.

yes, i'd like that very much but the current prices are too high for me.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#37

Post by jedbrown »

azim wrote: post #25
jedbrown wrote: post #16

Earlier quoted context omitted.

Where N is the number of physical cores? I do not use hyper-threading (it tends to be bad for the floating point and bandwidth limited operations that I do), but usually find minimal compile times at N+1 jobs (but with little penalty for several more).

Out of curiosity, What types of applications are you running where HT hurts performance?

Sparse matrix kernels and finite element/volume integration. For bandwidth-limited operations, it is sometimes possible to get better performance by using less threads than physical cores because the bus is already saturated (for examples, see STREAM benchmarks). For dense kernels, I'm usually shooting for around 70 percent of peak flop/s, and any performance shortcomings are from required horizontal vector operations, data dependence, and multiply-add imbalance. These are not things that HT helps with.

Additionally, HT affects benchmark reproducibility which is already bad enough on multicore x86 with NUMA, virtual memory, and funky networks. (Compare to Blue Gene which is also multicore, but uses no TLB (virtual addresses are offset-mapped to physical addresses), has almost independent memory bandwidth per core, and a better network.)

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#38

Post by deckard »

burgerbrain wrote: post #29
rufo wrote: post #13

Earlier quoted context omitted.

What sort of tasks? FWIW, I had a similar configuration to yours (just an older MBP) and installing an SSD helped immensely . I can hit 200% CPU load and not even realize it until the fans kick in…

What kind of application blocks on disk IO but nothing else for extended amounts of time? I'm having a hard time seeing how installing an SSD and maxing out your cores are terribly related otherwise. SSDs do a lot to reduce loadtimes, and thus make your computer seem much faster, but they do little for making your programs run full-speed-ahead constantly. Most every application out there blocks on network connections…

"What kind of application blocks on disk IO but nothing else for extended amounts of time? I'm having a hard time seeing how installing an SSD and maxing out your cores are terribly related otherwise"

Virtual memory paging to/from disk. This is probably why the new MacBook Airs feel faster than the CPU+RAM specs suggest.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#39

Post by maximilian »

bobf wrote: post #24

I don't follow kernel development extremely closely, but it fascinates me that people are still actively working on the kernel's scheduler and achieving a "huge improvement" like this.

I remember reading something about an anesthesiologist who got into hacking the scheduler targeting desktop use cases. He generally said that the scheduler gets a lot more attention from people who care more about server work loads and such. He had his own custom kernel patches that people used to get directly from him that weren't in the mainstream kernel -- This was before the era of multi-core, but people said his scheduler had better responsiveness than the default one.

Re: The ~200 Line Linux Kernel Patch That Does Wonders

#40

Post by AngryParsley »

zemanel wrote: post #36
AngryParsley wrote: post #35

Earlier quoted context omitted.

5400 rpm disk Even with a perfect scheduler you're going to have to wait on I/O. Disk speeds are the limiting factor on most machines, and this goes double for laptops. I highly recommend getting an SSD.

yes, i'd like that very much but the current prices are too high for me.

You can afford a MacBook Pro but you can't afford an SSD that costs 25% of that?

(A really fast expensive SSD is around $400.)

Post reply on HN