Run Kimi K3 using 29 GB of RAM at 0.50 tok/s

Post by asveikau » Sat, Aug 01, 2026, 2:53 PM UTC

walrus01 wrote: post #84

mappu wrote: post #81

Standard llama.cpp can mmap the gguf, so it'll stay on disk if it doesn't fit on memory, and the kernel page cache will ensure the hot parts ("resident trunk") stay resident. What's the benefit of a custom implementation at all?

Letting the kernel use SSD based swap space for something this big would be a good way to destroy its cumulative write endurance over a period of just a couple months. I would be very interested in seeing SMART self reported drive cumulative write and wear out stats if this was done for more than a short test. In my experience llama-server is better run with --no-mmap on things that will fit entirely into RAM. Though…

Post by knollimar » Sat, Aug 01, 2026, 3:31 PM UTC

IshKebab wrote: post #129

darkwater wrote: post #114

Earlier quoted context omitted.

It's not a cost if you self-consume your energy, even if you get paid for what you inject. You are just amortizing differently the cost of the PV installation. But what you get for the energy you sell is not real money, it's just money you MIGHT consume as energy in other periods.

I have no idea what you're trying to say. With PV, if you consume one additional kWh your bank balance will be 20c lower than it would otherwise have been. You can argue semantics about whether that is a "cost" all you want, but practically the end result is the same and any financial decisions about whether to consume that kWh should have an identical result to what you'd decide if you bought the energy in for the s…

Post by IshKebab » Sat, Aug 01, 2026, 3:57 PM UTC

knollimar wrote: post #144

IshKebab wrote: post #129

Earlier quoted context omitted.

I have no idea what you're trying to say. With PV, if you consume one additional kWh your bank balance will be 20c lower than it would otherwise have been. You can argue semantics about whether that is a "cost" all you want, but practically the end result is the same and any financial decisions about whether to consume that kWh should have an identical result to what you'd decide if you bought the energy in for the s…

they're saying you might not use the extra energy if your PV runs year round. If you continuously had a year round surplus, you wouldn't have a need to buy any energy, so the credits would be monopoly money.

Post by OutOfHere » Sat, Aug 01, 2026, 4:14 PM UTC

marcobambini wrote: post #103

airstrafer wrote: post #73

Earlier quoted context omitted.

wow. shamelessly stealing the name of one of the most robust and effective libraries ever, to push some ai bullshit. what an embarrassment

We are backed by the SQLite author, and we have the right to use the SQLite name. Please do your homework before writing such comments. We used an Elastic license for some projects to protect our work from being used in SASS without a prior agreement. WASTE is and will always be available with a very permissive license. I really don't understand why some people prefer to spread hate instead of just asking for clarifi…

Post by knollimar » Sat, Aug 01, 2026, 4:17 PM UTC

IshKebab wrote: post #146

knollimar wrote: post #144

Earlier quoted context omitted.

they're saying you might not use the extra energy if your PV runs year round. If you continuously had a year round surplus, you wouldn't have a need to buy any energy, so the credits would be monopoly money.

But unless you're off-grid you can always use extra PV energy by selling it to the grid.