Viewing profile — p12tic

> Only a matter of time for local models to reach Opus level. We are 1 or at most 2 years behind that and Anthropic knows that. Can confirm. Kimi K2.5 is pretty intelligent and mos…

> In the last few years California runs 100% renewable on many days (and growing) every year. How many is "many days"? Gas is still used for at least one fifth of electricity. http…

> not ... web/db servers, lightweight stuff like that. They scale very well for web and db servers as well. You just put lots of containers/VMs on a single server. AMD EPYC has a s…

Depends on a server. This test got 79W idle for _two socket_ E5 2690-V4 server. https://www.servethehome.com/lenovo-system-x3650-m5-workhors...

The problem is with the form factor, not the server hardware per-se. If one buys regular ATX motherboard that accepts server CPUs and fits it in regular ATX case, then there's lots…

Better build a single workstation - less noise, less power usage and the form factor is way more convenient. A budget of $3000 can buy 128 cores with 512GB of RAM on a single regul…

> Today it's a bit more complicated when you have servers with 100+ cores as an option for under $30k (guestimate based on $10k CPU price). If one can buy used, then previous gener…

That's just an artifact of Intel disabling ECC on consumer processors. There's no reason for ECC to have significantly higher power consumption. It's just an additional memory chip…

We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.

All of this is for batch size 1.

State of the art of local models is even further. For example, look into https://github.com/kvcache-ai/ktransformers , which achieve >11 tokens/s on a relatively old two socket Xeo…

Incorrect. https://en.wikipedia.org/wiki/USB_hardware#USB_Power_Deliver... is a good start about the subject: "PD-aware devices implement a flexible power management scheme by inte…

For all intents and purposes cache may not exist when the working set is 17B or 109B parameters. So it's still better that less parameters are activated for each token. 17B paramet…

Most laptops are severely limited by heat dissipation. So it's normal that performance is much worse. The CPU cannot stay in turbo as long and must drop to lower frequencies sooner…

Container security boundary can be much stronger if one wants. One can use something like https://github.com/google/gvisor as a container runtime for podman or docker. It's a good …

One can use something like https://github.com/google/gvisor as a container runtime for podman or docker. It's a good hybrid between VMs and containers. The container is put into so…

Seems like that's true: https://learn.microsoft.com/en-us/sharepoint/redirect-known-...

It's complicated, memory accesses can really block for relatively long periods of time. Consider that regular memory access via cache takes around 1 nanosecond. If the data is not …

That would be a significant downgrade. Threadripper CPUs top out at 64 cores / 128 threads.

I agree that just buying the most expensive monitor is waste of resources. On the other hand, I think price shouldn't even come into the picture when deciding on basic things, such…

It's strange when monitor costs are being discussed while they are the primary tool for software development. If software developer produces $100k/year value, then it's worth spend…

It uses a custom widget toolkit. Adding touchpad gesture support is certainly doable, but it would benefit only single application, so we haven't prioritized that so far.

Touchpad gesture implementation in X server has been released in version 21.1 Wayland touchpad gesture implementation in Qt widget framework has been released in version 6.2.0 X11 …

Fedora and Gnome is only one desktop environment and one widget toolkit. For example, Qt-based apps didn't have touchpad gestures at all anywhere until my work on Wayland gestures …

Interesting, this feature may only be for touchscreens because two-finger swipes are registered as scrolls on Wayland. This will indeed need further work. What does work though is …