Viewing profile — frontsideair

I was able to confirm that it actually runs on ANE, I'm impressed.

> Apple locked it behind Siri. apfel sets it free This doesn't feel truthful, it sounds like this tool is a hack that unlocks something. If I understand it correctly, it's using th…

Yeah, the initial experience with no colors doesn’t look great. I can implement this when I have some free time, if you feel like doing it please feel free to open a PR. Thanks!

14B Qwen was a good choice, but it became outdated a bit and seems like the new version of 4B surpassed it in benchmarks somehow. It's a balancing game, how slow a token generation…

Ollama adding a paid cloud version made me postpone this post for a few weeks at least. I don't object them to make money, but it was hard to recommend a tool for local usage and m…

This is the command probably: sudo sysctl iogpu.wired_limit_mb=184320 Source: https://github.com/ggml-org/llama.cpp/discussions/15396

I'm interested in this, my impression was that the newer chips have unified memory and high memory bandwidth. Do you do inference on the CPU or the external GPU?

Good point, let me add a quick note.

Thank you, it was the integral part of the whole post!

According to the benchmarks, this one is improved in every one of them compared to the previous version, some better than 30B-A3B. Definitely worth a try, it’ll easily fit into mem…

This is the first time I’m hearing about Nebula. How does it compare to Tailscale?

I saw from the comments that they provide smaller binaries, which may have worked on my Raspberry Pi. Maybe I’ll give it a try one day. https://tailscale.com/kb/1207/small-tailscal…

Thanks, this could’ve worked for my Raspberry Pi! I would’ve tried it if it was still in commission. Next time?

That’s exactly why I put the spend limit, and this was the first time to confirm that it actually works.

Exactly, I had a spend limit since I didn’t want to break the bank. It’s back up now.

Synthetic benchmarks should be retired. Build a real app, measure real impact. Unless your users insert 1000 rows, of course. Other than that, I agree with the general premise of R…

This looks like a treasure trove that I somehow missed! I'll give it a more thorough read later, but gave my star for now, thanks!

I'm really interested in `home-manager` honestly, and while I agree about the benefits of declarative package management, I find it easier to type commands at a terminal like `nix …