51–60 of 251 posts
I am the person that wrote that. Sorry about the font. This is a bit outdated, AI stuff goes at high speed. More models so I will try to update that. Every month so many new models come out. My new fav is GLM-4.5... Kimi K2 is also good, and Qwen3-Coder 480b, or 2507 instruct.. very good as well. All of those work really well in any agentic environment/in agent tools. I made a context helper app ( https://wuu73.org/a…
The setup could be: • Cursor CLI for agentic/dev stuff (example:https://x.com/cursor_ai/status/1953559384531050724) • A local memory layer compatible with the CLI — something like LEANN (97% smaller index, zero cloud cost, full privacy, https://github.com/yichuan-w/LEANN) or Milvus (though Milvus often ends up cloud/token-based) • Your inference engine, e.g. Ollama, which is great for running OSS GPT models locally
With this, you’d have an offline, private, and blazing-fast personal dev+AI environment. LEANN in particular is built exactly for this kind of setup — tiny footprint, semantic search over your entire local world, and Claude Code/ Cursor –compatible out of the box, the ollama for generation. I guess this solution is not only free but also does not need any API.
But I do agree that this need some effort to set up, but maybe someone can make these easy and fully open-source
I think there’s huge potential for a fully local “Cursor-like” stack — no cloud, no API keys, just everything running on your machine. The setup could be: • Cursor CLI for agentic/dev stuff (example: https://x.com/cursor_ai/status/1953559384531050724 ) • A local memory layer compatible with the CLI — something like LEANN (97% smaller index, zero cloud cost, full privacy, https://github.com/yichuan-w/LEANN ) or Milvus…
but you'll quickly notice that it's not even close to matching the quality of output, thought and reflecting that you'd get from running the same model but significantly high parameter count on a GPU capable of providing over 128gb of actual vram.
There isn't anything available locally that will let me load a 128gb model and provide anything above 150tps
The only thing that local ai model makes sense for right now seems to be Home Assistant in order to replace your google home/alexis.
happy to be proven wrong, but the effort to reward just isn't there for local ai.
I wonder how much energy this is wasting.
I bet it's crazy to some people that others okay with giving up so much of their data for free tiers. Like yeah it's better to selfhost but it takes so much resources to run good enough LLM at home that I'd rather give up my code for some free usage, anyway that code eventually will end up open source
Earlier quoted context omitted.
Is glm-4.5 air useable? I see it's free on Openrouter. Also pls advise what you think is the current best free openrouter model for coding. Thanks!
Well, if you download Qwen Code https://github.com/QwenLM/qwen-code it is free up to 2000 api calls a day. Not sure if GLM-4.5 Air is good, but non-Air one is fabulous. I know for free API access there is pollinations ai project. Also llm7. If you just use the web chat's you can use most of the best models for free without API. There are ways to 'emulate' an API automatically.. I was thinking about adding this to my…
Earlier quoted context omitted.
I agree, Google is definitely the champion of respecting your privacy. Will definitely not train their model on your data if you pay them. I mean you should definitely just film yourself and give them everything, access to your files, phone records, even bank accounts. Just make sure to pay them those measly $200 and absolutely they will not share that data with anybody.
You're thinking of Facebook. A lot of companies run on Gmail and Google Docs (easy to verify with `dig MX [bigco].com`), and they would not if Google shared that data with anybody.