Viewing profile — xyc

I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.

For TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/... There isn't much documentation available right now but you can ask coding agent eg. …

Great to see more local AI tools supporting MCP! Recently I've also added MCP support to recurse.chat. When running locally (LLaMA.cpp and Ollama) it still needs to catch up in ter…

It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with t…

recurse.chat + M2 max Mac

I recently discovered toolhive which is pretty handy too https://github.com/stacklok/toolhive

If you are on a Mac, give https://recurse.chat/ a try. As simple as download the model and start chatting. Just added the new multimodal support in LLaMA.cpp.

Actually this is a good way to find product ideas. I placed a query in Grok to find posts about what people want, similar to this. Then it performs multiple searches on X including…

It seems that this is possibly not necessary, since LLaMA.cpp already integrates Jinja with CPP implementation (through minja)

Check out https://recurse.chat/

The fact that there's no alternative implementation of SQLite also seems to play a part in preventing standardization of WebSQL. https://www.w3.org/TR/webdatabase/ "The specificati…

If anyone on macOS wants to use llama.cpp with ease, check out https://recurse.chat/ . Supports importing ChatGPT history & continue chats offline using llama.cpp. Built this so I …

You can get a release binary from https://github.com/ggerganov/llama.cpp/releases too.

Made tool use work! check out demo here: https://x.com/chxy/status/1861684254297727299

sharing the messy code here just for funsies: https://gist.github.com/xyc/274394031b41ac7e8d7d3aa7f4f7bed9

Just tried out the puppeteer server example if anyone is interested in seeing a demo: https://x.com/chxy/status/1861302909402861905 . (Todo: add tool use - prompt would be like "go…

^ asked the question in the discussion: https://github.com/modelcontextprotocol/specification/discus...

Thanks for the pointers! Will do. I've fired up https://github.com/modelcontextprotocol/inspector and the code looks helpful too. I'm looking at integrating MCP with desktop app. T…

Superb work and super promising! I had wished for a protocol like this. Is there a recommended resource for building MCP client? From what I've seen it just mentions Claude desktop…

With Claude you barely had to learn the language this days as you just need to prompt, but SQLite column is an interesting idea.

If you are interested in no config setup for local LLM, give https://recurse.chat/ a try (I'm the dev). The app is designed to be self-contained and as simple as you can imagine.

Shameless plug: If you are on a Mac, check out RecurseChat: https://recurse.chat/ A few outstanding features: - Fast: Import ChatGPT history, loads thousands of conversations at on…

Regarding chat history, I've been thinking that people should have ownership over chat history. We are migrating the chats towards SQLite, so your data is going to be a timeless fo…

Give https://recurse.chat/ a try - I'm the developer. One particular advantage over alternative apps is importing ChatGPT history and speed of the app, including full-text search. …

what's your experience with sqlite-vec? I'm considering using sqlite-vec in addition to/or replace qdrant vector db for a project (recurse.chat), since I'm moving all the data to s…