Viewing profile — danielhanchen

Oh thanks for sharing! The llama.cpp PRs should generally be fine for now - I'm fixing a few small edge cases as well!

Very cool write-up and GitHub repo!

Thank you appreciate the support! It's all thanks to you guys and the community!

Update - Just got rid of the spiced up intro

Thank you!

Oh thanks :) We're also going to add MTP support soon for Qwen3.6! 95% of it is fully human done - the maths, algos, code snippets, screenshots & benchmarks are done / conducted by…

[dead]

Sorry on the delay - so it installs https://github.com/Blaizzy/mlx-vlm and other components and sets up the commands - you don't need to use it but we thought it might be easier fo…

Sorry on the delay - oh haha that would be cool :) We did release 2bit dynamic ones, but unsure if they'll be helpful

Yes we do! Sorry on the delay

We use Duck Duck Go - sorry on the delayed response as well

Thank you and appreciate it! Sorry on the delayed reply as well

Oh yes LM Link is cool!

Hey sorry on the delay - we just added API support, so you can access a remote server - it includes optional python, tool call, bash and web search support if you enable them. For …

Hey! Sorry for not replying sooner - yes we'll keep publishing more KLD - sadly some are saying we are "optimizing" for KLD now since we posted so many haha - but the whole purpose…

Hey so sorry didn't reply sooner - yes the docker used to be I think 4-8GB ish since CUDA sadly itself is 4GB I think, and PyTorch takes the rest. So unfortunately the Unsloth Dock…

Apologies as well didn't reply sooner - Studio supports AMD out of the box now! We worked with AMD to make it work! One thing that is still missing is pre-compiled AMD ROCM binarie…

Oh my apologies I didn't respond - if only HN had a notifier haha Oh yes we added a custom folder button which can pull .gguf files for now from any folder - it supports LM Studio …

We made Unsloth Studio which should help :) 1. Auto best official parameters set for all models 2. Auto determines the largest quant that can fit on your PC / Mac etc 3. Auto deter…

Haha :)

Haha :) We had some issues with Kimi-2.6 since it was int4 and we were investigating how to handle it :)

We also made some dynamic MLX ones if they help - it might be faster for Macs, but llama-server definitely is improving at a fast pace. https://huggingface.co/unsloth/Qwen3.6-27B-U…

Yes sadly CUDA 13.2 is broken - NVIDIA will push a fix in CUDA 13.3

Yes we have started doing diffusion GGUFs but it's in it's infancy :) But yes we do generate images to test quants out!