Viewing profile — lukebechtel

> Chip Design > As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimized, …

[dead]

[dead]

Some people seem to be suggesting these are coverups for quantization... Those who work on agent harnesses for a living realize how sensitive models can be to even minor changes in…

[dead]

really fun :) thanks!

it sounds like you described it pretty well!

~/.claude/projects is where the real fun is :)

What is your domain?

so we need to make some crazy llms...

there used to be https://default-filename-tv.neocities.org/ but it got taken down :/

The bitter lesson strikes again, I suppose!

Good questions! It's clear I need to gather more metrics from our next generated inference library.

Unfortunately it hasn't been open sourced. We're debating how / when to do this right now.

This is a fair critique! We plan to use our system to generate many more inference libraries of this nature, and I'll make it a point to release better, broader correctness measure…

Yes, great question! The system started without paged attention, and recreated its own paged attention implementation automatically once it realized it was a bottleneck. Pretty coo…

Unfortunately, not at present; we went for FP8 because we believed it was generally the best tradeoff of quality and speed. Allowed faster iteration as well. We believe our improve…

Yes, speculative decoding will make both us and VLLM faster, but we believe it would be a relatively even bump on both sides, so we didn't include it in this comparison. Worth anot…

We validate with MMLU and Hellaswag presently, and are getting this independently verified by a 3rd party. We have considered open-sourcing some of our optimized inference librarie…

I also do this!

Thank you Georgi <3

sonnet 4.6 is a third, and equivalent to opus 4.5, which is enough for me usually :) EDIT: Gemini does have 1m context for "free" though so that's great.

Arc-AGI-2: 84.6% (vs 68.8% for Opus 4.6) Wow. https://blog.google/innovation-and-ai/models-and-research/ge...

Oh, I always use opus 4.5 thinking mode. Maybe that's the diff.