Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs

I found this super interesting! Excellent writing! And I loved the cowboy quote, that was the best part; poor thing.

Now it's making me wonder - instead of smashing things together more violently for MoE type stuff, perhaps it's more effective to create better toolsets to allow us to analyse smaller models.

Then small models can be trained (faster & cheaper) to be excellent at very specific tasks or domains, the toolset used to identify the organ and organ selection layers, a larger Frankenstein's monster model can be stitched together from these organs with perhaps a little extra training/fine-tuning to improve its organ selection abilities.

That makes me imagine some sort of future of layer standardisation, in which for a standard and optimal architecture sets of layers can be dynamically downloaded, added, swapped out etc to maintain fastest inference speed whilst allowing for flexible skills. Almost like the concept of subagents but within the architecture of the model itself. Hmmm.

I'm only versed in transformer architecture at a high level, does anybody know of any architectures where the layers branch & then coalesce like that? Or is it majority linear layer by layer?

How about, as you found repeating x-y was useful for locating the block of 7 layers in the first place; I'd be incredibly curious if, knowing that block of 7, if you then iterated from repeating x-y in that block z times.

Like for those 7 layers 1,2,3,4,5,6,7 does efficiency increase if you run 1,2,3,3,4,4,4,5,6,7 or perhaps 1,2,3,3,4,5,6,6,7 etc. If only GPUs grew on trees

dude thats sick! i tried it out and it works. theres a couple layers in there that are part of the voidy block that doesnt do much for the selected answer, so i narrowed it down to L48-53 where this model is mapping out its reasoning strategy, and repeated that twice, i got a big improvement over the original config (i chose some questions from atropos and claude code made some up so idk not like a real dataset).

so thats about %15 more compute per forward pass with 0 extra memory which is just nuts, so for a streaming or disk-based setup its just free better answers. def wasnt gonna think of this myself.

  config               layers   overall    delta          math     reasoning  word problems
  baseline                 80    0.5391  +0.0000        0.5850        0.6357        0.3500
  rys                      87    0.5452  +0.0061        0.6706        0.6000        0.2723
  cartographer_repeat_x2   92    0.7741  +0.2350        0.8455        0.8214        0.6000

looks like the model gets a second/third go at figuring out how to approach the problem and it gets better answers.

i tried a matrix of other configurations and stuff gets totally weird. like playing em through backwards in that block doesnt make much of a difference / order doesnt seem to matter (?!). doubling each layer got a benefit, but if i doubled the layers and doubled that block there was interference. doubling the block where the model is architecting/crystallizing its plans improves reasoning but at the cost of other stuff. other mixes of blocks showed some improvements for certain kinds of prompts but didnt stand out as much.

Yes, I have done these thype of experiments; thats for the next post

I stick with models I can run on VRAM, but DeepSeek Speciale have the best reasoning capabilities of the models I can actually run (https://huggingface.co/deepseek-ai/DeepSeek-V3.2-Speciale). What hardware can you access?

I have Deepseek etc, but inferencing on DDR5 would take about 2-3 weeks for a simple scan. I think this works best with dense models, but it also seems ok with MoE.

@everyone: Can someone hook me up with Nvidia sponsorship?