Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs

How did you get this idea? What was the inspiration behind it? I mean who would of duplication :) ?!

Fascinating! Congrats for the great work

I'm surprised the point/comment ratio is this skewed. There's so much meat in the post to chew on. I like your writing. This was one of those blogs where I can tell you spent a massive amount of time on the technical, but simplified it to layman's terms. I hope you keep putting out stuff :).

I have a couple questions:

1. I think this quote should be raising *many more* eyebrows.

> The astounding thing about Goliath wasn’t that is was a huge leap in performance, it was that the damn thing functioned at all. To this day, I still don’t understand why this didn’t raise more eyebrows.

You put a cat's brain into a dog's head and its still breathing! It didn't flatline immediately! Is yesterday's news? This seems like the biggest take away. Why isn't every attempting LLM-surgery at this moment? Have you noticed any increasede discourse in this area?

2. You mentioned you spent the beginning of your career looking at brains in biotech. How did you end up in a basement of GPU's, working not in biotech, but still kind of looking at brains?

Again, great post!

Fantastic. Really gets me thinking.

If more than two repetitions of the “thinking organ” leads to worse results (I think that’s what you’ve said in other comments), would it be possible to get better results by slicing and dicing some of the early-layer “preparatory organs” between the thinking organs?

Maybe that would still require fine tuning to “evolve” an intermediary organ that would allow for multiple repetitions.

You can still consider it logically from the point of view of in-order with optional looping and optional skipping. It stops being so combinationally explodey then but if you can always append an additional loop and and decide to skip based on worthiness of the layer with varying degrees of threshold then it could theoretically learn an arbitrary ordering where you skip all-bar-one layer per loop.

There's probably a number of common sequences of layers that are inevitable when working on a problem though. I think of it like a expression calculator which could do various parts of an expression tree merging leaf nodes on each iteration. I wouldn't expect it to be quite so explicit with neural nets, but I feel like the underlying principle of do the sub parts then do the same thing on the result of the subparts must be beneficial to some degree.

I think there's probably quite a lot to be revealed from study of representations in those middle layers. If there's a 'how-much-have-we-solved-so-far' signal to be detected from the data between layers, there would be quite a lot of options I think.