A walk through of the DeltaNet family of linear attention variants

Post by scraptor » Wed, Jul 29, 2026, 2:26 PM UTC

htrp wrote: post #60

olalonde wrote: post #58

Earlier quoted context omitted.

I never understood people who preferred traditional math notation (e.g. single letter symbols, weird characters like ∣q⟩ instead of writing down an explicit type, etc.). I guess the main advantage is terseness? To me, the mathematical expressions would be so much easier to understand if they were just written in pseudo code or an actual programming language like Python.

math notation doesn't bias towards English language understanding like pseudocode

Post by phoghed » Wed, Jul 29, 2026, 5:35 PM UTC

xp84 wrote: post #121

abixb wrote: post #101

Earlier quoted context omitted.

Well, the goal was to make them come up with cutting-edge theories and/or hypothesis to the most pressing problems facing humanity. Kids will naturally specialize (just like... MoE models?) as their find their groove while growing up, but the goal is to make sure highly gifted kids from diverse background get highest quality space to think without being moulded or "adulterated" by the noise of pop culture and "everyd…

> make sure highly gifted kids from diverse background It would be ideal to get those gifted kids before they spend 5-6 years in the 'real world' with all the "Baby Shark," Roblox, and various other brainrot, but if you pick kids randomly from a very early age, most of them will probably not be all that gifted. Though I suppose maybe you could rely on genetic heritage (parents of very high IQ) and/or just do a large…