261–270 of 362 posts
I had a dream that everyone had super intelligent AIs in their pockets, and yet all they did was doomscroll and catfish...shortly before everything was destroyed.
https://scienceleadership.org/thumbnail/34729/1920x1920
Just in case if someone still didn't realize - we do live in Idiocracy
Earlier quoted context omitted.
This is not entirely dissimilar to what Cerebus does with their weights streaming.
And IIRC the Unreal Engine Matrix demo for PS5 was streaming textures directly from SSD to the engine as well?
What was more interesting about the unreal engine demo, was that they can stream not only textures, but geometry too.
Virtual texturing had been around a long time, but virtual geometry with nanite is really interesting.
Earlier quoted context omitted.
I wonder if anyone has made a liquid cooling system for ipads / phones. Like, a sealed thing that seals onto the back of the device and circulates cooling water directly against the back surface.
A more whimsical method is to put the thing in a glass of water with the cord sticking out. :-) https://www.reddit.com/r/EmulationOnAndroid/comments/1m269k0...
Earlier quoted context omitted.
I wonder if anyone has made a liquid cooling system for ipads / phones. Like, a sealed thing that seals onto the back of the device and circulates cooling water directly against the back surface.
A more whimsical method is to put the thing in a glass of water with the cord sticking out. :-) https://www.reddit.com/r/EmulationOnAndroid/comments/1m269k0...
Was wondering, but this the most duct tap hacker solution!
Run an incredible 400B parameters on a handheld device. 0.6 t/s, wait 30 seconds to see what these billions of calculations get us: "That is a profound observation, and you are absolutely right ..."
Better than waiting 7.5 million years to have a tell you the answer is 42.
Earlier quoted context omitted.
Only way to have hardware reach this sort of efficiency is to embed the model in hardware. This exists[0], but the chip in question is physically large and won't fit on a phone. [0] https://www.anuragk.com/blog/posts/Taalas.html
I think for many reasons this will become the dominant paradigm for end user devices. Moore's law will shrink it to 8mm soon. I think it'll be like a microSD card you plug in. Or we develop a new silicon process that can mimic synaptic weights in biology. Synapses have plasticity.
I had a dream that everyone had super intelligent AIs in their pockets, and yet all they did was doomscroll and catfish...shortly before everything was destroyed.
Earlier quoted context omitted.
Yes but most people are still running MoE models with all experts loaded in RAM! This experiment shows quite clearly that some experts are only rarely needed, so you do benefit from not caching every single expert-layer in RAM at all times.
That's not what this test shows. It's just loading the parts of the model that are used in an on-demand fashion from flash. The iPhone 17 Pro only has 12GB of RAM. This is a -17B MoE model. Even quantized, you can only realistically fit one expert in RAM at a time. Maybe 2 with extreme quantization. It's just swapping them out constantly. If some of the experts were unused then you could distill them away. This has b…