Viewing profile — Manfrednotfunny

Memory bandwidtih between pro and max is double. 300gb/s vs. 600gb/s btw.

I did, it started to become too much work to run it well due to all the spam :| (even with the right signatures and configs, until you learn what a blacklisted ip is and that ips n…

My current image from a MoE is that the base/core might be the more generic thing and that things like python are part of one expert though.

I don't thinnk just MoE will solve it. If you hit constantly different expert layers, you can't outsource layers efficently and have to swap it in. MoE will be faster because it wi…

I'm waiting for the speed/quality per dollar metric to go down a little bit further and then I will def run it at home. Its not just that you send a sentence to an API endpoint, yo…

I would love to see any good research projects about it but i have the feeling that Frontier with MoE is making too fast of a progress so that a customized model would always be wo…

No he was not. The fix was a few hours, the customer called him regularly. The CTO had no understanding that a basic SQL Insert shouldn't take a second. This fix smoothed all respo…

C* can mean anything. The CTO of my last company thought that an API request under 1 second is okay as the customer at that endpoint wouldn't mind anything under 1 second. I explai…