Viewing profile — tovacinni

We’d just need to write a Rust client for the daemon and load the weights in a way that is compatible with candle- we can definitely look into this since parts of what we are build…

The advantage of loading from a daemon over loading all the weights at once in Python is that it can support multiple processes or even the same process consecutively (if it dies o…

Awesome to hear- that sounds like an application we'd love to help with! (Please feel free to reach out to us too at towaki@outerport.com !)

Yep, it'll work for multi-GPU as well!

Thanks!!

Maybe! Many people don't want to be vendor locked-in though and there are new GPU cloud providers gaining traction. Some still prefer on-prem. We hope to make it easier to bridge t…

That's a super neat idea- we should in fact be able to use this same system to support the orchestration of a 'system prompt caching' sort of thing (across deployments). I'll put t…

Variants do not have to be the same architecture- the demo ( https://hotswap.outerport.com/ ) runs on a couple of different open source architectures. That being said, there is som…

Thanks! If you mean the costs for users of Outerport, it'll be a subscription model for our hosted registry (with a limit on storage / S3 egress) and a license model for self-hosti…

Thanks! I think both are fits- we've gotten interest from both types of companies and our first customer is a "OSS model host". Our 40% savings result is also specifically for the …

Hi HN! We’re Towaki and Allen, and we’re building Outerport ( https://outerport.com ), a distribution network for AI model weights that enables ‘hot-swapping’ of AI models to save …

Render times are listed on table 3 of the paper- interactive, but perhaps not high-performance!

Here's a Twitter thread that explains what this paper is about: https://twitter.com/yongyuanxi/status/1354478763065528320 ?

It sort of can; something I observed while I was still a student at a "top CS university" is that there were certain individuals (superstars) in the school that a lot of people loo…

Sorry if this is a stupid question, but can't you do this without converting the image into an RGB 3D space? (i.e. iterate through the pixels and count the ones within a certain ra…

It's hackernews clickbait for sure :p

Worked for me - just searched "retail jobs" and came up with results from around my area