It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps. I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence. Didn't expect to go back to macOS but they're basically the…
> Didn't expect to go back to macOS but their basically the only feasible consumer option for running large models locally. I presume here you are referring to running on the device in your lap. How about a headless linux inference box in the closet / basement? Return of the home network!
Strix Halo