391–400 of 405 posts
Earlier quoted context omitted.
> Are win32 APIs proprietary? Yes. And crap. Not in my code bases.
If you're going to make apps in windows, you need to call their proprietary API somehow. Maybe you do it via a wrapper library, or via electron or something. But that's the same thing, just with more indirection.
...your troubles are starting
I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both. The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, e…
As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech .
Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft
I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both. The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, e…
As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso . As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech . Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Win…
Earlier quoted context omitted.
As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso . As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech . Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Win…
That sounds like it'll be easier to maintain. Will it also be faster?
Compared to OpenVINO (I tried ONNX runtime too, but never got it produce correct outputs with my quantized models) it is very nice to be able to build the exact kernels I need, at the quantization settings and precision that works for the models I have and with the custom operators required (speech models do a lot of non-standard stuff), run from a single set of sources, and not have to ship a hefty third-party DLL, and having to deal with their memory leaks and other stability issues.
Earlier quoted context omitted.
> Once you have allowed that proprietary cr*p into your C++ codebase People have been doing that all the time for every kind of codebase. It's just part of the business. I don't see how it's worth having any emotions or opinions about it. Seems like you are wasting your energy. Are win32 APIs proprietary? So you decide to use them, use a wrapper/UI framework, or don't develop for Windows. Easy choice. Developing for…
Many software engineers forget they're employee of a business .
Earlier quoted context omitted.
> Are win32 APIs proprietary? Yes. And crap. Not in my code bases.
The CPU on most machines is quite proprietary. I don’t understand this faux purity dogma. Practical computing is not and never has been an abstract pure concept. It’s about making machines built by corporations to do usefull things at scale. There is no ”non proprietary” computing unless you make your own stack.
Earlier quoted context omitted.
A stable instruction set would be nice. But hi, if I spent $10,000 on a piece of hardware, let me program the metal, thanks.
I've been programming GPUs since the PlayStation1. The way they work under the hood has changed fundamentally maybe 4 times in that span. I can't compare it to changes I've seen in CPU architecture since then. Maybe like: Compare the NES with its 6502 and per-cartridge mappers vs. a IBM 386 PC. Now repeat that shift 2 or 3 more times.
How many times since we got shaders? And when exactly? I bet each change takes longer to come than the last. I mean, GPUs are increasingly general purpose nowadays. Sure they're optimised to specific kinds of embarrassingly parallel problems, but as more and more of their capabilities move out of the fixed pipeline to shaders, the need to change lessens.
Okay, so, GPUs are taking one more step towards being general purpose massively parallel machines. That's cool. What would be even cooler though would be for GPU vendors to start giving us the user manual. An I mean the real user manual, that explains how to use their piece of metal when all you have is that piece of metal . That means a precise description of the wire protocols, the data format of the buffers we sen…
> GPUs are taking one more step towards being general purpose massively parallel machines this has nothing to do with becoming more general purpose (GPUs will never be general purpose - it's literally physically impossible).
Well in that sense, neither will CPUs. Just like GPUs, some workloads are better left to other kinds of hardware.
Earlier quoted context omitted.
Yes but there are business costs to using high-level proprietary tools and libraries. If you write your app using win32, you won’t be able to port is very easily. You’re also stuck with whatever bad or bizarre decisions Microsoft made. It’s even worse for CUDA. GPUs are expensive, and now you’re vendor locked. You’re between a rock and a hard place. Either spend millions in engineering time, or millions on price-gaug…
” If you write your app using win32, you won’t be able to port is very easily.” This is wrong way around. If you don’t support the platform your app runs on using the native api:s to the hilt your port is just bad. If you actually want to support multiple platforms _you actually need to support_ them from the ground up. This is speaking industrially and businesswise. A professional software business always has per-pl…
And Re: CUDA: yes if it doesn’t make sense then dont use it. That’s sort of my whole argument. It might make some level of sense from a technical perspective, but that needs to be balanced with business risk. I’m saying a lot of people aren’t doing the balancing right, which is why these new tools have value.