Deep Neural Networks from Scratch in Zig
monadmonkey.com
Deep Neural Networks from Scratch in Zig
1–10 of 34 posts
Re: Deep Neural Networks from Scratch in Zig
#2Awesome, thanks for the write up! Profiling would be a great addition to see.
Re: Deep Neural Networks from Scratch in Zig
#3[dead]
Re: Deep Neural Networks from Scratch in Zig
#4I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance.
EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?
Re: Deep Neural Networks from Scratch in Zig
#5I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance. EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?
Why would any DNN ever need to deallocate other than at program termination? (And allocate other than at startup)
Re: Deep Neural Networks from Scratch in Zig
#6I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance. EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?
Why would any DNN ever need to deallocate other than at program termination? (And allocate other than at startup)
That's the point. With an arena allocator, you don't "really" de-allocate the OS memory, you just free the bytes to be re-used again in the next iteration (by using a FixedBufferAllocator as the backing allocator, if you know how much memory will be needed in advance). This is much better than trying to re-use the structs manually as the OP mentioned at the end.
EDIT: FixedBufferAllocator has a reset method , so maybe you just need to use that to "free" all memory on each iteration? Perhaps using that directly is enough if you know the max memory usage at compile-time... I was just thinking that it should be possible to use an arena allocator to basically tell the delegate allocator to make all memory available again without freeing it back to the OS?! If anyone knows more about allocators I would be happy to know this. https://ziglang.org/documentation/master/std/#A;std:heap.Fix...
Re: Deep Neural Networks from Scratch in Zig
#7I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance. EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?
I just tried and they downloaded for me...
Re: Deep Neural Networks from Scratch in Zig
#8I didn't investigate this more deeply , but I think if you used an arena allocator on each iteration of the loop, and simply freed the whole allocator after each iteration, you might get much better performance. EDIT: the MNIST link requires authentication: http://yann.lecun.com/exdb/mnist/ How can we get access to the test run at the end?
> the MNIST link requires authentication
doesn't ask for auth for me?
Re: Deep Neural Networks from Scratch in Zig
#9> I would guess the constant memory allocation and frees in the training loop are the bottleneck
No, the bottleneck would be not utilizing the idling GPU.
Re: Deep Neural Networks from Scratch in Zig
#10It would be cool too see a port of llama.cpp and ggml to Zig