It might be worth using a lightness estimate like OKLab, OKLrab[1], or CIE Lab instead of the RGB luminance weighting, as it should produce a more perceptually accurate result. The other issue with your code right now, is that it is using euclidean distance in RGB space to choose the nearest color, but it would be probably also more accurate to use a perceptual color difference metric, a very simple choice is euclide…
[1]: https://juliagraphics.github.io/Colors.jl/stable/colordiffer...
[2]: https://github.com/JuliaImages/DitherPunk.jl
[3]: https://juliaimages.org/DitherPunk.jl/stable/#Dithering-with...