Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Julia and Mojo Mandelbrot Benchmark

discourse.julialang.org

Post reply on HN
Print view View original

61–70 of 168 posts

Re: Julia and Mojo Mandelbrot Benchmark

#61

Post by simondanisch »

blacklion wrote: post #54

I'm sorry, but it is not benchmark, it is farce. One run, 7ms, 2ms? It is statistical fluctuation, not data, especially if it was run on "typical" developer laptop under "typical" session where browsers and other high-hitters are run in background and all these turbo-boosts and freq-governors are not turned off. You need OS where almost all software (including most system services) are killed, CPU frequency is fixed…

This is using BenchmarkTools which does most of these things ;) And the numbers are very nicely reproducable. But yes, those benchmarks are hard to compare, especially if they dont run on the same machine! But, there is absolutely no reason to believe that Julia can't get optimal performance on a given hardware knowing Julia's compiler and architecture...This is much more a benchmark of Mojo, which hasn't been proven in that regard.

Re: Julia and Mojo Mandelbrot Benchmark

#62

Post by suavesito »

blacklion wrote: post #54

I'm sorry, but it is not benchmark, it is farce. One run, 7ms, 2ms? It is statistical fluctuation, not data, especially if it was run on "typical" developer laptop under "typical" session where browsers and other high-hitters are run in background and all these turbo-boosts and freq-governors are not turned off. You need OS where almost all software (including most system services) are killed, CPU frequency is fixed…

I think you might not be familiar with the package used to benchmark Julia [1].

It does not fix processes to CPU's, or set kernel governor to performance, and there are fluctuations from usage of the computer. But it does run the function for several seconds and returns the distribution of the runs (the little graphics underneath the benchmarks). It calculates standard deviation and if some runs are too small (sub-nano seconds) it emits warnings saying the results might be caused by inlining and constant propagation.

The differences in runtimes you refer to are from use of different machines or different routines, which is completely expected. They also argue they need to run the Mojo code in the same machine as the Julia code to be able to give meaningful results and comparisons.

While to someone outsider it might be seen as done without care, I can asure you that this people are used to take extreme care on how they do benchmarks. Again, it might just be that you're not familiar with the tooling developed to do it.

I do think there is more benchmarks needed to be done, as the Mojo code hasn't be optimised yet and none in that thread was able to run both the Julia code and Mojo code in the same machine (outside of the OP). But I'm sure this will be done (I guess rather sooner than later). :)

[1] Documentation of the package used for benchmark https://juliaci.github.io/BenchmarkTools.jl/stable/ Here you can find all the information you have said in your comment, and more, about reproducibility of benchmarks in different environments. White paper about the strategies used by the package https://arxiv.org/abs/1608.04295

Re: Julia and Mojo Mandelbrot Benchmark

#63

Post by msravi »

Oreb wrote: post #50
eigenspace wrote: post #42

Earlier quoted context omitted.

What do you mean by lisp-like? If you mean “primarily uses S-expressions” then I guess I dont really see why that’s so important to you. If you mean a language that is semantically similar to lisps and learned a lot of the important lessons that Lisp taught the programming world, I think Julia is one of the Lispiest languages in this space right now. The syntax may not be S-expression based on the surface, but our Ex…

I certainly consider Julia to be a Lisp, and I’m pretty sure that’s what the person you responded meant, too. His point remains true: Julia appears to have little chance of overtaking Python, except in some tiny niche areas. And even in these niche areas, I fear that Julia will end up losing to Mojo. I really hope I am wrong. I love Julia and would like to see it succeed everywhere, but it doesn’t seem to be happenin…

> I certainly consider Julia to be a Lisp

Can you elaborate some more on this? My worldview assumed that a lisp used a list as a primary code/data structure and Julia doesn't seem to be doing that... Of course it does provide a way to manipulate code and data because of its macros. But what makes a lisp a lisp?

Re: Julia and Mojo Mandelbrot Benchmark

#64

Post by ayhanfuat »

suavesito wrote: post #57
ayhanfuat wrote: post #11

Earlier quoted context omitted.

Not if you want to avoid condescending "I cannot look at the Python code my eyes hurt" comments. Good to know the Julia community hasn't made any progress in that regard, though.

I think this is a little bit unfair. The comment refers to the Mojo specific use of [], not to regular Python. It also starts saying > I know I shouldn’t say so but I can’t help... Remarking that the comment should not be taken too seriously, as it might be inappropriate. Finally, saying the whole community is condescending given 1 in 32 comments is... a little rounding up from the statistics there.

Seems like we are speaking from different experiences. As someone who witnessed multiple requests like “please avoid hyperbole when it comes to criticizing other languages” in Slack, Discourse or Twitter, I interpret “I know I shouldn’t say so” differently.

I would say 1 in 32 is also about the experience. I stopped visiting Discourse, chatting in Slack because I found it exhausting that every time Python is mentioned someone came up with a different way of saying how much they hate Python. I know I wasn’t the only one disturbed by this but in the end communities make their own choices.

Sorry about missing the mojo specific [].

Re: Julia and Mojo Mandelbrot Benchmark

#65

Post by martinsmit »

msravi wrote: post #63
Oreb wrote: post #50

Earlier quoted context omitted.

I certainly consider Julia to be a Lisp, and I’m pretty sure that’s what the person you responded meant, too. His point remains true: Julia appears to have little chance of overtaking Python, except in some tiny niche areas. And even in these niche areas, I fear that Julia will end up losing to Mojo. I really hope I am wrong. I love Julia and would like to see it succeed everywhere, but it doesn’t seem to be happenin…

> I certainly consider Julia to be a Lisp Can you elaborate some more on this? My worldview assumed that a lisp used a list as a primary code/data structure and Julia doesn't seem to be doing that... Of course it does provide a way to manipulate code and data because of its macros. But what makes a lisp a lisp?

Here's a meme that might help: https://www.reddit.com/r/LispMemes/comments/irkm5m/nobody_li...

Re: Julia and Mojo Mandelbrot Benchmark

#66

Post by goerz »

Oreb wrote: post #50
eigenspace wrote: post #42

Earlier quoted context omitted.

What do you mean by lisp-like? If you mean “primarily uses S-expressions” then I guess I dont really see why that’s so important to you. If you mean a language that is semantically similar to lisps and learned a lot of the important lessons that Lisp taught the programming world, I think Julia is one of the Lispiest languages in this space right now. The syntax may not be S-expression based on the surface, but our Ex…

I certainly consider Julia to be a Lisp, and I’m pretty sure that’s what the person you responded meant, too. His point remains true: Julia appears to have little chance of overtaking Python, except in some tiny niche areas. And even in these niche areas, I fear that Julia will end up losing to Mojo. I really hope I am wrong. I love Julia and would like to see it succeed everywhere, but it doesn’t seem to be happenin…

I’m not sure I would consider the scientific computing that Julia targets a “tiny” niche. It’s worth remembering that Python is 30 years older than Julia, and Julia has only been in a really usable state for 4 or 5 years. You can’t expect it to displace Numpy/Scipy/etc overnight. Especially if you include machine learning, where there’s just huge momentum with large corporations having massively invested in Python frameworks for at least 10 years. Subjectively, I’m seeing quite a lot of growth in Julia. It’s certainly a much stronger language than the Python/C++ combination that currently has the biggest market share in that area.

Also, there’s nothing wrong with niches. Julia is undoubtedly less of a general-purpose language like Python, but it very much shines in its domain.

Re: Julia and Mojo Mandelbrot Benchmark

#67

Post by DNF2 »

rvz wrote: post #13

As long as you can copy and paste Python code to Mojo and it is 1:1 compatible with all your existing libraries and is hundreds of times faster than Python, that is much better than wasting time rewriting it in another language that is 8x faster than Mojo (in its first release) with hand-optimizations from Julia language experts. I expect Mojo's first release to be fast enough that it would get the Python folks using…

Please actually read the Mojo code. It is full of complex hand-optimized simd instructions.

By comparison, the simd-optimized Julia code (especially the first version) is significantly more elegant and transparent.

Impressively, the ComplexSIMD Julia class was defined in a few simple lines, from scratch. I wonder what the, apparently built-in, complex simd functionality in Mojo looks like under the hood.

Re: Julia and Mojo Mandelbrot Benchmark

#68

Post by suavesito »

msravi wrote: post #63
Oreb wrote: post #50

Earlier quoted context omitted.

I certainly consider Julia to be a Lisp, and I’m pretty sure that’s what the person you responded meant, too. His point remains true: Julia appears to have little chance of overtaking Python, except in some tiny niche areas. And even in these niche areas, I fear that Julia will end up losing to Mojo. I really hope I am wrong. I love Julia and would like to see it succeed everywhere, but it doesn’t seem to be happenin…

> I certainly consider Julia to be a Lisp Can you elaborate some more on this? My worldview assumed that a lisp used a list as a primary code/data structure and Julia doesn't seem to be doing that... Of course it does provide a way to manipulate code and data because of its macros. But what makes a lisp a lisp?

I think the commenter refers to the fact that in Julia, code is data, even if it is not represented as lists. This allows the existence of macros, which are syntactic sugar to functions modifying code. Also that most (all?) lisps have multiple dispatch as a fundamental part of the language, and Julia does too.

Re: Julia and Mojo Mandelbrot Benchmark

#69

Post by suavesito »

ubj wrote: post #12
arbitrandomuser wrote: post #9

Earlier quoted context omitted.

If anyone is interested in compiling small binaries with Julia do check out staticcompiler.jl and supporting statictools.jl that manages to produce small binaries without the Julia runtime , Ofcourse it's a wip , and not fully mature , I'm just putting it out there for people to know. There's some really cool demonstrations come out of this ... Wasm fluid simulation in Julia: https://alexander-barth.github.io/FluidSi…

Yep, I was aware of StaticCompiler.jl. I wish it was more mature. Static compilation is indeed possible with Julia. But it's very limited in its capabilities and certainly not as effortless as a simple `mojo build myfile.mojo`.

Well, it is as simple as that... using PackageCompiler.jl [1] (PkgC.jl). It does create huge executable (you can easily trim them), but those are relocatable and portable between machines as they include most (if not all) of the dependencies needed to run them. They are already used in production in several places. I don't have the link at hand right now, but maybe someone else might jump to give the link to talks given by Chris Rackaukas on this.

From my personal experience. I've done graphical apps in GTK3 in Julia with PkgC.jl cross-compiling from Linux to Windows. And they worked. :)

[1] https://julialang.github.io/PackageCompiler.jl/stable/

Re: Julia and Mojo Mandelbrot Benchmark

#70

Post by DNF2 »

suavesito wrote: post #62
blacklion wrote: post #54

I'm sorry, but it is not benchmark, it is farce. One run, 7ms, 2ms? It is statistical fluctuation, not data, especially if it was run on "typical" developer laptop under "typical" session where browsers and other high-hitters are run in background and all these turbo-boosts and freq-governors are not turned off. You need OS where almost all software (including most system services) are killed, CPU frequency is fixed…

I think you might not be familiar with the package used to benchmark Julia [1]. It does not fix processes to CPU's, or set kernel governor to performance, and there are fluctuations from usage of the computer. But it does run the function for several seconds and returns the distribution of the runs (the little graphics underneath the benchmarks). It calculates standard deviation and if some runs are too small (sub-na…

Are you sure the Mojo code hasn't been optimized? It seems to be hand-tuned with simd operations and multi-threading.
Post reply on HN