Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Defense of Matlab Code

runmat.org

Post reply on HN
Print view View original

151–160 of 174 posts

Re: In Defense of Matlab Code

#151

Post by boscillator »

nallana wrote: post #22
boscillator wrote: post #4

Interesting... I wrote a similar post about MATLAB's syntax a while ago, and I still think MATLAB is one of the best calculators on the market. RunMat is an interesting idea, but a lot of MATLAB's utility comes from the toolboxes, and unless RunMat supports every single toolbox I need, I'm going to be reaching for that expensive MATLAB license over and over again.

Yep! Makes sense. Though I think the cost of writing these toolboxes is lim --> 0. Will have a really solid rust inspired package manager soon, and a single #macro to expose a rust function in the RunMat script's namespace (= easy to bring any aspects of the rust ecosystem to RunMat).

I wouldn't be so sure that writing those toolboxes is cheap. You need an aerospace engineer to write the aero toolbox, or you are going to miss subtleties. I assume you need a biologist to write the biology toolboxes. All of these domain experts are really expensive, and I would not trust a toolbox that hadn't been review by them.

Re: In Defense of Matlab Code

#152

Post by doawoo »

drnick1 wrote: post #111
doawoo wrote: post #90

I want to come out and say that a long time ago at a startup we needed to generate a very particular type of analysis graph for a human operator to review in our SaaS. and I just straight up installed GNU Octave on the server and called out to it from python, using the exact code the mathematician had devised.

These days however with all the AI coding tools that are available, it probably makes more sense to just ask Claude to port the Matlab/Octave script to Python and directly integrate it into your program. Numpy/Scipy often provide drop-in replacements for Matlab functions, even the names are the same in some cases. I have gone further and asked AI to port working but somewhat slow numerical scripts to C++ and it's com…

See the benefit of just using what the original mathematician wrote, is that if they had a problem with a way the graph was rendering, or they wanted to tweak it, they just had to edit the code, no translation layer needed. It shipped like any other component of the product at the time.

Re: In Defense of Matlab Code

#153

Post by SatvikBeri »

sbrother wrote: post #106
SatvikBeri wrote: post #33

Earlier quoted context omitted.

> Python is sometimes slower (hot loops), but for that you have Numba This is a huge understatement. At the hedge fund I work at, I learned Julia by porting a heavily optimized Python pipeline. Hundreds of hours had gone into the Python version – it was essentially entirely glue code over C. In about two weeks of learning Julia, I ported the pipeline and got it 14x faster. This was worth multiple senior FTE salaries.…

> People usually use C++ or Julia. All of the fastest answers are in Julia That's surprising to me and piques my interest. What sort of pipeline is this that's faster in Julia than C++? Does Julia automatically use something like SIMD or other array magic that C++ doesn't?

To be clear, the fastest theoretically possible C++ is probably faster than the fastest theoretically possible Julia. But the fastest C++ that Senior Data Engineer candidates would write in ~2 hours was slower than the fastest Julia (though still pretty fast! The benchmark for this problem was 10ms, and the fastest C++ answer was 3 ms, and the top two Julia answers were 2.3ms and .21ms)

The pipeline was pretty heavily focused on mathematical calculations – something like, given a large set of trading signals, calculate a bunch of stats for those signals. All the best Julia and C++ answers used SIMD.

Re: In Defense of Matlab Code

#154

Post by SatvikBeri »

drnick1 wrote: post #109
SatvikBeri wrote: post #33

Earlier quoted context omitted.

> Python is sometimes slower (hot loops), but for that you have Numba This is a huge understatement. At the hedge fund I work at, I learned Julia by porting a heavily optimized Python pipeline. Hundreds of hours had gone into the Python version – it was essentially entirely glue code over C. In about two weeks of learning Julia, I ported the pipeline and got it 14x faster. This was worth multiple senior FTE salaries.…

The C++ devs at your firm must be absolutely terrible if a newcomer using a scripting language can write faster software, or you are not telling the whole story. All of NumPy, Julia, MATLAB, R, and similar domain-specific, user-friendly libraries and platforms use BLAS and LAPACK for numerical calculations under the hood with some overhead depending on the implementation, so a reasonably optimized native implementati…

Are you aware that Julia is a compiled language with a heavy focus on performance? It is not in the same category as NumPy/MATLAB/R

Re: In Defense of Matlab Code

#155

Post by IAmBroom »

esalman wrote: post #150
D-Machine wrote: post #115

Earlier quoted context omitted.

"I have direct experience of universities doing horrifyingly wasteful computations" is not the ringing endorsement for Matlab you might think it to be... Granted, I've seen Python horrors on university HPC clusters too, but at least there are libraries and clear documentation (e.g. Lightning, Ray, etc) for how to properly manage these things. Good luck finding that with Matlab.

Universities are not wasteful. University graduates earn more and face fewer unemployment than high school graduates. More universities correlates with higher GDP per capita.

GP said "universities doing horrifyingly wasteful computations".

You claimed they asserted that "Universities are wasteful".

Put the goalposts back where they were.

Re: In Defense of Matlab Code

#156

Post by SatvikBeri »

pbowyer wrote: post #131
SatvikBeri wrote: post #33

Earlier quoted context omitted.

> Python is sometimes slower (hot loops), but for that you have Numba This is a huge understatement. At the hedge fund I work at, I learned Julia by porting a heavily optimized Python pipeline. Hundreds of hours had gone into the Python version – it was essentially entirely glue code over C. In about two weeks of learning Julia, I ported the pipeline and got it 14x faster. This was worth multiple senior FTE salaries.…

> Part of our interview process is a take-home where we ask candidates to build the fastest version of a pipeline they possibly can. People usually use C++ or Julia. All of the fastest answers are in Julia. It would be fun if you could share a similar pipeline problem to your take-home (I know you can't share what's in your interview). I started off in scientific Python in 2003 and like noodling around with new progr…

The closest publicly available problem I can think of is the 1 billion rows challenge. It's got a bigger dataset, but with somewhat simpler statistics – though the core engineering challenges are very similar.

https://github.com/gunnarmorling/1brc

Re: In Defense of Matlab Code

#157

Post by SatvikBeri »

jakobnissen wrote: post #119
sbrother wrote: post #106

Earlier quoted context omitted.

> People usually use C++ or Julia. All of the fastest answers are in Julia That's surprising to me and piques my interest. What sort of pipeline is this that's faster in Julia than C++? Does Julia automatically use something like SIMD or other array magic that C++ doesn't?

I use Rust instead of C++, but I also see my Julia code being faster than my Rust code. In my view, it's not that Julia itself is faster than Rust - on the contrary, Rust as a language is faster than Julia. However, Julia's prototyping, iteration speed, benchmarking, profiling and observability is better. By the time I would have written the first working Rust version, I would have written it in Julia, profiled it, m…

This is pretty much it – when we had follow up interviews with the C++ devs, they had usually only had time to try one or two high-level approaches, and then do a bit of profiling & iteration. The Julia devs had time to try several approaches and do much more detailed profiling.

Re: In Defense of Matlab Code

#158

Post by zackmorris »

nallana wrote: post #19
mNovak wrote: post #12

As an engineer, I use Matlab (or rather, Octave the free equivalent) all the time. It's really great for numerical computing and plotting. Most things 'just work', there's a sizeable collection of packages, and I personally like how flexible the function inputs are. Biggest drawback though is that it's over-optimized for matrix math, that it forces you to think about everything as matrices, even if that's not how you…

@mNovak -- super helpful note! Thank you! Author of RunMat (this project) here -- > The first thing they teach about performant Matlab code is that simple for-loops will tank performance. Yes! Since in RunMat we're building a computation graph and fusing operations into GPU kernels, we built the foundations to extend this to loop fusion. That should allow RunMat to take loops as written, and unwrap the matrix math in…

Piggybacking also to say that I hope you succeed, as your work aligns closely with the type of runtime that I had hoped to write someday when I first used MATLAB in the early 2000s (now mostly GNU Octave for small hobby projects).

The loop fusion idea sounds amazing. Another point of friction which I ran into is that MATLAB uses 1-based offsets instead of 0-based offsets for matrices/arrays, which can make porting code examples from other languages tricky. I wish there was a way to specify the offset base with something like a C #define or compiler directive. Or a way to rewrite code in-place to use the other base, a bit like running Go's gofmt to format code. Apologies if something like this exists and I'm just too out of the loop.

I'd like to point out one last thing, which is that working at the fringe outside of corporate sponsorship causes good ideas to take 10 or 20 years to mature. We all suffer poor tooling because the people that win the internet lottery pull up the ladder behind them.

Re: In Defense of Matlab Code

#159

Post by esalman »

IAmBroom wrote: post #155
esalman wrote: post #150

Earlier quoted context omitted.

Universities are not wasteful. University graduates earn more and face fewer unemployment than high school graduates. More universities correlates with higher GDP per capita.

GP said "universities doing horrifyingly wasteful computations". You claimed they asserted that "Universities are wasteful". Put the goalposts back where they were.

To find the goalpost look at the parent comment of the one you first saw.

Re: In Defense of Matlab Code

#160

Post by D-Machine »

esalman wrote: post #159
IAmBroom wrote: post #155

Earlier quoted context omitted.

GP said "universities doing horrifyingly wasteful computations". You claimed they asserted that "Universities are wasteful". Put the goalposts back where they were.

To find the goalpost look at the parent comment of the one you first saw.

Sorry if I was unclear, but GP is correct, you are misreading quite deeply.

I am saying that because it is much harder to find good documentation on using MATLAB on HPCs, a lot of computations on HPCs that use MATLAB are highly wasteful compared to if they had been written using a language and/or tools that make it much easier to use HPC resources more efficiently. I was NOT in any way saying that "universities are wasteful".

Post reply on HN