Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

KlongPy: High-Performance Array Programming in Python

github.com

Post reply on HN
Print view View original

61–70 of 90 posts

Re: KlongPy: High-Performance Array Programming in Python

#61

Post by MPSimmons »

itishappy wrote: post #25
MPSimmons wrote: post #14

Earlier quoted context omitted.

oooooooh I think I'm starting to see... sum::{} is the creation of a monadic function (which I just learned about because I don't have a CS background) :" is a comment sum([...]) is calling the function with a list, and the function will iterate over the list adding each of the elements count::{#x} is another monadic function that returns the number of elements, which then gets called

I'm new too, so take my opinion with a grain of salt, but I think you're right with one minor correction: name::{body} :" function declaration This isn't necessarily niladic or monadic or dyadic on it's own, it depends on the body. The bodies from the example all just happened to be monadic. +/x :" monadic : sum over argument x #x :" monadic : count of the argument x + :" dyadic : sum !10 :" niladic : an array from 0…

Man, that's so weird. Thanks for the clarification.

Re: KlongPy: High-Performance Array Programming in Python

#62

Post by CyberDildonics »

bunderbunder wrote: post #60
CyberDildonics wrote: post #59

Earlier quoted context omitted.

What are doing that is computationally intensive and are using python and a CPU with 64MB of cache for, but is under 64 MB total? How do you know you aren't losing performance by doing single operations on each array in python?

Model training and inference, for example. Despite the hype, there's actually a lot of data science work that uses modestly-sized sets of structured data and doesn't really need a billion parameter deep learning model. And even when you're running inference on larger datasets it's often fine to just use mini batches. I don't really think of it as a "losing" performance problem. It's all about tradeoffs. SIMD tooling…

It seems like instead of a "silent majority doing intensive calculations" what you're really saying is "I don't think of slower speeds as a performance loss, I don't pay attention to that, I don't care about slower speeds, my data is small and my programs run almost instantly".

If someone says "this isn't really high performance" and you say "I don't personally care, my stuff is small and doesn't take long time to run" that isn't a coherent reply.

Re: KlongPy: High-Performance Array Programming in Python

#63

Post by RestartKernel »

I've been meaning to give something APL-ish a shot, but I don't see how such languages can be justified in a non-academic engineering environment. I doubt any company would like to limit themselves to hiring programmers who are capable of maintaining such code.

Re: KlongPy: High-Performance Array Programming in Python

#66

Post by bunderbunder »

CyberDildonics wrote: post #62
bunderbunder wrote: post #60

Earlier quoted context omitted.

Model training and inference, for example. Despite the hype, there's actually a lot of data science work that uses modestly-sized sets of structured data and doesn't really need a billion parameter deep learning model. And even when you're running inference on larger datasets it's often fine to just use mini batches. I don't really think of it as a "losing" performance problem. It's all about tradeoffs. SIMD tooling…

It seems like instead of a "silent majority doing intensive calculations" what you're really saying is "I don't think of slower speeds as a performance loss, I don't pay attention to that, I don't care about slower speeds, my data is small and my programs run almost instantly". If someone says "this isn't really high performance" and you say "I don't personally care, my stuff is small and doesn't take long time to ru…

It's really more about acknowledging that, unless you're explicitly talking about something like HPC, "high" is a relative term. It quite reasonably means different things to different people in different contexts. In Python I can often improve a task's run time by multiple orders of magnitude, getting its run time from "irritating" to "not really worth thinking about" in the process, just by making sure I'm doing vectorized operations that can make use of SIMD instructions in the appropriate places. That's more than high enough performance for my needs in most cases. A similar situation may be true for the author of this library.

I find it to be much less coherent, nearly to the point of knocking down straw men, when people get lost in arguing semantics over a term that we should all know by now is perennially ill-defined.

Re: KlongPy: High-Performance Array Programming in Python

#69

Post by CyberDildonics »

bunderbunder wrote: post #66
CyberDildonics wrote: post #62

Earlier quoted context omitted.

It seems like instead of a "silent majority doing intensive calculations" what you're really saying is "I don't think of slower speeds as a performance loss, I don't pay attention to that, I don't care about slower speeds, my data is small and my programs run almost instantly". If someone says "this isn't really high performance" and you say "I don't personally care, my stuff is small and doesn't take long time to ru…

It's really more about acknowledging that, unless you're explicitly talking about something like HPC, "high" is a relative term. It quite reasonably means different things to different people in different contexts. In Python I can often improve a task's run time by multiple orders of magnitude, getting its run time from "irritating" to "not really worth thinking about" in the process, just by making sure I'm doing ve…

It's not ill defined to everyone else. If I say my phone is fast but it's nowhere close to the speed of a new phone, no one else would say it's fast.

High performance means you show that its speed is competitive against the other fastest programming methods.

You labeling something "high performance" because you wrote something slow and made it faster has nothing to do with releasing something and calling it "high performance" publicly.

Re: KlongPy: High-Performance Array Programming in Python

#70

Post by eismcc »

jayavanth wrote: post #67
eismcc wrote: post #31

KlongPy author here: AMA

why are so many array programming alternatives to numpy/scipy popping up recently? Is there a fundamental flaw or showstopper in numpy?

Numpy is just the backend and KlongPy has a lot more application features than Numpy. You can see something like kdb+ as inspiration.
Post reply on HN