Python vs Common Lisp, workflow and ecosystem (2019) • Hacker News Board

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#61

Post by lukashrb » Sun, May 02, 2021, 6:05 PM UTC

mark_l_watson wrote: post #34

Nice article. One thing, re “In Python we typically restart everything at each code change“: I sometimes run Python in Emacs with a REPL. I evaluate region to pick up edits. Not bad. The big win for the Common Lisp REPL is being able to modify data, do restarts, etc. I usually use Common Lisp, but for right now I am heavily using Clojure to write examples for a new Clojure AI book that I am writing. I miss the Common…

The first time I encountered "interactive development" was when I had to use python with jupiter notebook. I really liked this style of development. The tdd approach I practiced before was somewhat similar but by far not as visual.

Clojure was an eye opener for me and I think it offers a great developer experience (e.g. I'm addicted to C-c C-p)

It seems my journey hasn't ended and I definitely have to check out CL!

But atm it's hard for me to give up the things clojure offers to me: persistent datastructures, access to a great ecosystem and a very good designed standard library.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#62

Post by eikenberry » Sun, May 02, 2021, 7:03 PM UTC

agumonkey wrote: post #32

Not to start a flamewar, every time I see a python talk (pycon or else), with fancy tricks like metaclasses.. all I can think is that, well, CLOS would have been perfectly fit for this too. I know people are tired of the "lisp/smalltalk did it better" but what features of python are not possible (or hard) in CL[OS] ? ps: how many CL shops are out there ? I'd work near free just to try a CL team once.

> every time I see a python talk (pycon or else), with fancy tricks like metaclasses..

All I can think of is what a mistake those features were in python. "Fancy tricks" are generally the author trying to be clever (in the Kernighan sense) and ends up obfuscating the result. Not saying it works this way in CL, I don't have the experience to make the call, but in Python it was (and probably still is) prevalent.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#63

Post by jgilias » Sun, May 02, 2021, 7:07 PM UTC

tluyben2 wrote: post #22

jgilias wrote: post #11

Earlier quoted context omitted.

But that's fine, no? I mean, it's a pretty common workflow where the people close to the science part of something write a prototype in their language/ecosystem of choice, and then the engineering side is in charge of taking the prototype implementation and making it performant enough for production use. Finding people who know both, data science, and low level programming languages well enough to be able to implemen…

Yes, I guess it is fine if that is the flow. I just didn't expect it upfront (my bad).

Yeah, if it's actually OK or not depends a lot on the particulars. Like, if it's not actually your job, and the data people were supposed to produce production ready stuff themselves, and then you have to go out of your way to actually make it work, then it's not OK. But that's more to do with how organizations function, not technical merits of the involved programming languages.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#64

Post by jhayward » Sun, May 02, 2021, 7:23 PM UTC

enriquto wrote: post #17

7thaccount wrote: post #14

Earlier quoted context omitted.

Very much actively doing so on my end where nearly all work in the industry is in Python, with some Matlab, C, C++, and Julia sprinkled in. Python is a great high level language for basically everything, but hardcore low latency apps. I can parse text, connect to databases, do sparse matrix computations on massive matrices, calculate network flows, generate large node-graph diagrams, use a Python based API to connect…

> do sparse matrix computations on massive matrices This is completely impossible to do in the Python language, unless you resort to external tooling written in C or Fortran. Sure, you can call these codes from Python, as you can call them from any other language.

Every single bit of Python relies on code written in C. What is it that you are trying to refute here?

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#65

Post by kuratkull » Sun, May 02, 2021, 7:28 PM UTC

odonnellryan wrote: post #29

tluyben2 wrote: post #9

Earlier quoted context omitted.

> and is fast because the speedy parts aren't in Python. Having worked months with a slew of senior data scientists, this was a bit painful. Python is so slow and those data scientists were very good at coming up with solutions for the issues of the company, but the implementations (using Spacy, Pandas and other libs) had enough Python in them to make them not practical for the company use case. Nice prototypes which…

I've worked in this industry for a bit and had the opposite experience. Most of the slow solutions have been in other languages due to poor design and algo choice. There have been several projects I've been able to rewrite from R to Python where the runtime on a workstation went from days with R to seconds minutes with Python in a tiny tiny VM (like $10 DO box). Sure maybe a 3 minute task in python that reconciles a…

So the R code was pretty bad and your solution was more optimal - that's the only bit of information in here. The point is that rewriting something means you have extra domain knowledge, bottleneck knowledge, etc that you didn't have during the initial write. Tt might have gone the other way too, initial write in Python is too slow, rewrite in R is faster.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#66

Post by enriquto » Sun, May 02, 2021, 8:28 PM UTC

jhayward wrote: post #64

enriquto wrote: post #17

Earlier quoted context omitted.

> do sparse matrix computations on massive matrices This is completely impossible to do in the Python language, unless you resort to external tooling written in C or Fortran. Sure, you can call these codes from Python, as you can call them from any other language.

Every single bit of Python relies on code written in C. What is it that you are trying to refute here?

> Every single bit of Python relies on code written in C.

There's pypy, a jit python interpreter writen entirely in python, and it does not depend in C. It is also much faster than the common interpreter, cpython. Unfortunately it is still not appropriate for numerical computation, as the language itself makes working directly with numbers very cumbersome (and this was the point I wanted to make).

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#67

Post by pjmlp » Sun, May 02, 2021, 8:36 PM UTC

PeterStuer wrote: post #4

I like Lisp, and I'm not a fan of e.g. Python's whitespace sensitivity. That said, for niches such as ML and data science, I find you just can't beat the Python ecosystem.

Many of those libraries are implemented in C++, any language can call them.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#68

Post by 7thaccount » Sun, May 02, 2021, 8:46 PM UTC

enriquto wrote: post #17

7thaccount wrote: post #14

Earlier quoted context omitted.

Very much actively doing so on my end where nearly all work in the industry is in Python, with some Matlab, C, C++, and Julia sprinkled in. Python is a great high level language for basically everything, but hardcore low latency apps. I can parse text, connect to databases, do sparse matrix computations on massive matrices, calculate network flows, generate large node-graph diagrams, use a Python based API to connect…

> do sparse matrix computations on massive matrices This is completely impossible to do in the Python language, unless you resort to external tooling written in C or Fortran. Sure, you can call these codes from Python, as you can call them from any other language.

Nobody cares about what you're saying though (with respect) in this area. It's all about the ecosystem or Anaconda distribution itself rather than just the core language. I agree that what you're saying is accurate, but it also happens to be irrelevant in this particular case.

Numerical methods and data science are mostly done by engineers, mathematicians, and other random stem folks. I've yet to meet someone who is even cognizant that Numpy is really calling out to some low level C, C++, or Fortran library. They just know that you call a library like any other and the code works.

If you're trying to say that any language with FFI capabilities can do that, you'd be right, but it also doesn't matter much. Python has somehow found a sweet spot where it's easy to learn and onboard people and there is support for a lot of stuff with relatively low hassle. It certainly isn't lisp, but somehow seems to be orders of magnitude more successful.

I've been searching for a tool/language/ecosystem to replace Python for ages, but nothing ends up becoming close. I spent a significant amount of time learning lisp, but a lot of what I saw (besides the power of macros and restarting) was just a less intuitive way of doing things I could easily do in Python, Ruby, or Perl. Lisp is secret alien technology if you're coming from C or C++, but coming from Python it seems closer to a wash.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#69

Post by erik_seaberg » Sun, May 02, 2021, 10:15 PM UTC

eikenberry wrote: post #62

agumonkey wrote: post #32

Not to start a flamewar, every time I see a python talk (pycon or else), with fancy tricks like metaclasses.. all I can think is that, well, CLOS would have been perfectly fit for this too. I know people are tired of the "lisp/smalltalk did it better" but what features of python are not possible (or hard) in CL[OS] ? ps: how many CL shops are out there ? I'd work near free just to try a CL team once.

> every time I see a python talk (pycon or else), with fancy tricks like metaclasses.. All I can think of is what a mistake those features were in python. "Fancy tricks" are generally the author trying to be clever (in the Kernighan sense) and ends up obfuscating the result. Not saying it works this way in CL, I don't have the experience to make the call, but in Python it was (and probably still is) prevalent.

The goal is for a little bit of clever work to make the computer do a lot more stupid work. People get bored and make mistakes, and our brains are not getting any faster, so this kind of leverage is the only game in town.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#70

Post by enriquto » Sun, May 02, 2021, 11:19 PM UTC

7thaccount wrote: post #68

enriquto wrote: post #17

Earlier quoted context omitted.

> do sparse matrix computations on massive matrices This is completely impossible to do in the Python language, unless you resort to external tooling written in C or Fortran. Sure, you can call these codes from Python, as you can call them from any other language.

Nobody cares about what you're saying though (with respect) in this area. It's all about the ecosystem or Anaconda distribution itself rather than just the core language. I agree that what you're saying is accurate, but it also happens to be irrelevant in this particular case. Numerical methods and data science are mostly done by engineers, mathematicians, and other random stem folks. I've yet to meet someone who is…

> I've yet to meet someone who is even cognizant that Numpy is really calling out to some low level C

then you've never met anybody who builds the tools that you use. Which is alright. But if you disparage their point of view then you sound a bit funny.