Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python vs Common Lisp, workflow and ecosystem (2019)

lisp-journey.gitlab.io

Post reply on HN
Print view View original

11–20 of 83 posts

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#11

Post by jgilias »

tluyben2 wrote: post #9
gustavo-fring wrote: post #7

Earlier quoted context omitted.

Yeah, they're very much doing it. Pandas is huge, libraries like Spacy, NetworkX, etc exist. It's a massive and good ecosystem. Python is the goto for scientific computing in most of the sciences for newer students I'd hazard a guess over the older R and Julia. This will be blindingly obvious if you work in that area. Yes, you can do it in another language, but you're missing out on a lot of stuff that is just done a…

> and is fast because the speedy parts aren't in Python. Having worked months with a slew of senior data scientists, this was a bit painful. Python is so slow and those data scientists were very good at coming up with solutions for the issues of the company, but the implementations (using Spacy, Pandas and other libs) had enough Python in them to make them not practical for the company use case. Nice prototypes which…

But that's fine, no? I mean, it's a pretty common workflow where the people close to the science part of something write a prototype in their language/ecosystem of choice, and then the engineering side is in charge of taking the prototype implementation and making it performant enough for production use. Finding people who know both, data science, and low level programming languages well enough to be able to implement data science applications directly for production is pretty hard, I'm sure.

In either case, I much prefer prototypes in Python than, say, Matlab. To speed things up I once rewrote an internal Scipy function to a version that allowed me to use it in vectorized code on my end. If the prototype is in Matlab, the optimization and integration possibilities are much more limited due to licensing, toolboxes, and the closed ecosystem in general.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#13

Post by andi999 »

jgilias wrote: post #11
tluyben2 wrote: post #9

Earlier quoted context omitted.

> and is fast because the speedy parts aren't in Python. Having worked months with a slew of senior data scientists, this was a bit painful. Python is so slow and those data scientists were very good at coming up with solutions for the issues of the company, but the implementations (using Spacy, Pandas and other libs) had enough Python in them to make them not practical for the company use case. Nice prototypes which…

But that's fine, no? I mean, it's a pretty common workflow where the people close to the science part of something write a prototype in their language/ecosystem of choice, and then the engineering side is in charge of taking the prototype implementation and making it performant enough for production use. Finding people who know both, data science, and low level programming languages well enough to be able to implemen…

Also I think it is good to be able to use the python code as testcases/validation on smaller datasets for the C code.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#14

Post by 7thaccount »

rhlap wrote: post #5
PeterStuer wrote: post #4

I like Lisp, and I'm not a fan of e.g. Python's whitespace sensitivity. That said, for niches such as ML and data science, I find you just can't beat the Python ecosystem.

Are people actually doing science with Python or are they talking about doing science? There's so much buggy low quality stuff in that space that I'd write a serious application in C or C++ from scratch. It would be a custom application, sure, but not everything needs to be general. Also, I find Lisp much more natural for mathematical reasoning.

Very much actively doing so on my end where nearly all work in the industry is in Python, with some Matlab, C, C++, and Julia sprinkled in.

Python is a great high level language for basically everything, but hardcore low latency apps. I can parse text, connect to databases, do sparse matrix computations on massive matrices, calculate network flows, generate large node-graph diagrams, use a Python based API to connect to any vendor software I've seen, do any kind of statistical analysis thing I need with pandas, amazing and free IDE allows me to use a REPL, code editor, and data structure viewer with ease, Python notebooks for education...etc etc. I've frequently found that I can rewrite a vendor's 10k line C++ program in a few pages of Python as the built-in Python data structures make text parsing extremely flexible and simple.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#15

Post by ollran »

heisig wrote: post #2

Speaking of Common Lisp - the European Lisp Symposium starts tomorrow (May 3 and May 4, https://european-lisp-symposium.org/2021/index.html ). The entire conference will be broadcast on Twitch. Python programmers are invited, too :)

Thanks for sharing this. Call-site optimization for Common Lisp seems quite interesting after looking at the article.

http://metamodular.com/SICL/call-site-optimization.pdf

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#16

Post by diggan »

BiteCode_dev wrote: post #10

He forgot: in python you read the code of somebody else and it's familiar. In lisp, you are learning a new language with every lib because each author think they are a god language designer and that their macro rock. Also they don't need a good doc cause they are obvious. Or good error message cause they never break. Also the way the author dismiss the number gap of packages available is ignoring the elephant in the…

> In lisp, you are learning a new language with every lib because each author think they are a god language designer and that their macro rock. Also they don't need a good doc cause they are obvious. Or good error message cause they never break.

You have any specific examples of Common Lisp libraries you found hard to understand? I had a hard time when first learning the language, but once I got proficient to write my own code, I found it easy to understand most libraries I ended up using myself, the same as any language really. That the REPL makes it so easy to explore them with your own context, helped a lot as well.

> Also the way the author dismiss the number gap of packages available is ignoring the elephant in the room.

In the very same section, the author describes why the number of package don't matter as much as you think it does. Curation VS free-for-all-publishing (like APT vs NPM). Add together that Common Lisp "the language" has been stable for decades, makes it much more possible to be able to use any of the libraries you find as well, where in the Python world, we both know this not to be true (just Python2 VS Python3 makes this a whole other world of messes).

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#17

Post by enriquto »

7thaccount wrote: post #14
rhlap wrote: post #5

Earlier quoted context omitted.

Are people actually doing science with Python or are they talking about doing science? There's so much buggy low quality stuff in that space that I'd write a serious application in C or C++ from scratch. It would be a custom application, sure, but not everything needs to be general. Also, I find Lisp much more natural for mathematical reasoning.

Very much actively doing so on my end where nearly all work in the industry is in Python, with some Matlab, C, C++, and Julia sprinkled in. Python is a great high level language for basically everything, but hardcore low latency apps. I can parse text, connect to databases, do sparse matrix computations on massive matrices, calculate network flows, generate large node-graph diagrams, use a Python based API to connect…

> do sparse matrix computations on massive matrices

This is completely impossible to do in the Python language, unless you resort to external tooling written in C or Fortran. Sure, you can call these codes from Python, as you can call them from any other language.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#18

Post by p_l »

diggan wrote: post #16
BiteCode_dev wrote: post #10

He forgot: in python you read the code of somebody else and it's familiar. In lisp, you are learning a new language with every lib because each author think they are a god language designer and that their macro rock. Also they don't need a good doc cause they are obvious. Or good error message cause they never break. Also the way the author dismiss the number gap of packages available is ignoring the elephant in the…

> In lisp, you are learning a new language with every lib because each author think they are a god language designer and that their macro rock. Also they don't need a good doc cause they are obvious. Or good error message cause they never break. You have any specific examples of Common Lisp libraries you found hard to understand? I had a hard time when first learning the language, but once I got proficient to write m…

There are some famous examples[1], but to be honest, they are rare outliers for libraries published. And even for internal use, the amount of "local custom style" seems pretty minimal in my experience.

[1] Off the top of my head, I can recall Cells (early reactive/dataflow system - weirdness included symbol names made to sort first in Allegro CL IDE) and hu.dwim.* stuff which had its own wrapper around CL:DEFUN and CL:DEFMETHOD, iirc. But I successfully used their stuff without caring about that.

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#19

Post by diggan »

p_l wrote: post #18
diggan wrote: post #16

Earlier quoted context omitted.

> In lisp, you are learning a new language with every lib because each author think they are a god language designer and that their macro rock. Also they don't need a good doc cause they are obvious. Or good error message cause they never break. You have any specific examples of Common Lisp libraries you found hard to understand? I had a hard time when first learning the language, but once I got proficient to write m…

There are some famous examples[1], but to be honest, they are rare outliers for libraries published. And even for internal use, the amount of "local custom style" seems pretty minimal in my experience. [1] Off the top of my head, I can recall Cells (early reactive/dataflow system - weirdness included symbol names made to sort first in Allegro CL IDE) and hu.dwim.* stuff which had its own wrapper around CL:DEFUN and C…

> There are some famous examples

As someone new to the ecosystem of Common Lisp and in general a bit sadist (ref https://www.youtube.com/watch?v=mZyvIHYn2zk), could you share which ones these are so I can enjoy not understanding them at all?

Edit: I see now after I made my comment you added examples, thanks :)

Re: Python vs Common Lisp, workflow and ecosystem (2019)

#20

Post by pydry »

tluyben2 wrote: post #9
gustavo-fring wrote: post #7

Earlier quoted context omitted.

Yeah, they're very much doing it. Pandas is huge, libraries like Spacy, NetworkX, etc exist. It's a massive and good ecosystem. Python is the goto for scientific computing in most of the sciences for newer students I'd hazard a guess over the older R and Julia. This will be blindingly obvious if you work in that area. Yes, you can do it in another language, but you're missing out on a lot of stuff that is just done a…

> and is fast because the speedy parts aren't in Python. Having worked months with a slew of senior data scientists, this was a bit painful. Python is so slow and those data scientists were very good at coming up with solutions for the issues of the company, but the implementations (using Spacy, Pandas and other libs) had enough Python in them to make them not practical for the company use case. Nice prototypes which…

Python performance is something that I see about 15-20x as much in discussions about python than I do wrestling with real life problems.
Post reply on HN