Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How did Python become the lingua franca of ML/AI?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 99 posts

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#11

Post by dustintrex »

gonab wrote: post #9
MattGaiser wrote: post #8

My last job was at an ML company. Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place). Python gets…

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

Perhaps we're talking about different sets of people? You seem to be describing the people who build ML systems, while the previous poster was talking about the people who use them. Your average data scientist most definitely does not have (or, really, even need to have) deep hardware knowledge or any understanding of networking.

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#12

Post by gonab »

dustintrex wrote: post #11
gonab wrote: post #9

Earlier quoted context omitted.

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

Perhaps we're talking about different sets of people? You seem to be describing the people who build ML systems, while the previous poster was talking about the people who use them. Your average data scientist most definitely does not have (or, really, even need to have) deep hardware knowledge or any understanding of networking.

I personally consider the term "data scientist" a very successful creation by a clever marketer

It's sexy and most of the times ...

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#13

Post by perth »

gonab wrote: post #9
MattGaiser wrote: post #8

My last job was at an ML company. Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place). Python gets…

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

The most insane thing about Python is how you can override single methods in classes and use the class like normal. One time I was working on getting FIFO working on Windows, and none of the Python built-ins were set up to handle any random process writing to a named pipe that wasn't within the same Python instance. So what I did was I took the closest implementation Python offered, which was in the multiprocessing module [1], and overrode a single one of the methods to do what I wanted it to do. The module still handled all of the cleanup so I was confident there weren't any memory leak issues, and I was able to make a simple change to the flags it was passing Windows to allow for the functionality I wanted. A language that has a hackable standard library itself is insane and I don't think I've seen it on any other language. In addition, I've found the C/C++ bindings for Python wonderful and intuitive to work with. The setup takes no effort at all and it "just works", batteries included, via ctypes.

https://github.com/python/cpython/blob/main/Lib/multiprocess...

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#14

Post by sbashyal »

I have a historical perspective on this topic. Data Science popularity was rapidly growing and "R" was the lingua franca around 2010 - 2014.

I attended Strata conference in 2014 and after visiting various technology exhibition booths there, I saw a common theme: tech companies were building data solutions using Python as R was no good for the purpose

In a meeting scheduled to share my take-aways from the conference, I predicted "Python will emerge to be the language of DataScience in few years"

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#15

Post by [deleted] »

gonab wrote: post #9
MattGaiser wrote: post #8

My last job was at an ML company. Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place). Python gets…

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

[deleted]

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#16

Post by exdsq »

gonab wrote: post #9
MattGaiser wrote: post #8

My last job was at an ML company. Most ML people there cannot build large robust systems and some struggled with the non-algorithmic bits of software. I am sure that some can out there in the world, but for the most part our ML people were very good at creating models and not very good at the development part, especially as the program grew (part of the motivation to hire devs like me in the first place). Python gets…

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

Ethical hackers have written some of the worst code I’ve ever seen. They have to know a ton about the security of frameworks, networking, etc… it’s a very complex role for sure, but they are not shining beacons of software engineering quality

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#18

Post by karmasimida »

Python was already the lingua franca before the whole deep learning/AI thing. It has numpy/scipy/pandas/scikit-learn, etc. And when did numpy happen? It was in 1996.

Arguably its biggest competitor then was R, but R is not well accepted by programmers. Yet another alternative is Matlab, but OMG, using matlab for anything string related is killing me.

While there is some history to it, Python won in the end isn't a surprise to anyone. It is simple but not toyish for real world system. I am working in one of the big techs, and Python is running the production workload for most AI services just fine.

I took a LOT of issue with dynamic typing, but for ML/AL you are going to write a lot ad-hoc data wrangling code, sometimes even Python feels verbose.

TL;DR: It had already won.

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#19

Post by gonab »

perth wrote: post #13
gonab wrote: post #9

Earlier quoted context omitted.

I don't think this is a fair assessment of "most ML people" Some of the biggest distributed systems built today are used for statistical inference or scientific computation Most "ML people" I know are highly versatile in software, networks and deep hardware knowledge, i.e., essentially they have a very good understanding of what a computer is and what is capable from Its very naive to think that you can assemble mach…

The most insane thing about Python is how you can override single methods in classes and use the class like normal. One time I was working on getting FIFO working on Windows, and none of the Python built-ins were set up to handle any random process writing to a named pipe that wasn't within the same Python instance. So what I did was I took the closest implementation Python offered, which was in the multiprocessing m…

I understand exactly what you are saying. Let me add up by saying python has a beautiful learning curve

In the begging is very simple to churn out code and do wtv you want, but the more you are into it, you start realizing that there are endless possibilities

It's a great language for beginners and even better for experts that just want to solve problems with code without thinking to much about if coding is beautiful or not, or feeling cool, or arrogant about it

It just works

Re: Ask HN: How did Python become the lingua franca of ML/AI?

#20

Post by chubot »

chubot wrote: post #6

Because Python has NumPy, which implements vectorized math on arrays and matrices. Machine learning algorithms are implemented naturally and efficiently with those primitives. PyTorch, TensorFlow, and I think every other machine learning framework in Python all use NumPy. JavaScript, Ruby, and Perl either don't have this abstraction at all, or they have much weaker versions of it, and many fewer scientific libraries.…

I should also add a cultural / social reason why Python is used in scientific computing and machine learning much more than JS/Ruby/Perl:

Python was the only one of those languages (partially) funded by government research agencies. Guido was a research programmer in the Netherlands at CWI, and then he moved to the US when he was hired by CNRI, a research agency headed by Bob Kahn (loosely connected with DARPA as far as I remember).

If you look at the backgrounds of Brendan Eich, Matz, and Larry Wall (creators of JS, Ruby, and Perl), they are quite different. None of them really worked in a research setting, and they certainly didn't develop their language in a research setting.

https://en.wikipedia.org/wiki/History_of_Python

3 hour oral history with Guido: https://www.youtube.com/watch?v=Pzkdci2HDpU&t=12s

Lex Fridman interview with Guido: https://www.youtube.com/watch?v=ghwaIiE3Nd8

Lua was developed in a research setting, funded partially by Brazilian oil companies as far as I remember, but I don't think it ever had a "scientific computing" focus. It was picked up more in games and apps due to the C embeddability and features like coroutines. The ML framework Torch was built on LuaJIT because it has math nearly as fast as C. But I think the language Lua is less suited toward linear algebra, again due to the lack of operator overloading.

Not to mention that Lua doesn't even have separate ints and floats! This is also an issue with using JavaScript for scientific computing.

Post reply on HN