[deleted]
Why Python keeps growing, explained
301–310 of 459 posts
Re: Why Python keeps growing, explained
#302Earlier quoted context omitted.
This is just a property of having libraries. If you've got the same libraries in Java then the code looks identical except that you stick to word "var" in front of lines 4 and 6 and you don't have named arguments. Python does have a great data processing ecosystem. But that isn't really a property of the language.
Creating a new venv, installing a few needed libraries that you know the names of with a simple command and writing a quick low-ceremony script that uses those libraries is frictionless in Python. Doing that repeatedly in nearly every other major language is not as easy. Some languages have a good integrated tool chain (Rust, Go) but are not as approachable and forgiving. Some are approachable but lack the friction-f…
Poetry is indeed very nice for that. Bit of a learning curve but then very easy to push packages to PyPI or internal things like Artifactory.
Re: Why Python keeps growing, explained
#303Earlier quoted context omitted.
>"how much work do I need to put in to get a very simple JSON file from a web URL" (nothing fancy like POST, just an HTTP GET). With python, a call to urllib.request.urlretrieve and then a call json.loads are all you need. In C# you just have to do this: var things = await httpClient.GetFromJsonAsync >("url");
why would an http client know anything about json? that's a bad code smell.
It doesn't. It's a standalone (static) helper method that uses HttpClient to perform the request and feeds the response body into a json parser. It's an extension method [0] which means that there's syntactic sugar so that you can write client.GetFromJsonAsync() and the compiler transforms it into the actual static method call, HttpClientJsonExtensions.GetFromJsonAsync(client).
0: https://learn.microsoft.com/en-us/dotnet/csharp/programming-...
Re: Why Python keeps growing, explained
#304One thing I’d add to this conversation, though I’m certain it’s already been stated: As many have mentioned, there is a large subset of the user base that uses Python for applied purposes in unrelated fields that couldn’t care less about more granular aspects of optimization. I work as a research assistant for international finance faculty and I would say that compared to the average Hackernews reader, I’m technologi…
>As many have mentioned, there is a large subset of the user base that uses Python for applied purposes in unrelated fields that couldn’t care less about more granular aspects of optimization.
Nobody cares about this that much. Even a straight up software developer in python doesn't care. The interpreter is so slow that most optimization tricks are irrelevant to the overall bottleneck. Really optimizing python involves the FFI and using C or C++, which is a whole different ball game.
For the average python developer (not a data scientist) most frameworks have already done this for you.
Re: Why Python keeps growing, explained
#305Earlier quoted context omitted.
That is true. However, I hope that these C applications are written by people who are really good at C. I know that some of these Python applications are written by people who discovered the language as they deployed into production.
That’s a measure of programming prowess, not the actual security concern at hand. If the masterful C developer still insists on using a language that has so many footguns and a weird culture of developers pretending that they’re more capable than they are, then their C mastery could very well’ve not been worth much against someone throwing something together in Python, which will at the very least immediately bypass…
[deleted]
Re: Why Python keeps growing, explained
#306Earlier quoted context omitted.
Agreed. Python's REPL has basically totally replaced my usage of Emacs calc as a desk calculator, mainly because it is always there and if I don't know the big-brain closed-form solution for something like compound interest, I can just write a loop and figure it out that way.
So what you are saying is that Python is Excel for programmers :D
This is a really good line, the VAST VAST majority of programming in the world is done in Excel by people who would be horrified if you told them they were programming.
And I wouldn't be surprised if a large number of python programmers would say they're not programming, it's just scripting.
Re: Why Python keeps growing, explained
#307This was already posted at https://news.ycombinator.com/item?id=35000415 , I don't know why it didn't detect the duplicate. I'll repost my comment from there: This is a strange article. It's got the talking point about Python that we were hearing about 10 years ago - "tired of those pesky curly brackets in Java, try this new language you might not have heard of: Python!". Who reading the GitHub blog has not heard of…
[dead]
Re: Why Python keeps growing, explained
#308Earlier quoted context omitted.
I also use a python repl as an alternative to excel or SQL. I find myself just downloading the data as a CSV and then quickly cooking up some pandas to get a graph or aggregate some stats, it’s just so much quick easier imo.
A bit off topic, but what would you use for data "mangling"? Like joining csvs on complex conditions, cleaning tables etc. Pandas seems to be the wrong tool for this, but I still often find myself using it as in contrast to something like Excel, my steps are at least clearly documented for future use or verification.
If you asked this question 6 or 8 years ago the answer would be it depends on the volume of data (10s of gb, 100s of gb etc.) and I could give you just a single tool that would help you in most cases.
Today honestly most tools are pretty capable, pandas is a great choice and if you have really high volumes of data you might try koalas (spark) or polars.
Honestly the biggest design considerations for data science today are things things external to your project: what do you and others on your team know, what tools does your company already have setup, what volume of data are you processing, what are your SLAs, who or what else needs to run this script/workflow, what softwares do you need to integrate with, how often does it need to be processed, how are you going to assure the quality of your data and what tools are you using for reporting?
I tend to use pandas and SQLite for most use cases cause I can cook up a script in 2 hours and be done, I just code it interactively in a notebook and most people are able to work on a pandas or SQLite script productively if it needs to be maintained even if they don't know python. If its a large volume of data or a rapid schedule (minutes, seconds) or tight SLAs on quality or processing time, then I start to consider whether pyspark, Apache beam, dask or bigquery might be a good fit.
So it really just depends but for most people who are processing < 100 GB on a 1+ day schedule or ad hoc I would recommend just using pandas or tidyverse in R and getting really good at writing those scripts fast. Today you’ll get the most mileage out of those two tools.
Re: Why Python keeps growing, explained
#309Python keeps growing in number of users because it’s easy to get started, has libraries to load basically any data, and to perform any task. It’s frequently the second best language but it’s the second best language for anything. By the time a python programmer has «graduated» to learning a second language, exponential growth has created a bunch of new python programmers, most of which don’t consider themselves progr…
I fully agree with the description. What worries me, though, is that the features that make Python quite good at prototyping make it rather bad at auditing for safety and security. And we live in a world in which production code is prototyping code, which means that Python code that should have remained a quick experiment – and more often than not, written by people who are not that good at Python or don't care about…
Python code can be production code. There are many people and companies shipping Python production code and generating substantial value.
Re: Why Python keeps growing, explained
#310Earlier quoted context omitted.
I also use a python repl as an alternative to excel or SQL. I find myself just downloading the data as a CSV and then quickly cooking up some pandas to get a graph or aggregate some stats, it’s just so much quick easier imo.
A bit off topic, but what would you use for data "mangling"? Like joining csvs on complex conditions, cleaning tables etc. Pandas seems to be the wrong tool for this, but I still often find myself using it as in contrast to something like Excel, my steps are at least clearly documented for future use or verification.
I still use perl for some of that stuff, or even awk, but those are barely reusable or readable.