Live data from Hacker News

Every modeler is supposed to be a great Python programmer

statmodeling.stat.columbia.edu

121–130 of 131 posts

Re: Every modeler is supposed to be a great Python programmer

#121
post #11

This resonates so strongly with me. I'm a modeler and best described as a lifelong scientific programmer. I'm much, much better at doing the specialty science I was trained to do and have done than write unit tests (for Jupyter notebooks? why?) or struggle through big-O questions (again, why? I write ~100 line programs that are never production code). Like the author of the post, I am not a professional developer and…

I might be reading too much into your story, but if your CV indicates that you're "much better at doing the specialty science I was trained to do and have done than" CS stuff, a good interviewer might be concerned whether you had the necessary CS knowledge / programming skills.

To me at least, an interview is not to flatter the candidate by asking questions they're most comfortable with, but to make sure they have the requisite skills expected for the job. If the job expects X, Y and Z, and they seem like they're good at X and Y, you spend the time checking whether they're reasonably competent at Z as well.

Re: Every modeler is supposed to be a great Python programmer

#122
post #100

I think nobody here really pointed out a very relevant issue that's completely widespread at least with tech job market: companies don't want to pay people. You don't see this kind of problem in other established professions, you don't expect an accountant to be able to perform the job of a lawyer neither you expect a nurse to be able wear the hat of a nutritionist. Now with the technological professions, let's use t…

> you don't expect an accountant to be able to perform the job of a lawyer neither you expect a nurse to be able wear the hat of a nutritionist.

You'd be surprised. I mean, they're not competent to do so professionally, so they can't formally give professional advice outside of their field of competence, but you can be sure people do ask them such questions, and get disappointed if the answer is "that's not my job".

Re: Every modeler is supposed to be a great Python programmer

#123
post #121
post #11

This resonates so strongly with me. I'm a modeler and best described as a lifelong scientific programmer. I'm much, much better at doing the specialty science I was trained to do and have done than write unit tests (for Jupyter notebooks? why?) or struggle through big-O questions (again, why? I write ~100 line programs that are never production code). Like the author of the post, I am not a professional developer and…

I might be reading too much into your story, but if your CV indicates that you're "much better at doing the specialty science I was trained to do and have done than" CS stuff, a good interviewer might be concerned whether you had the necessary CS knowledge / programming skills. To me at least, an interview is not to flatter the candidate by asking questions they're most comfortable with, but to make sure they have th…

My CV is 7 pages long.

In my case, the ~30 years of experience I've had as a software developer, data scientist, assistant professor, and quantitative researcher - not to mention my refereed publications in several computation-heavy areas - should be proof enough that I possess the minimal amount of programming skills for the job.

Either that or I've been pulling the wool over everyone's eyes for 3 decades.

In my present and past experience, the asymmetry of what is required for a computational scientist (the science, the programming, the CS whiteboard stuff) vs. software engineer (software engineering) is quite pronounced. YMMV.

Re: Every modeler is supposed to be a great Python programmer

#124
post #109
post #73

Earlier quoted context omitted.

> often needs to be optimized for scale or refactored for reusability It's so easy to make subtle assumptions when you redo their code that completely invalidates the work they've done. ML completely collapses on extremely small errors. Handing something off to someone else to refactor is a dangerous step in the process that risks everyone wasting their time

> It's so easy to make subtle assumptions when you redo their code that completely invalidates the work they've done. It's equally easy to test whether those assumptions actually break anything. Especially when minor errors can be catastrophic. I'm part of a team that was tasked with producing a web app from something that was originally a piece of Matlab code. We considered just running the Matlab code in a containe…

The catastrophic errors aren't the problem, the problem is when things are degraded by 20% which results in a small loss when it could have been a big win. You just assume it didn't work when in reality it wasn't implemented correctly

Re: Every modeler is supposed to be a great Python programmer

#125

Earlier quoted context omitted.

I dunno... I am a software/data engineer who partners with data scientists. I think that comparative advantage here is a real and important. Don't get me wrong, I'm happy when my data scientist partners write good code or show interest in getting better, but I'm more than happy to take their janky code and make it production ready. It often needs to be optimized for scale or refactored for reusability, and a lot of t…

Why do you want them to write code at all, then? Why not just task them with writing user stories around the parameterized functions they need, and just let you figure out how to implement it all?

The idea that creating a model can be a “user story written in math” seems to me a variation on a common misunderstanding about what model creation, particularly the role of coding in it is. Data scientists, statisticians, modellers, don’t go in knowing what the model is and just coding or specifying it. They use code as a exploratory tool to test several hypotheses until they find one that seems to hold - the code, the data, the algorithm, the statistical test, the exploration and the reasoning are all tools in the process of discovering the right model. It simply can’t be specified in advance, it has to be run and tested.

Having som spaghetti code is a natural consequence of this exploratory, iterative process. Applying good SW engineering practices to this exploratory endeavour is just a natural consequence of the process when you don’t know if your code will be of any use before you finish running and checking the test results. Why would you bother modularising and doing test coverage on something that is very likely to be thrown away after one or 2 runs?

I say this from 28 years of experience both as a data engineer and data scientist. I am a good python developer, and I can write production grade code. But I won’t refactor my code into that until I know that is the code that generates the right model. And I certainly can’t specify this particular code before writing some dirty version of it, testing it and confirming that the model it trains passes some statistical tests, at least.

Basically, the code is not the product - the product is the result of applying some transformations on data and running that through some ML or statistical algorithm to generate a model. Transformations and algorithm being unknown to be useful until tested, hence specification being unknown until coded, run and tested.

Re: Every modeler is supposed to be a great Python programmer

#126

This was something that surprised me after I did my PhD as well. I thought that employers would focus on my specialized skills and "someone else" would somehow pick up the pieces and make something out of what I did. Turns out this is completely wrong, and I now see how frustrating it is to work with people that have this kind of attitude. Most of most jobs is a bunch of mundane stuff. I've seen it in software develo…

Having done both sides of the work, I totally understand the author’s perspective. Modelling and coding production are certainly two different skillsets, and many people will prefer one or the other, but not both. Now, there is one thing I guess the author doesn’t get (or he gets but doesn’t like) and a thing that most companies don’t get.

The former is that companies don’t care enough about what the individuals they hire prefer - especially not if it doesn’t address their need. They don’t need a beautifully crafted model that can’t be run - they want actionable results that hit the bottom line, and in the energy forecasting model that means generating new forecasts at every cycle (months, days, hours, minutes - whatever). A great model that can’t be put in production and run efficiently has about the same value as no model, and an average model in production will have much more impact.

The latter is that companies don’t usually understand that ML is not software development. Putting ML in production is, but finding the right model is research work, and code is mostly a discardable tool for research. Its goal is not to go live, it is to validate a hypothesis (in this case, that algorithm X, when presented with data Y, generates a model with enough predicting power to be useful to the business). This validation requires code to gather Y, clean/join/analyze/reshape/featurize it into a more informative and clean representation (clean from the algorithm’s perspective, not necessarily a human’s), run X, run inference with the generated model and run some test of the results against additional data.

If this test is negative, some or all the code written above is useless, and we go back to the drawing board. Given the very coupled nature of this process (a new data source has to be joined to the rest - coupling; a new data transformation changing a feature changes the data schema downstream - coupling; a new algorithm needs a different data input - coupling, and so it goes). If you have an experience like mine, you may actually be able to write in a way where you can reuse some of it, but I have 28 years of experience with data, there are simply not enough people in the market with that level of background or the interest in learning all this. Companies must accept that they will not always get this perfect candidate with all the skills they want, and start thinking of pairing the right people in teams.

Some who have been around for a while may remember the Venn diagram of the perfect Data Scientists - it usually was an intersection of business, math and programming skills (also often communication skills and a few others). My thoughts since I first saw this diagram were: “Even if there are people out there with all these skills, why would they want to work for others?”

This, more than anything else, is my guess at the core reason why so few companies are successful in putting ML in production.

Re: Every modeler is supposed to be a great Python programmer

#127

Earlier quoted context omitted.

Julia’s package management is miles ahead of R, and bests Python’s on a large number of factors. However the last time I used it, it still required you to issue commands into the repl to setup your packages. However, that might have been resolved by now? The language and tooling clips along at a pretty good pace.

I find the flexibility of being able to setup and switch environments from within the REPL very enabling, and the Python approach of using the command-line shell feels kinda messy and cobbled together to me. (I felt this way about the Python approach before I even came across Julia, by the way.) That said, there is a command-line utility jlpkg [1] that makes package management available from the shell. It's not very…

I’m well aware of pkg mode in the repl, wasn’t aware of jlpkg though, that seems good.

Re: Every modeler is supposed to be a great Python programmer

#128
post #96

Earlier quoted context omitted.

>the serialization format is derived from the implementation and not very stable, so if the implementation changes - sometimes in subtle ways like upgrading your Python version - the serialized data isn't readable anymore. Could be wrong, but I do not believe this is true anymore. There are pickle versions, but I believe the format has been standardized such that you should be able to freely move between Python relea…

The base serialization is stable and compatible across Python versions. However pickle files can serialize entire Python objects, including from custom and third party code. Compatibility of serialization then depends on the versions of the code that implements those objects, which is often not well managed.

Ah that's fair. I was only considering serializing native types (for which there are much better options).

Re: Every modeler is supposed to be a great Python programmer

#129

Earlier quoted context omitted.

>Tried putting R into production recently? It’s a frustrating and brittle experience. Oh man, can I sign onto this rant. While Python has spent a decade+ trying-and-failing to standardize on one of a dozen tools to properly manage dependencies, at least it is trying. R is still global-namespace, no-pinning by default. Sorta-kinda you can squint where renv is going, but still needs a lot of development. I had some hop…

Just use renv (successor of packrat). I had multiple versions of R working with multiple versions of Shiny running in production.

As I alluded to, renv exists, but it requires a lot of development work before it is a comparably robust option for the ecosystem. Basic things like a command-line interface [0], working with non-CRAN repos [1], using an existing DESCRIPTION file [2], etc. There are many use cases where renv does not work in a corporate environment (ie not open-source all public code scenarios). Some of those issues have been open for years.

I do not believe the situation is unsolvable, but there is significant work to be done. Renv provides value today, and I will encourage everyone to use it. However, it has significant blind spots which continue to make R deployments challenging.

[0] https://github.com/rstudio/renv/issues/1055 [1] https://github.com/rstudio/renv/issues/471 [2] https://github.com/rstudio/renv/issues/233

Re: Every modeler is supposed to be a great Python programmer

#130
post #93

> every one of them wants Python. I haven’t seen a single one where they’re looking for R or even C++; Python rules this roost. Tried putting R into production recently? It’s a frustrating and brittle experience. Don’t get me wrong, R is fantastic at what it does - analysis, research, statistics, and arguably the API’s on the R data frame packages are a lot saner than Pandas. C++ is out for different reasons I suspec…

> wish there was some more variety Kotlin is coming up on the outside. Late to the party, but the data science ecosystem is progressing nicely.

This is an interesting observation. Some of the data scientists I work with have been using Kotlin to define "analytic grammars", and is the first example (in my limited Kotlin awareness) i've seen of Kotlin outside of Android development.

It seems that Roman Elizarov (Kotlin Project Lead) has identified the opportunity for a better language ecosystem to enter the data science space

https://discuss.kotlinlang.org/t/ai-and-deep-learning-define...

Post reply on HN