The author simply has a grudge towards over-engineered code such as [FizzBuzz Enterprise Edition](https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...). Not all modern coding is like that, and you certainly cannot build complex software (at least maintainable complex software) without abstraction layers.
Is “data scientist” the new “programmer”?
161–170 of 246 posts
Re: Is “data scientist” the new “programmer”?
#162Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…
I've been hearing the same complaint between Cobol and Java for years : it was simpler before, more efficient, etc.
Of course it was, but you were tied to one system (no application server), security was login/pw, database had no constraint, typing systems were ultra limited, everybody has its own way of writing batches (no Spring), business code was mixed with tons of technical code (no JPA).
Now, sure, if you glue some R, some SQL, etc. you can extract insights worht millions of dollars. But all of that exist just because we have digitalized all of the processes, data collection, etc. And the rise of data scientists will continue only if there are more stuff put in the databases, thanks to you plain, regular, normal programmers...
Re: Is “data scientist” the new “programmer”?
#163Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…
The problem isn't the abstractions, it's the sheer size of the codebase. The author - and I think most people - prefer a codebase they can grok. Nobody can grok millions of LOC, at best they can have a high level overview of what does what.
At that point you need the abstractions and practices that make code boring.
Re: Is “data scientist” the new “programmer”?
#164The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…
A good programmer is someone who can communicate with the problem domain experts and provide an a solution to fit their problem. Someone who understands the limitations of the computing environment and can engineer a solution that is adequate for the problem space problem.
Many who consider themselves programmers produce solution space solutions that just don't get to the core of the problem space problems. This is a function of the simple fact that many programmers never have the opportunity to see what the problem space experts are doing or actually need. This is a real shortcoming in the education of programmers.
We don't have to be subject matter experts in all fields, we just need to become competent in being able to understand the kinds of problems that are being faced by the various subject matter experts that we build systems for.
On the other side of that coin are those who are subject matter experts who think it is easy enough to become competent programmers. What they miss is the essential problem that programming is, itself, a field that requires a subject matter expert. I have come across too many systems that have been developed by the subject matter experts that were just wrong. Wrong in design, wrong in understanding the limitations of the tools being used, wrong in oh so many ways.
To build properly functional and functioning systems requires the cooperation, input and continual communication between those who are subject matter experts facing problem space problems and those who are subject matter experts in computing systems. This is a rare event and so we see the problems in every field with the computing systems that currently exist.
Re: Is “data scientist” the new “programmer”?
#165This is such a bizarre post. The reason why people use a language like R is because it is easy to learn and use (and install, via RStudio) for data analysis without having to be a well-trained programmer. I can’t recall ever hearing from anyone who has relied on R, doing so because it was computationally efficient. The point of the language is convenience — particularly with how easy it is to create attractive graphi…
The fact that R has such buy-in despite being a rather awful programming language (a friend of mine worked on the next Lisp-like version of R under Ross Ihaka, and the next version is based on the fact that current R is a bit awful) is precisely because it offers such convenience to non-programmers. In my sister company, they have data scientists, and data engineers. The data scientists write their algorithms in the…
a friend of mine worked on the next Lisp-like version of R under Ross Ihaka
Did anything come of this? I seem to remember seeing a paper or article where he proposed doing this, but I’ve never seen an implementation.
Edit: paper I was thinking of is https://www.stat.auckland.ac.nz/%7Eihaka/downloads/Compstat-...
Re: Is “data scientist” the new “programmer”?
#166The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…
Did you ever the pleasure to work with a Six Sigma Black Belt who had the most of one week statistics training? I am one, but honestly this is just enough to do some back-of napkin number crunching for purely operational purposes. That you need a math PhD is maybe exaggerating as well on the end of the spectrum.
That being said, the abilit to talk to domain experts and accept their experience is one of the most important skills for a true data scientist. Without proper context all the data in the world gets you nowhere.
Re: Is “data scientist” the new “programmer”?
#167One thing I don't like about my source code is it "doing something interesting".
I like my code to be boring. I like my frameworks to be boring. I like my APIs to be boring. So I can focus on important things in life (or even the important things at work), and be done with it.
Re: Is “data scientist” the new “programmer”?
#168Earlier quoted context omitted.
I sometimes wonder if programmers instinctively overcomplicate things in interest of collective job security. Some of the stuff I've seen in (particularly awful) Java code bases is perplexing to the point where it seems intentional.
It's more likely the natural entropy of code - it's easy to add stuff to a system in a way that makes it more messy; and if the system already is a big mess, then it's much harder to do non-messy additions and the bigger mess it is, the harder it is to start cleaning it up.
True that. Adding complexity is easy, keeping things at the right level of simplicity is hard work and requires skill. Not just for coding.
Re: Is “data scientist” the new “programmer”?
#169Something about this article strikes me as a thinly-veiled complaint about poorly designed object-oriented systems. Take, for example this comment by the author: >Even if the money were half of what today’s coder gets paid it might still be a better job because one is spared the tedium of looking at millions of lines of Java that do almost nothing! What all those millions of lines of code are is abstractions, decoupl…
Personally, I am increasingly convinced that a lot of this hate comes from programmers with weak abstract thinking who simply cant do it. Instead of admitting that there is learning cure involved, they will claim the system is bad and everyone else is bad. Compounding factor is difficulty dealing with system that was written by different people who holded different opinions. Yes, there are badly designed large system…
Badly designed systems aside, for most systems that currently exist or have existed in the past, there is little or no documentation that is worthy of existing. Most comments in code and the associated documentation in manuals fails to provide the reasoning as to why the code exists, why it is written that way, what underlying assumptions have been made, etc, etc, etc.
I am going through a process at the moment of documenting all of my local codebase. It will, in turn, be turned into a literate programming base. The problem I am finding is understanding all of the assumptions that underlay the original code. Why was it written this way or that, what is it trying to do, is the code actually doing what it is supposed to do?
There are, at present, some questions that I am having difficulty answering and this is my code. How much more difficult is it for someone to come in and look at a historical piece of code and follow what the original authors and designers were trying to achieve and what were the changes that have been made over time trying to achieve.
Documentation at the level we need to be able to adequately maintain any code base is just not done - it is very hard to do and to do so in a way that will help future people manage and maintain that codebase. On of my projects involves restructuring the code base. However, I need to understand the history of that codebase and that means talking with those people who are still living who knew the original authors and give an oral insight as to why things have been done the way they are. This oral history has to be written down and the codebase documented with it. When that information is in place, why the code is written the way and how we can now rewrite that code to be more effective is now achievable.
If we then put on top of missing history and documentation all of the bad designs, well, we are then facing even bigger problems. Then we put on top of that all the egos and politics involved, we get an even bigger mess.
So just reading up more doesn't actually help, because that which is needed was never written in the first place.
Re: Is “data scientist” the new “programmer”?
#170The take down on abstraction and software engineers (by using Java as an example) is similar to saying "back in the day to find a prime number we would simply use a sieve, but today it is a tedium, what with all the pi's and e's and thetas that get in the way, and what are geometry and polynomials doing here, and what in the god's name is this i , I just want to count the prime numbers which are nice round whole numb…
I'm a scientist (wet lab) by training, a programmer (back end) by profession, and a data scientist by hobby (I have a machine learning project that I'm working on), and most of "data science" is not really stats... There will be a bit of stats at the end product but really the bulk of the necessary work is data curation. Annoying stuff like making sure my data fit into the right buckets. I did have to debug a memory…
N == 22 specifically? Or N >= 22? Interesting threshold value