Every year that passes these “SWE is an idiotic field” posts come earlier. So annoying…
Bad scientific code beats code following "best practices" (2014)
251–260 of 333 posts
Re: Bad scientific code beats code following "best practices" (2014)
#252Earlier quoted context omitted.
>They are also less productive when coding than the scientists because they care too much about the quality of their work and not enough about getting shit done. Ultimately I’d say the core issue here is that research is complex and those environments are often resource strapped relative to other environments. As such this idea of “getting shit done” takes priority over everything. To some degree it’s not that much d…
maintainable prototypes are overengineered
Is there any metrics which proves that making maintainable code is slower? Because in my experience there is no difference.
Re: Bad scientific code beats code following "best practices" (2014)
#253Not sure what the exact context the blog refers to (they are a scientist turned software engineer?, their field is data/software intensive and see this as an improvement area?). Our team does engineering test and evaluation that include aspects of R&D and struggle with this. Scientists have academic approaches (must have sufficient sampling/statistical significance, sometimes delaying findings/reports checking additional aspects when a sanity check will suffice) but it does give rigor. On the software side, we definitely have spaghetti code, tools that rely on some file sitting in someone's H-drive, and plug-ins built by someone's old collaborator with little documentation. This is juxtaposed by the PM types that must be agile (fine when tasks are understood and defined enough to go nicely into a sprint, not always the case). Better communication on both sides would probably alleviate some of this but that is the great challenge in any group.
Re: Bad scientific code beats code following "best practices" (2014)
#254> Invariably, the biggest messes are made by the minority of people who do define themselves as programmers. After 15 years of writing JavaScript professionally I know that is a lie. The biggest messes are made by the majority of people hired that cannot really program.
Want to give any examples or reasoning rather than state pure opinion? I’m not a fan of using “lie” when you believe something isn’t true. Lie implies intentional dishonesty, and there’s absolutely no reason to suspect the author doesn’t believe what they said. Their experience certainly could have involved larger messes made by programmers than scientists. Just say you think it’s not true, and why, even if lie seems…
Most people that write JS professionally cannot program, or at least cannot program in JavaScript though not programming at all is more generally true. More than 90% of people doing this work, for work, are fully reliant upon multiple artificial layers of abstraction. For example if you take away a developer's favorite framework they suddenly become hopelessly irredeemable. Even with their favorite framework if you ask most developers to write original functionality beyond merely putting text on screen, such as a common CRUD app, they are hopelessly lost.
This becomes immediately clear when you confront developers about this. Most of their answers will be irrational qualifiers which might make sense to them, but from a perspective of objectivity and product delivery its really mind blowing. In most cases the insanity stems from poor preparation followed by what then becomes unrealistic expectations.
Just as a real world experiment ask a front end developer to write to the DOM directly. The DOM is the compile target of the browser accessed via standard API which can be mastered in less than 4 hours of practice. Despite that prepare to be under impressed and dazzled by the equivocations, unfounded assumptions, red herrings, and so forth. The DOM is just an in-memory data structure with a standard API, but seems large data structures scare people.
---
All a person really needs to know to be good at this language:
* Functions are first class citizens. This means a function can be expressed or referenced any where a primitive can be used. This is incredibly expressive.
* Lexical scope is native. This means lexical scope is always universally on, not hidden behind syntax, and can never be turned off. This is also incredibly expressive.
* OOP is optional. The language never forces OOP conventions upon the developer, which is great because the concept of polyinstantiation, on which OOP is based, greatly increases complexity.
* The language is multi-callstack. This is commonly referred to the event loop, and allows executing externalizing instructions without locking the language.
* A casual understanding navigating data structures.
That being said anybody can build large, fast, robust applications in JavaScript using only functions, statements/expressions, events, and data structures. TypeScript interfaces help tremendously as well. Despite this most developers need all kinds of vanity to make sense of the most simple tasks and anything original is like asking people to crawl across the Sahara.
> Want to give any examples or reasoning rather than state pure opinion?
Its based upon 15 years of doing that work professionally for multiple employers. By far the biggest messes in this language come from the absence of confidence in the developers writing in it. I imagine scientists, non-professional programmers, writing messy software are at least passionate enough about their subject matter to do it well enough the first time so they aren't spending the rest of their existing fixing bugs, regressions, and performance traps of their own creation.
Perhaps the word lie was incorrect and something like wrong in practice would have worked better.
Re: Bad scientific code beats code following "best practices" (2014)
#255This is partly because, in my opinion, some "best practices" are superstitions. Some practice was best because of some issue with 80s era computing, but is now completely obsolete; problem has been solved in better ways or has completely disappeared thanks e.g. to better tooling or better, well, practices. e.g. Hungarian notation. Yet it is still passed down as a best practice and followed blindly because that's what…
[flagged]
Business epistemology is not about knowing Truth, it is about knowing currently useful information and practices, and it is expensive to validate or generate this knowledge.
Hence we get the same thing over and over again until someone convinces people there is a better way, or simply does something different and makes more money.
Re: Bad scientific code beats code following "best practices" (2014)
#256Earlier quoted context omitted.
maintainable prototypes are overengineered
Is there any metrics which proves that making maintainable code is slower? Because in my experience there is no difference.
I have tons of examples of code where I did the simplest thing to solve the problem. Then later needed a change. I could refactor the entire thing to add this change or just hack in the change. Refactoring the entire thing takes more work than the hack so hack it is unless I forsee this is going to matter later. Usually it doesn't
Re: Bad scientific code beats code following "best practices" (2014)
#257Earlier quoted context omitted.
This might work for CERN. But a great deal of science is done by small teams who don't have a professional programmer available. Basically all of the social sciences, for a start; a lot of genetics too.
If you cannot model using decent code is it worth writing models at all? What if bugs mean the model is simply wrong? It has consequences too. There has been a lot of argument about how much impact the poor code quality of the Imperial college covid epidemiology model (which was the basis of British government policy during the pandemic) had on its accuracy. I do not know how bad it was, but it cannot be good the cod…
How are people supposed to do science without running statistical models?
Re: Bad scientific code beats code following "best practices" (2014)
#258I'm a scientist programmer working in a field comprised by biologists and computer scientists, and what I've experienced is almost exactly the opposite of the author. I've found the problems that biologists cause are mostly: * Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months * Writing completely unreadable code, even to themselves, making it imp…
> Not understanding dependencies, public/private, SCM or versioning, making their own code uninstallable after a few months
I'm not sure what "uninstallable" code is, but why does it matter? Do scientists really need to know about dependencies when they need the same 3 libraries over and over? Pandas, numpy, Apache arrow, maybe OpenCV. Install them and keep them updated. Maybe let the IT guys worry about dependencies if it needs more complexity than that.
> Writing completely unreadable code, even to themselves, making it impossible to maintain. This means they always restart from zero, and projects grow into folders of a hundred individual scripts with no order, depending on files that no longer exists
This is actually kind of a benefit. Instead of following sunk cost and trying to address tech debt on years-old code, you can just toss a 200-liner script out of the window along with its tech debt, presumably because the research it was written for is already complete.
> Foregoing any kind of testing or quality control, making real and nasty bugs rampant.
Scientific code only needs to transform data. If it's written in a way that does that (e.g. uses the right function calls and returns a sensible data array) then it succeeded in its goal.
> They are also less productive when coding than the scientists because they care too much about the quality of their work and not enough about getting shit done.
Sooo...another argument in favor of the way scientists write code then? Isn't "getting shit done" kind of the point?
Re: Bad scientific code beats code following "best practices" (2014)
#259This is so true I don't think I ever read something so true. It's not even scientists vs software developers. It's people who are really into software development and clean code. They say the program needs a total rewrite and proceed to add 20 layers of inheritance and spreading out every function over 8 files. Ever since I make sure to repeat my mantra every week to developers: How maintainable code is is measured i…
Am I missing something? Opening files is not my most intensive work as a developer.
[deleted]
Re: Bad scientific code beats code following "best practices" (2014)
#260Earlier quoted context omitted.
This might work for CERN. But a great deal of science is done by small teams who don't have a professional programmer available. Basically all of the social sciences, for a start; a lot of genetics too.
Most teams at CERN don't have a professional programmer available either. In a few of larger projects (those with a few hundred active contributors) there might be one or two more tech savvy people who profile the code regularly and fix the memory leaks. But few (if any) are professional programmers: most contributors are graduate students with no background in programming.
And this is scary. At least with "high-energy experiments" (like the one that discover Higgs) in colliders, a lot depends on so-called triggers, which dismiss 99.9% of information produced in a collision "on the spot", so that this information is never recorded and analyzed.
They have to: there is way too much information produced. So the triggers try to identify "trivial" events and dismiss them immediately, relaying only the ones that are may be somewhat unusual/unexpected.
Essentially, the triggers are computers with highly specialized programs. Very smart people work on this, and supposedly they figure out problems with triggers before they affect the results of experiments...