Why internal SW is almost invariably sh

Re: Why internal SW is almost invariably sh_t?

#11

Post by nivertech » Thu, Jan 20, 2022, 1:43 PM UTC

codingdave wrote: post #9

nivertech wrote: post #4

There are so many reasons for this. Here are some of them that I can write without much thinking or checking my notes. 1. Enterprise problem domains are usually much more complex than the ones which are targeted by the OSS. 2. Enterprise problem domains are full of unclear, wrong and even contradicting requirements. 3. Enterprises usually have lots of legacy code/systems. 4. Enterprises usually have lots of glue code…

This list seems to carry a heap of bias with it. Enterprise cares about what works for them, and quick-and-dirty is just fine. Startups care about what works for a broader customer base and need higher quality. Different needs, and different styles. Neither approach is wrong... but they serve different purposes and will appear like cruddy work if you put an enterprise approach into the startup world. And vice versa.…

> The one point which seems completely wrong is #9 - I've never seen agile-hamster-wheel-rituals bog down an enterprise shop. I've seen the other extreme, where in an enterprise, platforms are dealt out to the devs like a deck of cards, and as long as you keep your platforms running and working for the business, your manager barely even checks in with you. I'd easily go days, often weeks, and sometimes even months without any discussions at all with my boss.

I've seen it all. At one place it went from Cowboy Coding where you were left to your own devices for months, straight to SAFe/Scrum/standups;) Both are equally bad and no match to RUP, informal mini-waterfall (think->design->build(iterate)->ship) or Shape Up method.

Re: Why internal SW is almost invariably sh_t?

#12

Post by heartbeats » Thu, Jan 20, 2022, 1:50 PM UTC

It's intentionally built to be crap. If your requirements are constantly changing, you want to be able to turn on a dime. If you have "nicely architected" software, that's going to be a lot of work. If the "software" is just a folder of Python files and some shell scripts, adding in some hack to meet some business need is very quick work.

In mathematical terms, the ratio of A - the amount of hours spent writing the software to B - the amount of hours spent running the software is going to be very high. It doesn't make sense to spend two hours to fix a bug, if you can just spend ten seconds applying a workaround, unless you'll run it more than 720 times within its intended design lifespan.

Re: Why internal SW is almost invariably sh_t?

#13

Post by codingdave » Thu, Jan 20, 2022, 1:55 PM UTC

nivertech wrote: post #11

codingdave wrote: post #9

Earlier quoted context omitted.

This list seems to carry a heap of bias with it. Enterprise cares about what works for them, and quick-and-dirty is just fine. Startups care about what works for a broader customer base and need higher quality. Different needs, and different styles. Neither approach is wrong... but they serve different purposes and will appear like cruddy work if you put an enterprise approach into the startup world. And vice versa.…

> The one point which seems completely wrong is #9 - I've never seen agile-hamster-wheel-rituals bog down an enterprise shop. I've seen the other extreme, where in an enterprise, platforms are dealt out to the devs like a deck of cards, and as long as you keep your platforms running and working for the business, your manager barely even checks in with you. I'd easily go days, often weeks, and sometimes even months wi…

But how are you defining "equally bad"?

Lets ask what it means for tech work to be "bad". If I kept the business working, the tech evolving, while saving the business money and meeting every need it asked, including migration to new tech as the systems aged... which part is bad?

Re: Why internal SW is almost invariably sh_t?

#14

Post by captaincaveman » Thu, Jan 20, 2022, 1:57 PM UTC

dusted wrote: post #8

You answered your own question:" The internal messes are always being built by people on a steady payroll." The internal software is a direct money sink to a company, it is _VERY_ visible to the organization that they're pouring money into a product that's only tangential to making money. So internal software is allowed to be developed until the point where it becomes useful, but not more. You can argue how wise it i…

Agile or not software projects are normally .. projects, with an expected budget. Much of that budget gets 'used' figuring out what the point of the project is, requirements, managing the project deliver, stakeholder management and reporting, working with other teams to access, integrate and understand systems ... then the building of the system is built with what left, which leaves no room for your 'gold plating' ...

Re: Why internal SW is almost invariably sh_t?

#15

Post by Trex_Egg » Thu, Jan 20, 2022, 2:05 PM UTC

dusted wrote: post #8

You answered your own question:" The internal messes are always being built by people on a steady payroll." The internal software is a direct money sink to a company, it is _VERY_ visible to the organization that they're pouring money into a product that's only tangential to making money. So internal software is allowed to be developed until the point where it becomes useful, but not more. You can argue how wise it i…

Also the means of the many internal tools that I've been using are mostly short sighted and built with minimal expenses.

Re: Why internal SW is almost invariably sh_t?

#16

Post by pydry » Thu, Jan 20, 2022, 2:10 PM UTC

mattbrewsbytes wrote: post #6

I'm going to use an analogy from my hobby: because internal software is like shop furniture in a woodworking shop. Those things are mostly used in the production of other things that make the company revenue. It does its job, has room for many improvements, but isn't something you'd go out and sell as a product. It might be built from scraps, using crappy lumber and pieced together with whatever fasteners were around…

There are other reasons too:

* Time pressure - OSS is rarely under deadline pressure so the temptation to half ass it is mitigated.

* Survivorship bias - there's actually a lot of OSS shit out there too but it is much easier to ignore coz nobody is gonna force you to use it.

Re: Why internal SW is almost invariably sh_t?

#17

Post by nivertech » Thu, Jan 20, 2022, 2:12 PM UTC

codingdave wrote: post #13

nivertech wrote: post #11

Earlier quoted context omitted.

> The one point which seems completely wrong is #9 - I've never seen agile-hamster-wheel-rituals bog down an enterprise shop. I've seen the other extreme, where in an enterprise, platforms are dealt out to the devs like a deck of cards, and as long as you keep your platforms running and working for the business, your manager barely even checks in with you. I'd easily go days, often weeks, and sometimes even months wi…

But how are you defining "equally bad"? Lets ask what it means for tech work to be "bad". If I kept the business working, the tech evolving, while saving the business money and meeting every need it asked, including migration to new tech as the systems aged... which part is bad?

1. Cowboy Coding - means no knowledge transfer, bus factor of 1 - bad for business.

2. Agile Manifesto says "working software over comprehensive documentation", which also leads to information silos and no knowledge transfer, and thus bad for business.

Did you ever inherited an enterprise project without any documentation, written in an esoteric language on a proprietary platform, with one-letter identifiers and without comments, and then the new management asking you to reverse engineer it and to explain to them what it does?

3. "Agile"/SAFe/Scrum are incentivizing short-term over long-term thinking. It's like picking up small-value coins on the street in front of a steamroller: you make small profits until you get wiped out. The Demo meetings are like Social Media dopamine injections - you feel good, but it's impossible to build something meaningful in a 1-3 weeks sprint.

Re: Why internal SW is almost invariably sh_t?

#18

Post by gorjusborg » Thu, Jan 20, 2022, 2:22 PM UTC

heartbeats wrote: post #12

It's intentionally built to be crap. If your requirements are constantly changing, you want to be able to turn on a dime. If you have "nicely architected" software, that's going to be a lot of work. If the "software" is just a folder of Python files and some shell scripts, adding in some hack to meet some business need is very quick work. In mathematical terms, the ratio of A - the amount of hours spent writing the s…

> the amount of hours spent writing the software to B - the amount of hours spent running the software is going to be very low

I agree on this point, the amount of time writing software is almost always going to be dwarfed by the maintenance period for software that makes it to production.

I've also seen businesses pay a lot of money software maintenance, but you seem to be implying that under-investment is a conscious choice. I would suggest that's not the main driver for software chaos.

The most common reason I've seen good software turn into an unmaintainable mess is knowledge loss / employee turnover over time. This usually results in a culture of fear around making changes to something that is poorly understood. When changes are made, they are done with incomplete understanding, and are consequently not in line with the original design choices.

Over time, even the maintainers move on. This continues until the whole thing is a complicated mess that nobody can understand, even the original author. That's when The Rewrite is suggested.

Re: Why internal SW is almost invariably sh_t?

#19

Post by TheRealDunkirk » Thu, Jan 20, 2022, 2:30 PM UTC

There are a lot of good points here, but I think the most important reason is being overlooked: not grokking the needs of the users. In my Fortune 150, all of the internal software is developed by teams of outsourced developers. Whether it's over the wall, or halfway around the world, the effect is the same: the developers don't truly understand how the users will use the software. So it winds up being crap, regardless of how carefully some middle manager -- who also will not use the software -- writes the specs.

I've made a career out of being a mechanical engineer who writes software for other engineers, while being embedded in the group, and not spirited away in some dusty development group in another building or another continent. As an engineer, I understand precisely how the other engineers will be trying to use my software, and it makes for a better overall product. Of course, I'm biased, but I think my software has had much more success than competing projects, because of this. If I had the time, I could name at least 2 large, direct examples where this has proven true, and several more smaller ones.

TL;DR: Programmers need domain-specific understanding of the problem they are trying to solve with the software.

Re: Why internal SW is almost invariably sh_t?

#20

Post by heartbeats » Thu, Jan 20, 2022, 2:36 PM UTC

gorjusborg wrote: post #18

heartbeats wrote: post #12

It's intentionally built to be crap. If your requirements are constantly changing, you want to be able to turn on a dime. If you have "nicely architected" software, that's going to be a lot of work. If the "software" is just a folder of Python files and some shell scripts, adding in some hack to meet some business need is very quick work. In mathematical terms, the ratio of A - the amount of hours spent writing the s…

> the amount of hours spent writing the software to B - the amount of hours spent running the software is going to be very low I agree on this point, the amount of time writing software is almost always going to be dwarfed by the maintenance period for software that makes it to production. I've also seen businesses pay a lot of money software maintenance, but you seem to be implying that under-investment is a conscio…

> the amount of time writing software is almost always going to be dwarfed by the maintenance period for software that makes it to production.

Oh sorry, I meant it the other way around!

So, I mean, I've seen this in scripts that I wrote to "quickly do some stuff".

1. Write the easiest possible implementation you can think of, just string together code in the REPL until it works, then put it all into a .py file

2. Some new need arises. Make the easiest possible change that causes it to work, like adding a new switch that completely changes the control flow.

3. Some new bug arises. Make the quickest possible change that causes it to work, like running the entire program in a while-true loop in case it crashes.

4. Continue along this manner until half of the code is commented-out garbage.

5. Rewrite.

There's no need to make it a thing of beauty, because it's easier to just scrap it and rewrite it. For the short term, making a "quick fix" is nearly always better, and for the long term, running a buggy piece of shit might still be preferable to spending good money to rewrite it.

(Also, the good stuff often already exists as open-source software, so it's only the hyper-specific stuff that's proprietary, and that will be closely tied to business needs.)