Live data from Hacker News

Fighting Complexity in Software Development

github.com

91–100 of 117 posts

Re: Fighting Complexity in Software Development

#91

Is it just me? I feel this is confused and adding complexity in some areas, not genuinely optimal for simplicity. I tend to feel that unexpected, unrecoverable exceptions are best treated as just that -- exceptions. Applying a C-style function return value check seems backwards. And the "Interpreter"?? Proper purpose of Interpreter or DSL is for dynamic (configurable or user-input) code, not to implement basic sequen…

Seems like a pretty classic case of second system syndrome to me.

Re: Fighting Complexity in Software Development

#92
post #85

The main source of complexity is requirements. Gatekeeping against the influx of requirements will keep complexity down. Then there is unnecessary complexity from doing incomplete refactorings and rewrites. If some code cannot handle the addition of a new requirement, it should be replaced. Otherwise you add complexity that roughly takes the logical (if not actual) form if (these cases) { new code } else { old code }…

It's easy for us to blame "requirements" as the main source of complexity. This isn't accurate. Software exists to serve the needs of the business. Depending on the maturity and stage of the business or the software itself, it's possible that there's a changing set of requirements. As developers, it's out job to figure out how to deliver, not to say "no" to new enhancements and requirements. The main source of comple…

I think it's fair to say that changing requirements coupled with a limited amount of time results in non-incidental complexity.

It's a systemic problem that results when the entire leadership stack isn't aware of how good software is created. Because of the limited amount of time and resources given, quite often it's a business/management problem.

And that's not to say that it isn't also a software dev problem. We've all seen some horrific things. But I've also seen horrific things because there was no one senior there because they wouldn't pay enough for it.

it's all intertwined, there's not a simple explanation. But changing requirements is definitely a source of complexity.

Re: Fighting Complexity in Software Development

#93
I agree that OOP is limiting, but Functional is not the fix. Table-Oriented-Programming is the future. Your code snippets for validation etc. could be associated how your domain prefers, and you can query them to be together by field or by any other grouping as needed. You just have to make sure you have the proper meta-data in place, such as field name/ID, entity, screen, event type, etc.

File-centric code forces a hierarchical big picture structure, but many relationships are either not hierarchical, or need additional non-hierarchical ways to view/group them. Relational is more powerful and more flexible than file systems. (It has some rough areas, but they can be worked around or fixed.)

Start backward next time and think how you would LIKE your code organized, forgetting about frameworks you know. If you do this often enough, you'll realize RDBMS-based code management is where we should be heading. About 90% of validation and field management could also be attribute-driven: data-dictionaries would do most of the grunt work.

With OOP and FP you are forced into choices such as "should this be its own class, or an object instance, or a group of classes for composition?" etc. etc. When tablizing your event & validation snippets, you are not forced to choose. They are grouped "by" anything you want, and by multiple groupings at the same time: multiverse. I agree that FP is probably more flexible than OOP, but it's also less disciplined: large FP systems look like the spaghetti-pointer databases that preceded RDBMS. Hierarchical, logical, and pointer-based DB's thrived for a while, but relational won, for a reason.

Re: Fighting Complexity in Software Development

#94
post #43

Earlier quoted context omitted.

"Started from Scratch". I have never seen any project where something has been started from scratch actually turn out well.

Now read that again. Every single piece software that exists, successful or otherwise, was at some point started from scratch.

In the context I was replying to it should have been obvious that I was talking about rebuilding ab existing project from scratch.

Re: Fighting Complexity in Software Development

#95
In my experience, one of the best ways to fight complexity is by reducing the amount of things you have to keep in your head. In practical terms this translates into being able to do local reasoning about your code.

I find that imperative OO style naturally leads to complexity. Passing references to mutable data all over the place creates tight coupling across your entire application. This makes it impossible to guarantee that any change you make is local without considering every other place that references the data. Meanwhile, objects are opaque state machines and programs are structured by creating many interdependent objects.

These aspects make it pretty much impossible to tell what any particular piece of code is doing just by reading it in large applications. The only option is to fire up the debugger, get the app in a particular state and look at the data. However, there are typically many ways to get into any particular state, and it's really hard to know that you've accounted for them all. So, a debugger is a heuristic at best.

FP and immutability tackle both of these problems head on. Immutable data directly leads to the ability to do local reasoning about your code, and allows you to write pure functions that can be reasoned about independently. Meanwhile, data is not being abstracted inside opaque state machines that provide ad hoc DSLs as their API. Instead, it's explicitly passed through function pipelines to transform it.

Re: Fighting Complexity in Software Development

#96
post #93

I agree that OOP is limiting, but Functional is not the fix. Table-Oriented-Programming is the future. Your code snippets for validation etc. could be associated how your domain prefers, and you can query them to be together by field or by any other grouping as needed. You just have to make sure you have the proper meta-data in place, such as field name/ID, entity, screen, event type, etc. File-centric code forces a…

Spreadsheet-like dataflow programming actually works quite naturally in FP. See re-frame and javelin as examples:

https://github.com/Day8/re-frame

https://github.com/hoplon/javelin

Re: Fighting Complexity in Software Development

#97
post #95

In my experience, one of the best ways to fight complexity is by reducing the amount of things you have to keep in your head. In practical terms this translates into being able to do local reasoning about your code. I find that imperative OO style naturally leads to complexity. Passing references to mutable data all over the place creates tight coupling across your entire application. This makes it impossible to guar…

Reasoning locally, means maintenance on FP software should be easy.

So beginners could solve simple bugs and add simple features to FP applications.

Yet it's not a common thing.

Why ?

Re: Fighting Complexity in Software Development

#98
post #85

The main source of complexity is requirements. Gatekeeping against the influx of requirements will keep complexity down. Then there is unnecessary complexity from doing incomplete refactorings and rewrites. If some code cannot handle the addition of a new requirement, it should be replaced. Otherwise you add complexity that roughly takes the logical (if not actual) form if (these cases) { new code } else { old code }…

It's easy for us to blame "requirements" as the main source of complexity. This isn't accurate. Software exists to serve the needs of the business. Depending on the maturity and stage of the business or the software itself, it's possible that there's a changing set of requirements. As developers, it's out job to figure out how to deliver, not to say "no" to new enhancements and requirements. The main source of comple…

One source of complexity is accretion. We have certain requirements for our application. Some of those requirements, we don't implement ourselves; we need libraries and frameworks. For instance, we don't make a GUI toolkit from scratch because we need a GUI. But those third-party components are built to requirements of their own. Those requirements outnumber ours. Many of them aren't required for our use cases (like everything that is implemented in any API function we don't use, directly or indirectly). Many are. A simple requirement like "provide a screen where the user can edit their profile settings" translates to numerous detailed requirements, down to how the pixels are pushed to the frame buffer.

Re: Fighting Complexity in Software Development

#99

Earlier quoted context omitted.

>The problem is that what you throw together is invariably the base for the real thing. This isn't invariable at all. More often what you throw together gets thrown away. I'd estimate this happens to more (working) code I've written over my career than not. The hardest thing to get right is often getting the contours of a tool right and ascertaining what it should do - not making it work right after it has proven its…

> If F# or rust or haskell really was quicker and more effective in a prototyping environment as well as when writing production hardened code, programmers would likely eventually converge on only using them. That isn't what is happening. This does not seem to me to be a safe assumption to make. The market is irrational. There is no reason to believe popularity has any significant correlation to the effectiveness of…

> The market is irrational.

To some degree. But programmers are not, in general, easily herded, even by other programmers.

> There is no reason to believe popularity has any significant correlation to the effectiveness of a tool.

Programmers are not stupid. They are not totally ineffective at pushing management to use sane tools. One or both of those would have to be true for your statement to be true.

Well, you may say, it's a slow process. But Haskell has been out there for more than 30 years. F# is 14 years old, and supported by Microsoft. These languages have had plenty of time for programmers to notice that they were actually more effective in the real world.

Re: Fighting Complexity in Software Development

#100
post #96
post #93

I agree that OOP is limiting, but Functional is not the fix. Table-Oriented-Programming is the future. Your code snippets for validation etc. could be associated how your domain prefers, and you can query them to be together by field or by any other grouping as needed. You just have to make sure you have the proper meta-data in place, such as field name/ID, entity, screen, event type, etc. File-centric code forces a…

Spreadsheet-like dataflow programming actually works quite naturally in FP. See re-frame and javelin as examples: https://github.com/Day8/re-frame https://github.com/hoplon/javelin

I didn't see any actual spreadsheets at those links. But my point was that paradigm matters less if you use table-driven designs (TDD). With TDD you are dealing less with the issues of attaching & managing behavior to/with structures (which the article seemed to emphasize), focusing mostly on specific business logic and exceptions to rules (oddities). The majority of your app would work without writing a snippet of code (except maybe regular expressions for validation & formatting.)

Your actual code would be "dumber" and event-specific such that paradigm differences matter less. Complex associations are managed via the RDBMS so that code rarely has to manage them.

I should make a distinction between framework coding and application coding. I won't say which paradigm is "better" for the first; I'm mostly focusing on the application-side coding here.

Post reply on HN