Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How’d you go from a noodler to writing large well structured programs?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 77 posts

Ask HN: How’d you go from a noodler to writing large well structured programs?

#1

Post by hodder »

I do a lot of coding for work (Quant Analysis) mostly in Matlab, but also VBA, and occasionally python. My team has come to write something like 15k lines of code for our overnight process.

How do you learn to structure a large program? We basically have one Matlab file that calls tons of other routines in different files in a straight order. All are in the same folder. The whole thing is out of hand. We also do not have any form of source control or testing. Wen things break, we fix them on the fly.

Do you have a good book recommendation on project structuring?

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#2

Post by sshrinivasan »

I suggest you hire a software developer, preferably a senior one if you can afford it. The difference between a bunch of impressive algorithms written by specialists in the field stitched together by brittle connections, and a bunch of impressive algorithms that are part of a well structured, tested and maintainable suite is huge. Reading a book or two is not going to solve the problem in the long term, which I assume is what you want.

Hiring a software developer who can take a lead role in organizing and structuring the codebase will also make the specialists better programmers, since they learn good programming techniques from someone who actually knows them.

Source: Senior software developer, where I initially started as a "algorithm/application developer" and saw the team grow and benefit by hiring some experienced developers.

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#3

Post by wavefunction »

Lots of mistakes, and working with smart people who were kind enough to point out the deficiencies in my early and crude approaches, reading many articles and blog posts on every sort of programming topic, and finally and most directly important was reading through codebases at places I worked or open-source projects in my early years and learning from what other people were doing more correctly.

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#4

Post by testouts »

https://bramcohen.livejournal.com/4563.html ... the founder of bittorent.

Ill add a personal note. Being a good professional programmer is also about understanding the context of your program. You will not always have the time to optimize for performance. You will not always have the time to refactor functionality to make it more modular for further variety. Sometimes you will need to skip test becuase deadline says it should be done yesterday. Learn to not be emotionally tied to the code. Get your job done, to the best quality you can for the time you have allotted. Dont be made if its not perfect. But during the train ride or in the shower think about how to make it better. This process of just grasping with the ideas of clean code are good training for when you have to approach a similar problem on the fly.

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#5

Post by alook »

Look at some good open source projects, and look at the overall project/module structure as well as the OO graph.

Re: high-level project layout: - where are their build files? - how are the source packages broken up? - where are the tests stored?

Trace back from the entry point to the code and look at how things are encapsulated, where code gets re-used.

Diagramming out with something like UML is cumbersome but can be helpful to visualize the structure as you're getting started.

Looking at software design patterns (while often overkill) can give you a good sense for some common ways people try to re-use code. For example: - https://en.wikipedia.org/wiki/Design_Patterns#Patterns_by_Ty...

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#6

Post by snarf21 »

It wouldn't hurt to create some diagrams describing your current process. Then try to identify what parts are dependent on other parts. Also, try to encapsulate some of the pieces so you can separate concerns. This should start you down the path...

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#8

Post by mlazos »

It seems like you have a grasp of what is wrong already. Stuff you can do right now without reading a book - Use source control - Enact a policy where all new code is to be well tested (which you can enforce with some build/checkin system) - Make a refactoring change to isolate sections of code that minimally touch other sections and separate them into projects

As for books, I would suggest looking at Head First Design Patterns by Freeman and Robson. This isn't explicitly about project design, but it is about designing reusable code modules which can really help in any language. It is aimed toward object-oriented languages, (the examples are in Java) but it has helped organize my code in other languages as well. If you're using the OO facilities of Matlab, VB and python this can be useful.

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#9

Post by sounds »

There are distinct schools of thought. "How to structure a program" is subjective but also divisive!

Here are the major ones (that I'm aware of), making an effort to be inclusive:

1. Functional programming. It's possible I'm doing a "retcon" on several models here, but these are a fair start:

1.1. http://www.erlang.se/doc/programming_rules.shtml

1.2. https://www.seas.upenn.edu/~cis341/current/programming_style...

2. C (the language) organization. Embedded, linux kernel, and many open source projects follow this pattern. O'Reilly books are a good start.

3. Java. There are so many competing strategies that the "no true scotsman" fallacy applies. Here's one that seems uncontroversial: https://www.udemy.com/java-design-patterns-tutorial/

Re: Ask HN: How’d you go from a noodler to writing large well structured programs?

#10

Post by trengrj »

Have a read of Clean Code and the Pragmatic Programmer.

I’ve seen this happen before in teams that code all day but don’t see themselves as developers. You need a instigate a cultural change at work where it becomes unacceptable to not apply modern development practices just because the team sees themselves as “quants” first.

Post reply on HN