I work on a machine learning team that has gone through the exact same process. As others have noted, the biggest change for us was cultural.
If you send me a message I would be happy to chat about it.
Ask HN: How’d you go from a noodler to writing large well structured programs?
21–30 of 77 posts
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#22Recommended resources to consider:
* Code Complete https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...
* Programming Pearls https://www.amazon.com/Programming-Pearls-2nd-Jon-Bentley/dp...
* Pragmatic Programmer https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#23Recommended resources to consider: * Code Complete https://www.amazon.com/Code-Complete-Practical-Handbook-Cons... * Programming Pearls https://www.amazon.com/Programming-Pearls-2nd-Jon-Bentley/dp... * Pragmatic Programmer https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...
One more I missed regarding review of architecture, code structure/design of OSS: The Architecture of Open Source Applications (Vol 1 & 2) https://www.amazon.com/Architecture-Open-Source-Applications...
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#24I will reiterate some of what others have said:
- Use version control!
- Hire a professional software developer
- Good architecture and abstraction comes mostly from practice but if you really want to do some reading on software architecture check out "The Architecture of Open Source Applications" (vol 1 & vol 2)
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#25As a baseline recommendation try this: turn some of the apparently unfactorable pasta code into a large inline function and then start factoring from that point. Usually an opportunity will appear to realign the code in a new way that gets you something you didn't have before. Repeat a few times and you will end up with some original code abstractions that you never would have seen without a careful evolutionary process.
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#26You learn it by writing out of hand code :)
But more on the point, what has helped me immensively has been writing out my thought processes before starting to code. Write out the problem, and solve it in a journal or somewhere before just starting to write out code and go along.
There's nothing wrong with just writing code as you go and solving things while you find problems, but to get structured, documenting the behaviour of the program is a major point. Draw images how the code should work. What is connected to what, who owns what ?
By doing this process, you start to see the big picture. It's not always easy, and the tempation to just "go at it" is always there, but this easily leads to code you need to re-factor or re-think more than you would like.
But of course, sometimes you just have to write the crap version first, and then move on to a more advanced model. Actually, this is the expected way, as coding is a field where you just can't know what lies in that valley before venturing into it. So, accepting the fact that re-factoring is something you do constantly is also key. If you don't refactor, the old crappy solutions hinder and slow you down.
Experience of course helps you make better decisions in new projects.
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#27Do searches that specifically address what you are trying to do and learn from the resulting blog posts....
https://www.google.com.au/search?q=how+to+structure+matlab+c...
Each time you encounter a new concept, search for that too.
Search for "beautiful matlab code"
Learn from others. Aggressively search for open source matlab code bases and read them.. look at their structure and try to understand why they are doing things the way they do... email the authors and ask them questions.
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#28IMO, the two easy things:
- read other similar projects and understand how / why they work
- fix problems that you see.
If there are packages you're using from other systems that's often a very easy place to start with the first one... follow through all that code and see how it's organized.
With the second one, I'll say that it's really easy to put up with crappy workflows just because it is what you're used to. You have to not be willing to put up with the things you don't like in order to fix things. Use source control. Don't write functions that are 300 lines long. Think about the times that you have problems in your code and don't let those happen in the future.
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#29As other commenters have said, I think you should consider hiring a software engineer. It's easy to get overwhelmed by tools and minor details.
It's difficult to give you advice, since you've asked such a general question. I've written some thoughts on the matter, but take it all with a grain of salt. At best, my comments are spherical cows [0].
There's nothing wrong with having everything in a single folder; it really depends on the project. If you find it difficult to navigate your way around the source, I'd say there's two high-level approaches for organizing application code: you either group things by feature or by functionality. For example, many traditional web frameworks group source by functionality, so you have a folder for all models, controllers, views, configs, etc.
When structuring application components, you'll typically want to separate things into pure modules and anything which produces side-effects. Pure modules are typically easier to test and debug. If a module depends on anything, consider passing in any parameters during startup or when calling the module, rather than having it look things up on its own. That's the basic idea behind dependency injection.
Git is pretty much the standard for version control. You can get pretty far by just memorizing a few commands [1] and throwing up your code on Bitbucket, GitHub, or GitLab. To make life easier on yourself and your teammates, I'd suggest using a GUI tool like SourceTree. It makes it much harder to shoot yourself in the foot, while providing a far more approachable and discoverable interface. I'm not the biggest fan of git, but since it's what everyone else uses, you pretty much have to suck it up.
Since I'm not familiarized with the ecosystem I can't give much advice with regards to testing. The closest thing to a standard testing interface that I'm aware of is TAP [2]. I'd highly suggest checking out "An introduction to property based testing" [3]; it's one of the most insightful testing-related talks I've seen.
[0] https://en.wikipedia.org/wiki/Spherical_cow
Re: Ask HN: How’d you go from a noodler to writing large well structured programs?
#30Have a read of Clean Code and the Pragmatic Programmer. I’ve seen this happen before in teams that code all day but don’t see themselves as developers. You need a instigate a cultural change at work where it becomes unacceptable to not apply modern development practices just because the team sees themselves as “quants” first.
I would actually encourage OP to avoid Clean Code.
I found Code Complete to be a much better book. It encourages a deliberate, thoughtful, sustainable approach to the structure of your code, and provides good examples.
Clean Code encourages you to decouple everything as much as possible, which in my experience leads to poor abstractions. (See https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstracti...)