This is XP (http://www.extremeprogramming.org/) applied to a one-man-army. I wish this would be common knowledge.
My approach to building large technical projects (2023)
51–56 of 56 posts
Re: My approach to building large technical projects (2023)
#52I have huge respect for Mitchell, it's impressive what he achieved. I agree with all the points of this article and would like to add one: Have a quick feedback loop. For me, it's really motivating to be able to make a change and quickly see the results. Many problems just vanish or become tangible to solve when you playfully modify your source code and observe the effect.
Would you say that testcases help here? I've been thinking about applying e2e tests on any bugs I find so I know they're fixed
MitchellH also talks about this in some interviews he gave about Ghostty.
Re: My approach to building large technical projects (2023)
#53I start with the standards. What file types and protocols and external tools do I want to support? How do I make Git and SyncThing understand my files?
How do I build this on an unmodified system with unmodified popular and trusted libraries?
How do I make sure that users don't have to modify my project to use it, unless they are going to be contributors?
If I'm truly just building for myself, which almost never happens, I look at resuability. If I have to learn a new tool, what's the most general one, that I'm likely to be glad I learned when I'm doing some other project?
Re: My approach to building large technical projects (2023)
#54Earlier quoted context omitted.
This perfectly aligns with my experience. Every large project I have worked on showed a clear correlation between the ease of setup and running and the number of problems on the project, like bugs and missed deadlines.
Totally agree. I work in LLM training software and I believe progress in the field is actually much slower than it should be because of the excruciatingly long feedback loops involved in development. The software stacks are deep and abstract and much of the testing involves full integration tests that take a long time to spin up.
Interesting. What aspects of the development workflow/cycle have the most room for improvement (i.e. is there ranking of the "height" of the "hanging fruit" throughout the process)? What sort of software tooling would help?
Re: My approach to building large technical projects (2023)
#55I feel there is a balance to strike with rushing to a demo. .. (maybe it's an implicit tension between the satisfaction of finishing a component and getting a demo) I think this is where the choice of language makes a big difference. In Clojure, the difference between a "component" and a separate library/application is literally just adding a `deps.edn` file and then pointing to the directory from the parent project.…
> By contrast, if I was working in C++ making an API and decoupled library would be such a chore that'd never bother
Are you sure? Isn't it common to build convenience libraries, not because you need to have it as a library, but as an internal documentation? To the produced binary it has exactly the same effect.
Granted maintaining a stable API/ABI is work, but that comes down to architecture and is true in every language.
Re: My approach to building large technical projects (2023)
#56Great read, but I was expecting something different based on the title. This sounds like his approach working on personal projects. I'm really curious about large technical team projects though. What's the best approach to getting stuff done and making sure everyone is working towards the same goal etc. After 15 years I have yet to see a technical project that hasn't run over budget, over time, under delivered or bur…
Maybe I’ll get maligned for saying this, but as someone who’s managed and successfully delivered software projects of all sizes, I’ve become somewhat of a convert to the Scaled Agile Framework. I treat it exactly as intended, as a framework. It’s a strawman to adapt based on context, and what I value most is how it led me to explore deeper source material and form my own conclusions. Over the years, I’ve learned that…
> If we instead focused on continually steering toward desired, valuable functionality (things people genuinely want or will pay for), more software projects would appear like successes.
Yes! All overbudget, late to complete, successes of course.
I jest a bit. I think the term "deadline" is completely inappropriate for most projects.
I prefer, "target series", for any significant project.
Each word communicates something very important.
The first target should be set via (1) known knowns/unknowns, (2) some wisdom about unknown unknowns, and (3) mindful optimism.
The last should almost always ensure the project misses its first target. Which sounds backwards, until accounting for the considerable creative impact, and absolute time savings, that a time constraint creates. And how much a team can improve its productivity over time, by continually targeting informed optimistic targets.
Improving a teams rate of return has compounding value, for everyone.
But this is a self-imposed, let's see how well we can beat ourselves, time constraint. Never an, OMG I might lose my job / standing / reputation / customer credibility time constraint.
If the first target is missed, or as soon as it is clear it is wildly unachievable, a second target is set the same way, with the new information.
Each target in a series should require less padding. If more padding is added it is a sign to step back and consider what is not being understood.
Over time, a team should develop a pretty good pattern of completion at target 2 or 3, or something like that. In a way that becomes fairly predictable. It doesn't really matter what the number is, just that the team develops a rhythm that actually does provide some predictability based on current knowledge, even if it isn't in absolute time terms.
This is the best path for optimizing both rate of progress, fast recognition of problems with progress, and realistic levels of project arc predictability. In my experience anyway.