The art of over-engineering your side projects

It's interesting that you choose Instagram - a highly successful company/product - as an example to warn people against.

It may be harder to come up with examples of people who failed, for the obvious reason - they failed and are therefore not well known, but I really don't find the statement that Instagram "nearly lost it all" very compelling. Did they? Really? If they had had even more problems scaling, would they really have lost users? Look at Twitter, they took years to resolve issues.

I actually think the importance of stability and uptime are overblown in the early stages. If I think about the times downtime has significantly impacted my view of a company or product, it's when large established services I'd come to rely on professionally went down. You need to establish a reputation for reliability before most people expect it of you. That's not to say it's not important, but you need to look at context. Noone relies on Instagram minute by minute for their professional work, so the priorities are different. UX, marketing, network effects, etc. are going to trump stability.

Yep. That's why I've rebuilt my little blog about a dozen times. I've built CMS's from scratch before (actually built a white label CMS generator) and it's a non-complicated process that I know deeply enough to push the pain points when I try things out.

When I do side projects it's normally because I want to explore something which is too risky for someone to do (or it would most likely have been done). In my experience that normally requires some sort of overengineering because your problem and solution isn't common.

A few of them turned into great side businesses but most of the times it help me explore technologies and projects while still solving real problems.

I've had plenty of ideas that I'd love to see through and think are great, I just don't have the time to do them. Since my time is so limited, I have to choose to focus on one project, or I end up with a ton of unfinished ones (which is the case). So unless I have strong motivation to complete a project (which can be compromised by some sort of gratification), it stalls.

But the majority of mine stall for other reasons; I don't usually announce/discuss them.

A major roadblock I find is that when you want to release your side project and start charging for it, I feel it becomes much harder to make changes after. This means you get caught in a loop of trying to polish it more and more before biting the bullet and releasing it.

If you change the interface a lot or how features work, you'll annoy users that don't like the change and you'll have to be extra careful not to introduce any bugs. If users can save data, the new version will have to work with the old data format. If you want to move features from a free version to the paid version you'll annoy users. So to me, releasing a MVP you charge for that might change a lot later might lead to more headaches than developing the idea further.

Are there ways around this?

Side projects you release for free are one thing but when you want to charge for them the amount of polish and attention to detail you have to pay to marketing jumps by an order of magnitude. People will tolerate bugs and rough edges for free projects but not for paid ones. I agree with the advice about not wasting time with CI, switching frameworks etc. though.