I have worked a very similar way for the last decade. My company works largely in OSHW/OSS and I consider the work I do for closed source companies as "extra stuff" that I expect no long term income from.
I am a one man show...everything from Janitor to CEO duties are my responsibility. Every hour you are not in the shop you are paying for an unused building. Work (and therefore income) stops when you stop. It is hard to go on vacation for a week when you know that the meter is running and that shop costs you $xxx.xx/day.
Automation is key, in my case I use machines to supplement my time (3d printers, CNC mills, Pick and place machines, website sales) and if it is something which is better done out of house I use other companies to do the work (production machining, contract manufacturing).
Even still...it is a very difficult path. Paperwork is neverending, phone calls and emails come at any time and you are typically expected to answer them immediately. Health insurance is not included (United states), vacation/sick days are not guaranteed and in my case...retirement is something I don't even consider an option.
Why would I do all this? Because it leaves me the ability to work 20hrs a week (physical hands on time...not counting admin stuff) half of the month to focus on my loved ones and the other half of the month I work 50-80hr weeks. I have the freedom to choose to get up in the morning and take a Tuesday morning hike without having a manager upset that I clocked in late, or go swimming during the summer midday during the time when I am out running errands.
Would I make more in industry? Probably...but not likely locally. Besides...I am the type of person who does not do well in a "conventional" situation.
Yes, you do trade one boss for many customers...but you can also fire customers. From what I have found in my experience though is that it isn't the customers who are the hardest on you...it is yourself. Trying to let go for a few hours to just watch a movie without thinking about work is difficult. When I contract myself to do a design for a customer I tend to think about that design nonstop until it is complete...then often after I am still revising it and thinking of how it could have been done better/faster.
Someone once said to me "When you are self employed your boss is always an asshole." I recommend anyone who is debating becoming a solopreneur to remember that.