I did this as well - I quit my job, started reading Ruby books and wrote the Alpha version of Pikimal.com. I did this because I'd been the CMO at another start-up where I'd really not understood the dev process well enough and had thought that it had real costs so I have a pretty good sense of the benefits.
We're pretty busy, so let me cut to the chase:
The good:
I'm credible to our tech folks. I understand when to slow down to integrate a library, and I'm more than willing to find time to re-factor. I learned that pair programming can be awesome, that bringing on new people slows you down, and a lot of other things that probably seem obvious to technical folks but are non-intuitive when you imagine software as assembling widgets.
The bad:
Ultimately, we gave up 3-5 months that we could have been hiring, fund-raising, and building software. On balance, I think that this was good for me and bad for this start-up.
The ugly:
As soon as I'd hired people, I think that we should have thrown out most of the code I'd written. We kept it because it worked and was fairly fast, but I'd made A LOT of newbie mistakes that have slowed things down.
So, tl;dr? I'd suggest learning to code but I'm not sure that you should do it for your current start-up. At the very least launch something more sophisticated than a CRUD app and THEN get started on your small biz.