Earlier quoted context omitted.
Alpha tests are, by definition, internal-only. There are several different definitions. One definition I like is that alpha testing is when you know there are still missing features, while beta testing is when the planned features exist and you're just trying to make sure that they all work properly -- in which case you could very easily have public alpha testing.
We have done lot of testing on our application using some external users as well and fixed close to 2k bugs to make it more stable. As you are suggesting, we are planning public Alpha.
* blink *
How much code do you have? I've been writing code for my own startup for about 20 months, and after 25k lines of code I'd say I've fixed maybe 50 bugs in total.