Here is what I have discovered after grinding at that process many times: You have to find a mechanism of internal feedback and drive the project through that.
What I mean there is, simply, when you get the idea and your first instinct is to seek confirmation from others, you've already erred by basing the entire thing on the whims of others. Even if you get them to become stakeholders with a substantial interest, they are still most likely operating on their whims. They go to the meeting because they are obliged to have a meeting, and the extent that they care is simply the extent that you have entangled them in it. You cannot rely on others in this regard; they can be great friends in most aspects of life and still be utterly perplexed by your ideas, even if they start seeing them working in practice. We all fly to our comfort zones. But this means that being a good student, a hard worker or a loyal friend is not sufficient to cross the threshold into realizing something original.
What you can rely on to make that crossing are tools of logic and reason: first do some philosophy and determine your thesis. Find principles, themes and references to give it structure to hang the work off of. Make sure it coheres in an explicable way, with logical premises: compare everything pair-wise, make diagrams, whatever you have to do. There are a lot of possible "systems" but having one is the important thing. You do not want to be in a position of blind faith in the idea alone, and a strong high-level concept has the power of making it easier to rationalize, explain, and argue for a certain way of doing, thus eliminating your doubts.
Once you have some starting point, generate requirements with binary true/false factors: it must have this, it cannot have that. Sketching out requirements can tell you something about the principles you have in mind so it helps to cycle between the two a bit. But you don't build a solid project just on requirements because those are so often subject to whims: "why do it that way?" "meh, felt like it." You instead challenge each requirement against the broader idea to ensure that it works at scale.
From the aggregate of binary factors, key metrics to focus on also appear. And as you do requirement generation, the motivation to do the work appears naturally because you have to do it to answer the question!
And of course, doing this, it becomes a marathon, not a sprint, because it quickly takes you out of the space where you just grab off-the-shelf solutions and slap it together. You start going well out of the comfort zone to answer your question. Now you have experiments to run and research to follow up on. The principles and themes of the project never change once established, they only become more refined and nuanced as you evaluate what the best means of achieving them are.
And I think that is, in essence, how you do it. You can achieve something resembling a successful creative work by accident, but if you stick to a principled approach your chances improve sharply.