>most of the cost comes from building the plant in the first place, not from the fuel.
I don't think that's correct. The costs of handling spent fuel is ongoing for an unknown amount of time. Eventually the running costs add up.
The main storage site in the UK (which admittedly does more than just store waste) costs around USD 3 billion per year, basically as much as building a new nuclear plant every 5 years.
Then you have unforeseen events that can drastically change the calculations. Like the Asse II mine in Germany where a storage site started leaking and has to be relocated at an estimated cost of at least 3.7 billion euro over the coming decades. After that is completed, the relocated waste will continue to cost money to handle as well.
Such cost are generally NOT included, or has a fraction of the true liability included, in the cost estimates of nuclear power.
When you take such things into account it quickly becomes clear that nuclear is not nearly as cheap as many have been lead to believe, but instead very expensive and very heavily subsidized.
Consider also that even if it were cheap, it is strategically (both in a military sense and for reliability) a bad idea to have few large power plants instead of many smaller ones. The most recent nuclear reactor in the EU has had several emergency stops already and causes severe stress on the grid since it provides such a large percentage of the total power.
Having a geographically diverse set of solar and wind plants combined with local batteries, gas generators and small peak-handling gas plants has to be a better solution than nuclear power. More resilient, cheaper and faster to build, easier to maintain, not to mention a much more predictable total cost.