I think a major problem with this proposal is that all votes could eventually be knowable. Let's be honest, the state of the art encryption we use today will one day be sub-par. New has become antiquated a dozen times over the last 30 years already.
Put enough compute behind that blockchain and eventually you'll crack it, or run Shor's algorithm on the first Quantum Computer and votes from a decade ago are now public.
My contacts in the intelligence community say this is their biggest fear with intercepted communications today. They may be 'secure enough' today, but some of the information moved around would still be devastating if it were decrypted 20 years from now by a bad actor, not all, but some.
The way you should go about this is little different IMHO. Have two blockchains, one that checks if people have voted, and one that records the vote. If you have not voted, then one anonymous vote may be added to the ledger of votes for final tallying. If you have voted, your request is blocked. Maybe you get a random key that gives you the right to add a vote if you pass the first checkpoint, but that key can not be associated with the record that you voted on the other blockchain.
Cool idea. I for one prefer physical voting. It sucks, but I feel safer with it.