As much as I hate Vimscript, I don't think changing it to something else would work.
First, there's just a ridiculous amount of plugins and such already made. Either
they'd need to be ported to the new interface (an insane amount of work) or the old
interface (Vimscript) would need to be maintained alongside the new. Neither option
seems very practical.
Second, the main advantage of Vimscript is that it uses the same commands you use
every day while working with Vim. `edit foo.py` in a Vim plugin does the same thing
as `:edit foo.py` in your daily routine. This makes it really easy to get your feet
wet with Vim scripting -- in fact it makes it all but unavoidable.
Users learn tidbits of Vimscript while working, then start hacking stuff into their
.vimrc files, then move on to learning more and writing plugins. It's a fairly
natural progression.
If you're going to change Vimscript to something else you have a couple of options,
all of which suck:
* Replace Vimscript in the daily routines with Lua/whatever. Instead of typing `:e
foo.py` I now type `:edit("foo.py")`. No way this would ever work with
Vim's community.
* Keep using Vimscript for daily commands, but use Lua when reading from files. This
loses the main advantage of Vimscript (which I think is a really important one),
plus you now need to maintain two interfaces.