On one hand, you're totally right.
I've been writing perl for twenty years, and while I love its practical expressiveness (higher than any other language currently in use, as an objective measure), I've frequently wished it had better performance and lower memory footprint.
The go-to solution for that has been to profile one's code and rewrite the critical path in C, with which perl interfaces readily. It's the same story with python -- very expressive, but slow and fat (slower and more memory-hungry than perl, even), with easy C integration as the common solution. C integration works, though it's less than ideal.
On the other hand, you're being a bit uncharitable by assuming the MoarVM developers aren't focusing on performance.
They prioritized getting it working -first-, but have since paid more attention to improving its run-time performance. Their efforts have already made considerable impact, with more on its way.
I've benchmarked its performance a couple of times, and it's still too slow to interest me, but will keep an eye on it. At some point there may be an inflection point which makes it a compelling alternative to other languages.
Though perl is still my bread-and-butter, and I've written python for a living and liked it, I've become quite infatuated with D of late. It's essentially C with some straightforward extensions and improvements, which gives it roughly 3x to 4x as much practical expressive power than C (but only about 1/3 the expressiveness of perl), and excellent support for casual parallelism, while matching C for run-time performance and small memory footprint.
Before I wrote perl for a living, I wrote C for several years, and D has "clicked" with me in a way that other languages have not. Learning python and Go was rough, but I muscled my way through them. Learning D hasn't been like that. It's actually pleasant to learn, and I find myself thinking about it when I really should be focusing on other things.
One thing D doesn't have is a lot of jobs. Maybe it will someday, but I expect to write perl6 for a living before writing D for a living (and expect to be writing perl and python for a living for a while before that).
Anyway, my point is that the MoarVM developers are actively working to give perl6 better run-time performance, and it is way too early to discount perl6 on account of its current performance.