I also live in Australia, and am an avid cyclist - I ride to work every day, and I'm sorry but
no, I strongly disagree.
There is nothing wrong with wearing a helmet - I've come off my bike before and its made a potentially very bad situation painful rather than life-threatening - and really look, on the list of worries of cyclist, having to wear a helmet is the least of our worries.
(The law about "not riding on a footpath" is probably more annoying - occasionally, you need to, when its the only safe option).
However, what is an issue in Australia with cyclist is attitude particularly with car drivers. Until that changes, it will always discourage potential cyclists.
Whether it's cars who will literally chase you down and scream about how they "deserve to be on the road and you don't", or people who open doors into the bike lane without checking for cyclist etc.
Ironically, I find it actually tends to be the lower socio-economic areas that find the concept of cycling most puzzling.
In the more expensive areas, cycling seems to be more of a thing - sadly, road cycling is a rich man's hobby here - when you have groups of Cervelo and Bianchi's out on weekends, those people often tend to be a bit more considerate when you're on the road yourself during the week.
I do wish cycling was more mainstream, or at least car drivers better educated - but I don't think that removing helmets is the way to do that.