I'm not a fan of the label 'neoliberalism' (outside of IR
theory where it's well-defined and different) and think it's often a marker for poor quality partisan analysis, but the fact is it's simply prepending a 'neo' in front of the well established and broad classical liberal school of thought to distinguish it from the colloquial US use of 'liberal' to mean 'left wing'. And most of the people referred to wouldn't argue with the idea their ideas are in the ideological tradition of Adam Smith, even as they take great issue with some of the nonsense written about their motives and ideals and argue a label so broad that the entire EU political project
and most of its opponents falls into it isn't particularly useful for understanding politics or economics.
That's quite different from referring to the other side of a debate on sexuality as 'Cultural Marxism', despite Marx having had zero interest in sexuality, many self-proclaimed Marxist states having conservative policies on it and few people on that side of the debate having any interest in Marx. At best, you could argue right wingers describing their culture war opponents as Marxists are trying to invoke the spirit of Joe McCarthy rather than Lyndon LaRouche's ridiculous Frankfurt School conspiracy theory and literal Nazi origins of the term.