Well, are you an executive, or one step from becoming one? Then it's entirely possible that spending more time socializing recreationally can directly help your career. Golf happens to be a common choice, but maybe your exec team likes rock climbing or playing video games... just depends on the company or the social group.
Also, whether you are an executive or middle-manager or an IC, overall I don't doubt that socializing and networking and investing in good relationships, is helpful to one's career, but to differing degrees (i.e. for execs = a lot, for middle-managers = a medium amount, for line-level tech ICs = a much lesser amount). And building relationships or earning trust by spending time together != "sucking up".
Just the implication that you associate "golf" with "sucking up" apparently, shows that you likely have an inherently antagonistic view of personal relationships. For any given group of people, either they truly like playing golf, or golf just happens to be a convenient activity that allows for lots of downtime, and therefore encourages discussion/communication to happen, either way it's very likely that they are getting together for reasons other than "sucking up".
Why do you assume that what happens on the golf course is inherently or inevitably about "sucking up"? Have you never encountered a group of people, getting together for some event (recreational or otherwise), where they just communicate on equal footing, or debate ideas together, but with no one being fawned over or worshipped?