Yes, I mean it as a metaphor. I do enjoy beer and video games personally and absolutely don't mind a group of colleagues going for beer if they like. Because then it is a group of colleagues out of many not company culture. I wont investigate the fridge during interview for stocked bottle. It is when the beer and pong come up as larger point during interview, when hiring manager brags about non-work oriented things whole company does together often. If they went the diversity by cringy art way and bragged about common coloring sessions, I would run away too. It suggests a couple of things:
a.) Less meritocracy - whether you appear as good programmer while you play that ping pong matters more then your code and responsibility. So it is signaling and charming game where I am at disadvantage. Also in worst case, in case you are good looking and succeed, such culture makes it too easy to assume/imply behind your back that you got position because of being sexy.
b.) It suggest that people with different personality types left. E.g. they might have different genders and races, but they are still kinda clones of each other.
c.) I hate having to stay overtime or during weekend because basically there was 4 hours long ping pong match at Wednesday and that is when CEO shares requirements, what (s)he learned from customer last time he visited and when design is discussed. I strongly prefer one hour long meeting about the above and then going to work and after work everybody does his thing (which may include ping pong).
d.) True story, I found it unfair when a dude was seen as hard worker because he spend 12 hours a day at work - except that he spent majority of time attempting to charm lady at reception and otherwise socializing.