Live data from Hacker News

The silent majority of experts (2012)

prog21.dadgum.com

191–200 of 206 posts

Re: The silent majority of experts (2012)

#191
post #25

I think about this constantly while reading Hacker News. So many articles and comments by people whose level of enthusiasm doesn't match their experience. Try to voice a comment that goes against the flow of that enthusiasm and you'll be downvoted to oblivion, even if you speak with more experience and context than the masses. I'm horrified by some of the stuff I see here and feel like it's often useless to speak out…

>So many articles and comments by people whose level of enthusiasm doesn't match their experience

That also perfectly describes LinkedIn these days

Re: The silent majority of experts (2012)

#192

Earlier quoted context omitted.

Data Science is a poor term (unfortunately, the right one, statistics or statistical modeling, was deemed to be "unsexy"), but the expertise is certainly very relevant in today's data-driven businesses. Think predictions, forecasting, clustering, A/B experiments, recommender systems. Yes, some parts of what's commonly called Data Science can also be called Machine Learning, then maybe Deep Learning can be called AI,…

'Over optimization' means that a lot recommendation systems, A/B tests, forecasts in business are nearly irrelevant, or rubbish i.e. a lot of cost for very marginal value add. Such a small margin that it's within variance of what might even be considered a positive outcome. There are few business that have the kind scale in which a lot of that will be hugely useful (at high degrees of sophistication), let alone, affo…

This is take I don't agree with on a real problem, which is the usefulness of data science in modern business, and for which processes in the business.

I try to explain.

Recommender systems add tremendous value, millions when not billions in revenue, to any retail business, streaming business. Is it by chance that all the online retailers had to set up a recommender system?

A/B tests are not needed? If A/B testing and actions taken looking the results would not be useful, don't you think that business would know it?

Forecasts in business are nearly irrelevant, or rubbish, you say. Like in the sense that in complex systems is irrelevant to know what's going to happen next, that is better just to play it along at best? Having worked in companies in media, retail, healthcare, and telco for which forecasting is crucial for planning the use of resources, negotiation of contracts etc., this comment is off-base.

Going from abstract to practical, I have worked recently with one telco giant on one of the services they offer. Their forecast of subscribers done by drawing a straight line that follows the latest data is making them leave tens of millions on the table each year. I proposed to make some A/B testing on their "coupon" strategy and they were not interested, preferring to send a "coupon" to everybody and their grandfather because they did not want to study for a week how to set up an experiment. It is mostly laziness. Clearly to know if A/B testing is working or not (say increase engagement, revenue etc.), one has to try it. But they don't, because they are mentally lazy.

One thing I agree with is that most legacy business are not ready to adopt a data-driven modern way of doing business, due to stiff processes, old-timers that would like to go on with their business like they were doing 30 years ago, and general fear of changing.

The fact that the legacy businesses are still "working" may give the impression that all these data-driven "baloney" is subtracting rather than adding to the business. Why do we need forecasts, they say, I cannot believe that these people want to use data, isn't our expertise enough? Don't you see we are the still among the 5 top companies that do what we do?

But their position in the market is not due to sound business practices, but due to their dominant position in the market. But then the dominant position goes away and the old-timers finally recognize that A/B testing, forecasting etc. combined with domain-knowledge are a plus for the business.

Maybe you don't need 1,000 DS, maybe you need 20 (practitioners, not "researchers"), that depends on the business. Clearly the mom and pop business does not need any data science.

Re: The silent majority of experts (2012)

#193

Earlier quoted context omitted.

There's a weird parallel to the pizzagate kerfuffle. Lots of people who wanted to get to the "real truth" by only looking at online sources, like it's some sort of virtual escape room that's been pre-built with clues. Staring at google street view images and trying to find the pattern in all the store signs on the same block, coming up with bizarre circular logic around "cp" where references to pizza at a pizza parlo…

I'm not sure I see the parallel. In the first example, a forum poster was taking the absence of evidence to be evidence of absence. In fact I've never met a person who didn't have a story about making such a mistake, so I'd venture to guess we all do that at some point in our lives[1]. In your example, however, people were confusing the absence of evidence with an imagined preponderance of evidence. That's an error o…

I think I see the parallel as being more a streetlamp problem (looking for the keys under the streetlamp even though you dropped them in the yard, because there's no light in the yard). "Absence of evidence/evidence of absence" and "seeing patterns in random noise" are just two different ways brains mess up when confronted with empty sample data.

The thing that's in common is a belief that if there was something to observe, it would necessarily be in the place you're looking. Maybe it's a form of map-territory confusion.

I've seen it quite a lot professionally, whether it's "let's sell business travel via facebook rather than linkedin, because facebook has better analytics and conversion tracking" or "a disorder is only real once it's listed in the DSM-V" or "this UX issue can't be a real issue because all of our current customers say they are ok with it." People sometimes just pick a systematically wrong approach to learning about the world, leading to a wide range of silly outcomes.

Re: The silent majority of experts (2012)

#194

Earlier quoted context omitted.

> I will be[t] $1000 immediately that kid will do well without knowing a single other thing about that family or situation. I'd take the other side of that bet; in my experience, a parent who still feels the need to take on their adult child's domestic labor likely hasn't prepared them well for independence in other respects, such as managing their own education well enough to get good grades in university classes.

Eh, it's a bit culturally dependent. My native culture prizes gender equality and self-reliance; has washer-dryers in every home; and doesn't care about wrinkled clothes. If a grown man from my culture needed help with his laundry, it's probably because his life's a shambles - if you're struggling with depression or something, that ain't a good sign for your exam success. On the other hand, someone from a more patria…

These responses are getting quixotic.

Nobodies 'native culture' prizes 'equality' to that extent, if you're from a culture we're all familiar with I'd fathom you're projecting pretty hard.

In some places in the world, there are some 'modern ideals' around those types of things, but it's more 'ideology' than 'culture'.

Students are not running around forlorn about not having their 'pressed khakis and dinner shirts needing pressing'. God forbid. The Mother who pops in with Rizotto & Scnitzel for their kid and her roommates, grabs their laundry, making sure they have their train ticket to 'Auntie Lucy and Uncle Chuck's Wedding' (because the date was changed!), gossips about their cousin who just got into Flight School in the Air Force. That's 'conscientious parenting' and extremely hard to do as a single parent at that level. 20-year-olds are barely adults, arguably, transitioning to adulthood.

Re: The silent majority of experts (2012)

#195

Earlier quoted context omitted.

I tell myself it's the difference between starting adulthood at 14 or whenever your family fails you, and starting it at 24 or 26 after college. The extra years of "streetwise" usually come at an expense of early onset of burnout, for various reasons. Leaving your shell with the energy to take over the world at 26 usually results in more productive endeavors than when you are a teenager.

>The extra years of "streetwise" usually come at an expense of early onset of burnout I almost cried reading this sentence. I've never realized how my precocious adulting wasn't healthy: every adult compliments you about how "mature" and "responsible" you are. You're actually just anxious because you paternal responsibility on yourself : I had to basically raise myself. The cost of this "independence" has been that,…

Hey cool, are you me?

26 years old with a father with Asperger's (with the anger issues and emotional non-understanding that comes with it), an over-bearing mother who sheltered me from pretty much everything crippling my social and personal skills growing up preventing me from getting help when I needed it - now I'm playing perpetual catch-up with both my personal life (social/romantic relationships), and the untreated mental problems I've been suffering from - I'm finally realizing the problems I've been beating myself up about through my whole life are not really my fault and I should have gotten help with my issues much much earlier.

I can only wonder what life would have been like if I would have stood up for myself much earlier and gotten the help I need - despite the repeated claims from both parents that I'd ruin the family and rip everybody apart if I did try to get help.

Re: The silent majority of experts (2012)

#196
post #106
post #25

I think about this constantly while reading Hacker News. So many articles and comments by people whose level of enthusiasm doesn't match their experience. Try to voice a comment that goes against the flow of that enthusiasm and you'll be downvoted to oblivion, even if you speak with more experience and context than the masses. I'm horrified by some of the stuff I see here and feel like it's often useless to speak out…

Horrified is a bit strong, but you have a point. There's a few themes that are strongly popular or unpopular by some type of community consensus. In those cases, it doesn't seem to matter what you have to say as minds are already made up. Another interesting effect I experienced is regarding expert credibility. When I found this place, I was impressed. I figured the world's top engineers are posting. I see them writi…

I’ve noticed this when it comes to subjects I have a higher than average (but not necessarily expert) level of knowledge in, in articles in my country’s major news channel.

Re: The silent majority of experts (2012)

#197

Earlier quoted context omitted.

>> As I grow older, I see a huge divide between those who have/had parental support vs those who don't/didn't. This really gets lost in the traditional nuclear family argument. From personal experience, having both biological parents in the same house doesn't mean a healthy family environment, and no amount of moralizing is going to wish it into being. Also overlooked is the impact one can have through simple acts of…

> Didn't know it then, and doubt that teacher has any idea, but it changed the course of my life. Reach out to him and let him know. It's these kinds of feedback that can really, really make a person's day, and it's what every engaged teacher dreams of/works for.

I had tried over the years and even went back to the school, when I visited the city a number of years after. It was a large school and turn over was common. Not to date myself, but this was decades ago - they would be near or in their 90's and a very common name.

Maybe it worked out better this way, it keeps me honest - like the lack of closure keeps me from storing it away and forgetting the lesson in it.

Re: The silent majority of experts (2012)

#198

Earlier quoted context omitted.

>> As I grow older, I see a huge divide between those who have/had parental support vs those who don't/didn't. This really gets lost in the traditional nuclear family argument. From personal experience, having both biological parents in the same house doesn't mean a healthy family environment, and no amount of moralizing is going to wish it into being. Also overlooked is the impact one can have through simple acts of…

The data is conclusive on Single Parent vs. Two Parent and it's not even close. The 'moralizing' is from those who want to equivocate. This is also from deep life experience myself, with close family and colleagues. In everything but the best scenarios: 'Everything Is Hard' in a single family household, whereas things 'Can Be Hard' in regular family scenarios. I think a 'single mentor' can make a big difference, but…

>> The data is conclusive on Single Parent vs. Two Parent and it's not even close.

Not sure where single parents come into this? I have no strong opinion on it one way or another. I was referring to the the argument for having both biological parents in the house, that it doesn't automatically mean the home environment is healthy. It doesn't mean that having both biological parents in the home automatically equals support for the child. That is what is lost on that argument, that the actual physical and mental well being and support of the child is not automatically granted just because it's traditional nuclear family home.

There are other family structures that can more than provide for the physical, monetary, and mental health of the child. Coming from a traditional (US) Southern up-bring, the moralizing I heard is how all the social problems can easily be solved if the parents would just "work out" their problems. At that time, and still in places today, it was code for the wife to shut up and obey her husband - and if he beats her, what did she do to upset him? if he sexually assaults her, wait, just kidding, he cant because providing sex was her on-demand-duty. Is he beating, berating or sexually abusing the children? No one will ever know because if it's happening, the wife is socially blamed (if it leaks out) for failing her duties and no-one dares talk about it. Mostly the same for the wife treating a child in this way, except this time the blame and responsibility is rightfully hers, if it ever gets out. Having women work outside of the house really messed up this racket, thats a lot of self-serving control to lose for those that have lost it, or the promise of it, and the moralizing I hear because of it got louder. My point being, obviously our experiences of childhood and family are all different - however, this is the lens I, and many others, view through

Re: The silent majority of experts (2012)

#199

Yep. On a related note, when I was younger, I searched for advice from experts for how to develop expertise in studying and productivity on Reddit. It led to lots of highly-upvoted advice (including stuff like supplements, largely with few real benefits besides placebo), popular blogs by influencers (like Scott Young), and popular self-help books. However, the actual experts I knew in high school who later went on to…

I disagree, it depends on the subreddit - a subreddit that discusses supplements is by it's very nature going to be populated with snake oilsmen selling remedies or doofuses who believe them. Similarly self-help books. You have provided examples of domains where charlatans are most likely to operate. I have learned a lot from user communities in reddit and hackernews. And hopefully contributed back as well. There are…

>“There are folk who have domain expertise and have time on hand and are hungry to contribute.”

There are great people willing to share who do exist, though there is a selection bias where experts with certain values and beliefs are far more likely to post than experts with minority views about technical subjects.

From the article on popular beliefs in online communities at the time: “Just because looking down your nose at C++ or Perl is the popular opinion doesn't mean that those languages aren't being used by very smart folks to build amazing, finely crafted software. An appealing theory that gets frantically upvoted may have well-understood but non-obvious drawbacks.”

The view expressed in the article, which I agree with, is that there are experts with minority views on technical topics, who are difficult to find on forums.

>“You have provided examples of domains where charlatans are most likely to operate.”

I agree that you’re unlikely to find these kinds of life advice on technical niche subreddits (like on r/compilers).

However, let’s say a user is a young person looking for life advice. This subject would typically be off-topic on r/compilers and similar forums, so it’s off to more lifestyle-oriented forums (like r/decidingtobebetter or r/selfimprovement).

>“a subreddit that discusses supplements is by it's very nature going to be populated with snake oilsmen selling remedies or doofuses who believe them. Similarly self-help books.”

This isn’t wrong, and there has since been more awareness of paid marketing on subreddit. However, it’s non-obvious to new users, especially young people. If legitimately useful comments get lots of upvotes, there’s an effect where the ones promoting self-help books and supplements seem credible too.

For lifestyle advice in particular, sticking to smaller technical communities may not yield useful discussion. Going to other online communities leads to the problem you’ve described. By process of elimination, it’s viable to start observing and talking with people in real life.

Re: The silent majority of experts (2012)

#200
post #33
post #28

Earlier quoted context omitted.

Sometimes there really is a "one weird trick" that can help people. I used to get stuck on problems and it would cripple my productivity. A programmer with two decades more experience than me noticed what I was doing and gave me the best advice of my career. If you are interested in the solution, you can signup for my . . . Seriously though the answer was to just do something else. Don't sit around thinking about the…

That's the problem - the self help books often have good actionable advice just like the diet books do; but it can be summed up in a few paragraphs. Most of the book is often trying to convince someone it's worth doing, or providing ways to get around procrastination, etc that might prevent it. But as you've shown, often a good mentor can do just as well, or better.

Many people reading self help books would prefer a good mentor. Books are easier to find.
Post reply on HN