Live data from Hacker News

The Silent Majority of Experts

prog21.dadgum.com

11–20 of 38 posts

Re: The Silent Majority of Experts

#11
post #2

But without any way of correcting for this silent majority, or any sort of predictable systematic tendency, so what? We don't know everything? There's a lot of things we don't know, and on most topics we hear from only a small fraction of people, both expert and otherwise. Why is this worth pointing out? (The point of knowing about things like publication biases in science is that they are systematic: once you know a…

But the non-expert bias online is systematic. Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise. There are real, pragmatic ways to correct for this bias and get more useful information out of online forums, too. For example: 1.) Look for people who have a solid, independently-verifiable track record who are…

"Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise."

Anecdotally, this is absolutely wrong. I can think of almost no one who participates online, who isn't better than most people who don't participate online.

Maybe it's an issue of averages, and at the edges this is true - that the average "superstar" spends less time online, but that the average "OK" person does spend time online.

But I can certainly say, the silent majority of programmers, the ones who don't take part in anything except just focusing on their work, are almost always worse. I've seen this time and time and time again.

Re: The Silent Majority of Experts

#12
post #11

Earlier quoted context omitted.

But the non-expert bias online is systematic. Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise. There are real, pragmatic ways to correct for this bias and get more useful information out of online forums, too. For example: 1.) Look for people who have a solid, independently-verifiable track record who are…

"Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise." Anecdotally, this is absolutely wrong. I can think of almost no one who participates online, who isn't better than most people who don't participate online. Maybe it's an issue of averages, and at the edges this is true - that the average "superstar" spend…

Availability heuristic.

I think your perspective may be warped by two things: not having worked with excellent engineers in person, and only considering relatively famous devs in your set of people who "participate online" - e.g. you're excluding the denizens of the last 80% of pages on StackOverflow.

One of the most capable engineers I've ever worked with is a guy called Yooichi Tagawa. The guy has an incredible appetite for complexity, as well as spooling up on old codebases and new technologies. But you'll find very little by him online, both because he's Japanese and doesn't use English often, and also he's squirrelled away inside Embarcadero, working on Delphi compiler as he's been doing for the past 15 years or so.

Re: The Silent Majority of Experts

#14
post #12
post #11

Earlier quoted context omitted.

"Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise." Anecdotally, this is absolutely wrong. I can think of almost no one who participates online, who isn't better than most people who don't participate online. Maybe it's an issue of averages, and at the edges this is true - that the average "superstar" spend…

Availability heuristic. I think your perspective may be warped by two things: not having worked with excellent engineers in person, and only considering relatively famous devs in your set of people who "participate online" - e.g. you're excluding the denizens of the last 80% of pages on StackOverflow. One of the most capable engineers I've ever worked with is a guy called Yooichi Tagawa. The guy has an incredible app…

"Availability heuristic."

It's certainly possible. And there are definitely a great many devs who don't participate online but are brilliant, don't get me wrong.

But I still think the average person who does participate online is better than the average person who doesn't.

In retrospect, what bothered me was the implication that participating online is detrimental, when I think it's just the opposite, if for no other reason than that people who do participate online tend to care more about programming than people who don't, and are therefore better.

Also, availability cuts both ways. Don't think about the amazing devs you've worked with, think of the averages. Trust me, I've worked with people of wildly, wildly varying ability, I know what's out there. Both on the amazing side, and on the "can't code worth a damn" side.

Re: The Silent Majority of Experts

#15
There's some sort of point about experts in general which I am not sure exactly how to articulate but I think is relevant. I'm going to try to klutz through it anyway.

There are lots of fields where fundamental theories are relatively weak or have poor predictive powers. Macroeconomics, climatology, nutrition etc. Basically, we don't have real Knowledge. We have a bunch of data and a bunch of theories. Some of the theories that aren't very general or aren't very applicable to real life scenarios are predictive but relatively useless. We know that certain nutrients have some importance. We know that restricting caloric intake leads to weight loss. We know that money supply, inflation and other things are linked together in various ways. The theories don't answer the questions we want then to with any kind of certainty. Still, we sometimes need to make decisions and some knowledge is better than none so we go and find experts anyway. There are people who are experts. They're experts in the study and they are aware of our knowledge in the field such as it is. But they don't have real answers because there just aren't useful answers to be had at this point. All of medicine was like this until pretty recently.

When Darwin published "The Origin of Species" evolutionary biology came to being as a different kind of field. One where the knowledge was real and the theory predictive. The theory was fundamental and strong. Darwin could make claims & predictions with a lot of confidence. Subsequent biologists could keep making predictions and when new discoveries (like genetics) were made, they were found to be consistent with the predictions of evolutionary biology. In fact, if they hadn't been, a careful researcher would probably assume that the mistake was in their own conclusions, not in Darwins. So if Chimpanzees are closer to humans than to Gorillas, we share common acceptors not shared with Gorillas and the distinction between Humans and Apes (if we want to keep Apes as a category) is morphological (which is allowed by Darwin) rather than one of proximity on a family tree.

Darwin was careful. He didn't publish until he was sure. If he didn't been sure we wouldn't have published. There are lots of Darwins in every one of the former type of field. They haven't discovered real Knowledge that can tell them what to do in an economic recession or what people should eat so they shut their mouth and keep looking. They are still experts but their expert opinion is "I dunno." That doesn't register as expertise so we go on to find someone that will explain about Aggregate demand, antinflamatory diets, carbohydrates or something like that.

Development methodologies, executive compensation, distributed companies etc. are in the category of things that we don't have real "scientific" knowledge about. Most business-ey knowledge is like that.

Now it sounds like I'm bashing people who talk about this stuff and I don't mean to be (hence my disclaimer at the start). I'm just pointing out that our knowledge in different fields is different. Joel Spolsky is very insightful in his essays about the Software industry, for example. But there are certain people that are comfortable with anecdote and generalities and assertions that may turn out to be untrue. There are certain people (like Darwin) who are not. If we're talking about development methodologies, the people we here from are self selected. They are comfortable making grand statements, manifestos and such even though they may be wrong.

That's still useful and certainly interesting, but there is a big category of people we aren't hearing from and they are relevant to the discussion.

Re: The Silent Majority of Experts

#17
post #11

Earlier quoted context omitted.

But the non-expert bias online is systematic. Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise. There are real, pragmatic ways to correct for this bias and get more useful information out of online forums, too. For example: 1.) Look for people who have a solid, independently-verifiable track record who are…

"Time spent participating online is time not spent honing your craft, and so there should be a strong inverse correlation between posting frequency and expertise." Anecdotally, this is absolutely wrong. I can think of almost no one who participates online, who isn't better than most people who don't participate online. Maybe it's an issue of averages, and at the edges this is true - that the average "superstar" spend…

I can think of such people. In fact, i don't think any of the really excellent people i've worked with blog, or spend much time in forums, or use Twitter for technical stuff. Some are involved with local meetups. The rest are just getting on with things.

On the other hand, i routinely come across blog posts or HN comments by people who clearly have much less expertise than them.

Re: The Silent Majority of Experts

#18
post #3

One place in which this problem was particularly apparent was (is? I haven't kept up with it for a while) the PHP community. PHP gets a bad rap as a terrible language, but a large part of this was the ease of beginning the learning process with PHP; download [X|W|M]AMPP, spin up an Apache instance, and get just dive in. What this led to was a lot of sub-par instruction material from people who didn't have a solid gro…

I've worked on huge web applications that were super maintainable and fun to work on, solving pretty interesting problems for multi-billion dollar companies... In PHP. I sometimes want to defend the language, but it does have some terrible parts to it, and a lot of the people that use it aren't software engineers. But a lot of them are, and so I just choose to work with them instead of complaining about it all!

On top of that, the latest developments in the language and community have seen some big changes for the better around tooling, best practices, and the like. So I'm firmly of the belief that with the RFC, PSR and Composer/Packagist trio, as well as things like HHVM, we will see things begin to change for the better on this front in the future. Call me Mr Optimistic ;)

Re: The Silent Majority of Experts

#20
post #15

There's some sort of point about experts in general which I am not sure exactly how to articulate but I think is relevant. I'm going to try to klutz through it anyway. There are lots of fields where fundamental theories are relatively weak or have poor predictive powers. Macroeconomics, climatology, nutrition etc. Basically, we don't have real Knowledge. We have a bunch of data and a bunch of theories. Some of the th…

This comment is sufficiently interesting that I want to grab a coffee with you and talk about it IRL, but you don't have any contact information in your profile so I don't know where in the world you are.

The next time you roll through Austin make sure to look me up.

Post reply on HN