Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Silent Scientist: When Software Research Fails to Reach Its Audience

cacm.acm.org

Post reply on HN
Print view View original

41–50 of 54 posts

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#41

Post by bsoles »

The audience of software research is other software researchers.

The expectation that a practicing CS graduate, even with a master's degree, should be able to read, understand, and apply in their work research articles published in academic journals is not very meaningful.

Not because they are not capable people, but because research articles these days are highly specialized, building upon specialized fields, language, etc.

We don't expect mechanical engineers read latest research on fluid mechanics, say, making use of Navier-Stokes equations. I am a mechanical engineer with a graduate degree in another field and I would be immediately lost if I tried to read such an article. So why do we expect this from software engineers?

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#42

Post by ngriffiths »

> active science communication has been sparse in the area of software research, and those who have tried often find their efforts unrewarded or unsuccessful.

The authors suggest:

> Identify your target audience to tailor your message! Use diverse communication channels beyond papers, and actively engage with practitioners to foster dialogue rather than broadcasting information!

What I would emphasize is that many researchers just don't know how to do it. It isn't as simple as just thinking up a target audience and churning out a blog post. If you are the median researcher, ~0 people will read that post!

I think people underestimate:

- How hard it is to find the right target audience - How hard it is to understand the target audience's language - How hard it is to persuade the target reader that this work you've done should matter even a little to their work, even when you designed it specifically for them - How few people in the audience will ever understand your work well - How narrow your target audience should be

I also think many researchers want to be able to, if not as a primary career goal then at least as a fulfilling, public service type activity. Currently testing this out a bit (more: https://griffens.net).

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#43

Post by rossdavidh »

mmooss wrote: post #37
mkozlows wrote: post #36

Earlier quoted context omitted.

Because it's usually not that useful. I have a friend who was software-adjacent, and would post all these exciting studies showing that this or that practice was a big deal and massively boosted productivity. And without fail, those studies were some toy experiment design that had nothing to do with actual real-world conditions, and weren't remotely strong enough to convince me to up-end opinions based on my actual e…

Thanks for your POV. > those studies were some toy experiment design that had nothing to do with actual real-world conditions Isn't that the nature of understanding and applying science? Science is not engineering: Science discovers new knowledge. Applying that knowledge to the real world is engineering. Perhaps overcoming that barrier, to some degree, is worthwhile. In a sense, it's a well-known gap.

In real-world software development, managing complexity is often (usually) the core of the challenge. A simplified example, is leaving out the very thing that is the obstacle to most good software development. In fact, it is sometimes the case that doing something that helps with managing complexity, will impair performance as measured in some other way. For example, it may slow execution speed by some amount, but allow the software to be broken into smaller pieces each of which is more compehensible. Managing this tradeoff is the key to much software development. If you test with "toy experiment design", you may be throwing out the very thing that is most important to study.

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#44

Post by KalMann »

bsoles wrote: post #41

The audience of software research is other software researchers. The expectation that a practicing CS graduate, even with a master's degree, should be able to read, understand, and apply in their work research articles published in academic journals is not very meaningful. Not because they are not capable people, but because research articles these days are highly specialized, building upon specialized fields, langua…

Well I think you have to ask what the goal of the researchers are. In the case of fluid mechanics they may research new algorithms that make into the software mechanical engineers use, even if they don't understand the algorithms themselves for example. So mechanical engineers still benefit from the research.

So I guess what I'm wondering is if software engineers benefit from the research that software research produce? (even if they don't understand it themselves)

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#45

Post by ngriffiths »

bsoles wrote: post #41

The audience of software research is other software researchers. The expectation that a practicing CS graduate, even with a master's degree, should be able to read, understand, and apply in their work research articles published in academic journals is not very meaningful. Not because they are not capable people, but because research articles these days are highly specialized, building upon specialized fields, langua…

Not all engineers are in the target audience, and not all details of research findings need to be conveyed to the target audience to make a real impact. The point is if no findings ever make it to engineers (in the broadest sense), there is zero real world impact. I guess real impact is not the only goal but it's a valid one.

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#46

Post by mmooss »

rossdavidh wrote: post #43
mmooss wrote: post #37

Earlier quoted context omitted.

Thanks for your POV. > those studies were some toy experiment design that had nothing to do with actual real-world conditions Isn't that the nature of understanding and applying science? Science is not engineering: Science discovers new knowledge. Applying that knowledge to the real world is engineering. Perhaps overcoming that barrier, to some degree, is worthwhile. In a sense, it's a well-known gap.

In real-world software development, managing complexity is often (usually) the core of the challenge. A simplified example, is leaving out the very thing that is the obstacle to most good software development. In fact, it is sometimes the case that doing something that helps with managing complexity, will impair performance as measured in some other way. For example, it may slow execution speed by some amount, but al…

Great point. I think that also emphasizes the necessity of the D in R&D: The research has to be adapted to the real world to be useful, for example to organizational frameworks and processes that manage complexity as you say.

Most software organizations I know don't have anything like the time to do D (to distinguish it from software development), except in a few clear high-ROI cases. Big software companies like Microsoft and Google have research divisions; I wonder how much they devote to D as opposed to R, and how much of that is released publicly.

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#47

Post by mmooss »

Strilanc wrote: post #40
mmooss wrote: post #34

For the audience here, the opposite side of the coin is more relevant: Why don't you read software research? Based on this and other articles (and on experience), it's an especially underutilized resource. By reading it, you would gain an advantage over competition. Why aren't you using this advantage that is there for the taking? And why don't we see papers posted to HN?

Well, for example, consider this recent study that claimed developers using AI tools take 19% longer to finish tasks [1]. This was their methodology: > we recruited 16 experienced developers from large open-source repositories (averaging 22k+ stars and 1M+ lines of code) that they’ve contributed to for multiple years. Developers provide lists of real issues (246 total) that would be valuable to the repository—bug fix…

Taking into account issues like that reading critically, which is great and essential. Dismissing ideas on that basis - often done on HN generally, even for large medical studies - is intellectually lazy, imho:

Life is full of flaws and uncertainty; that is the medium in which we swim and breath and work. The solution is not to lie at the bottom until the ocean becomes pure H2O; the trick is to find value.

Re: The Silent Scientist: When Software Research Fails to Reach Its Audience

#49

Post by nitwit005 »

This assumes a wide audience, but that tends not to be the case. Say you have a paper on some sort of database optimization. How many people are genuinely working on database optimizers in industry? Even a quite successful social media post has low odds of reaching them.
Post reply on HN