Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We found no evidence that Bionic Reading has a positive effect on reading speed

blog.readwise.io

Post reply on HN
Print view View original

81–90 of 144 posts

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#81

Post by hoodwink »

noobhacker wrote: post #54
hoodwink wrote: post #2

Hey everyone, this is the follow up post to the experiment posted here on Hacker News a few weeks ago: https://news.ycombinator.com/item?id=31826204 . I'll be hanging out in the replies today if there are any questions I can answer :)

Excellent experimental design and analysis! Is there a way for me to buy you a coffee (or some cash equivalent)? I'd love to incentivize such rigor in online discourse. An obvious way is to try out readwise ;) but I mostly read online articles and (pirated) books, so I'm not sure I'm a good use case.

these might be the kindest words i've ever gotten in a hacker news thread so thank you! :D

we've actually built an app for reading articles and ebooks (and RSS and PDFs and email newsletters and Twitter threads): https://readwise.io/read

still in private beta but we'll be entering public beta before summer is over!

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#82

Post by thorum »

hoodwink wrote: post #14
codingdave wrote: post #11

I'm not surprised that this slows down most people. But I'd be curious about a more targeted audience of people with known reading problems: people with dyslexia, for example. Or even regardless of speed, does it help overall with people who have comprehension problems with written communication?

i got that feedback a lot after initially posting the experiment: > Since posting this experiment, I've received a lot of side comments along the lines of, "Well, of course I don't expect Bionic Reading to work for most people, but for [my subpopulation], it really works." If that were the case, we might expect to see disproportionate benefits for those participants who read faster with Bionic Reading than for those…

> Let's look at how many participants read faster with each font and their average speed gains.

“Average speed gains” doesn’t seem like the best metric here. A graph showing the distribution of users across different WPM differences would be much more informative. Sure, the average user doesn’t see outsized results, but are you saying that zero participants in your test read substantially faster with BR?

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#83

Post by dang »

t_mann wrote: post #62

Meta comment: I'd appreciate if we could adopt the 'academia style' for headlines (ie tell the results right away) instead of the 'clickbait style' (click here to find out). So in this example: "Bionic reading does not change reading speed: results from our experiment" instead of "Does Bionic reading work? ...".

Yes. We've changed the title to a representative sentence from the first paragraph.

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#84

Post by thaumasiotes »

thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

> the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading!

Well, that's not really something you'd want to admit in your paper, much less advance as an argument. The obvious conclusion is that there's a mistake in your data, not that some users will see an increase in reading speed of 4.9 words per second.

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#85

Post by tristanho »

thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

The results are what you would expect if you measured two identical fonts, ie random.

You can make the _claim_ in your comment, but there isn't statistical evidence that this effect is more than random, or if the same people were tested again their results wouldn't be opposite. Put another way, it doesn't pass the null hypothesis.

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#86

Post by tpoacher »

thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

I think the idea here isn't that we're treating all brains the same, but that the deviations represent statistical noise/outliers rather than an actual effect fron the bionic intervention.

I agree though, it would have been an interesting follow-up if they asked only people who felt they benefit from it, and then conducted their analysis again on that sample!

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#87

Post by gnicholas »

thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

Reminds me of The End of Average, [1] a book about human variation. It talks about how the Air Force tried to design a cockpit by averaging various measurements, only to find that their pilots weren't actually average.

Your point is especially apt here, where there is a possibility that the technology could be helpful for a subpopulation, and act somewhat like an assistive technology. It is very tricky to assess the utility of assistive technologies (I work in this field and have asked many experts how they do it) because the average impact isn't the most important thing. It doesn't matter if high-contrast mode is bad for 90% of people, if for the 10% who would actually use it, it's helpful.

One way that the study could have been optimized is if participants had been asked (after taking the test, to avoid priming) if they have any specific reading challenges. That would have helped identify whether there appear to be any subgroups that disproportionately benefit from the approach.

1: https://www.amazon.com/End-Average-Succeed-Values-Sameness/d...

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#88

Post by [deleted] »

thaumasiotes wrote: post #84
thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

> the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! Well, that's not really something you'd want to admit in your paper, much less advance as an argument. The obvious conclusion is that there's a mistake in your data, not that some users will see an increase in reading speed of 4.9 words per second.

[deleted]

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#89

Post by thorum »

tristanho wrote: post #85
thorum wrote: post #70

The article’s conclusion doesn’t seem to match the data they collected. They found that most people (52%) do read faster with bionic reading, the effect is generally quite small, but large for some minority of users - at least one user read 293 WPM faster with Bionic Reading! The authors then average results (good and bad) across all users, resulting in a number close to zero, and conclude Bionic Reading doesn’t work…

The results are what you would expect if you measured two identical fonts, ie random. You can make the _claim_ in your comment, but there isn't statistical evidence that this effect is more than random, or if the same people were tested again their results wouldn't be opposite. Put another way, it doesn't pass the null hypothesis.

I think the best approach would be a graph showing the distribution of users across different WPM differences, and if possible a second experiment with two non-BR fonts (to see how much random variation is expected). The positive results might all be noise, but I don’t think that should be the default assumption.

Re: We found no evidence that Bionic Reading has a positive effect on reading speed

#90

Post by Barrin92 »

can we kill this entire fad of speed reading and productivity life hack nonsense already? I propose we start to advocate slow reading in the spirit of Peter Norvig's learn programming in 10 years, and tell people to actually enjoy what it is they're reading.
Post reply on HN