11–20 of 31 posts
Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
It's like those maps that are supposed to reveal something interesting but all they reveal is population.
Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
Part of that is explained in the comments here by an employee saying they assume Google (and to a degree iPhone/iPad I'm sure) get increased numbers because devs might use them for testing and thus more bugs are seen, but that does raise interesting questions about why LG leads them all in that metric.
Almost all the graphs that are for total bugs instead of normalized to number of users show very little that is useful. One exception to that I noticed is the bugs to battery level, and that was only useful in that it's a reminder that mobile devices spend a lot of time running while plugged in at full battery, which is just as easily said than inferred through a graph like that.
Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
We couldn’t find any other data on mobile bugs like this, so we decided to share what we have for the app dev community to have some benchmarks and insights.
They haven't analyzed how quickly bugs are resolved, they've analyzed how quickly bugs are marked as resolved.
The distinction is important. Nowhere in the report there's any attempt to judge the quality of the data and its reliability.
Oh, and the honest answer to the question "Why did we create this report?" is probably PR.
Pretty much all of these "most" findings are explainable by the distribution of the installed user base, i.e. they are not real results but just artifacts of the population sizes.
Yes, you’re right! The report covers data from Instabug users only. We extracted data from 30K apps with a range of user base sizes, locales, devices, etc. I definitely agree with you, it’s not a definitive representation of the market, but we believe we have a good enough sample and that the findings are valuable for app developers. We couldn’t find any other data on mobile bugs like this, so we decided to share wha…
The complaints are e.g. "Most bugs are reported from iPhones" because they are a very popular type of phone with the customers more likely to report bugs. It doesn't necessarily mean the iPhone is buggier than others.
I think the y axis: "% Bugs" in Fig. 1 has an incorrect scale. It doesn't add to 100%. Also, it doesn't seem consistent with the claim: "Bugs discovered through Instabug are most likely to be resolved within 24 hours of being reported"
The figure is showing percent of all bugs, not percent of resolved bugs. Likely the rest of the 100% is unresolved bugs. The confusion on the second point hinges in "most likely". You're likely interpreting that as the expectation of resolution time whereas they are using maximum likelyhood estimation. MLE is rather useless in this case, but it is technically still correct.
Earlier quoted context omitted.
Yes, you’re right! The report covers data from Instabug users only. We extracted data from 30K apps with a range of user base sizes, locales, devices, etc. I definitely agree with you, it’s not a definitive representation of the market, but we believe we have a good enough sample and that the findings are valuable for app developers. We couldn’t find any other data on mobile bugs like this, so we decided to share wha…
Sample size isn't the issue here, although a large heterogenous sample is good. The complaints are e.g. "Most bugs are reported from iPhones" because they are a very popular type of phone with the customers more likely to report bugs. It doesn't necessarily mean the iPhone is buggier than others.
That's also why we included the bugs/user data since it shows a completely different distribution across devices.