Earlier quoted context omitted.
Sorry, but what part of "they only surveyed software developers in the United Kingdom" gives you the impression that their sampling might actually be good ? Worth noting that the study itself doesn't seem to mention their sampling methodology.
Studies are never meant to be an ultimate answer to everything. And just because they're not the ultimate answer, doesn't mean that they are therefore worthless, bad or "bullshit".
Study finds 83% of software developers feel burnout
251–260 of 308 posts
Re: Study finds 83% of software developers feel burnout
#252Re: Study finds 83% of software developers feel burnout
#253Earlier quoted context omitted.
have you taken intro statistics? all small sample sizes do is require your effect to be more pronounced to still be statistically significant.
Ah, then I guess that's why so few "statistically significant" studies end up being contradicted by further research...
Re: Study finds 83% of software developers feel burnout
#254This is such a great example of bullshit studies. Yeah I'm gonna go on a limb and say this is a marketing campaign. It's an insult to science to call this a "study". From the pdf: > The fieldwork of this study was carried out at remarkable speed, from the 23rd to 24th of June 2021 Data was collected in 1 day. Is this a joke? No it's not! > The population sampled included Software Engineers aged 18+ living in the Unit…
I wouldn't be so quick to discard it. From personal experience, the past 14+ months of working from home has been very stressful. I didn't think I was burnt out, until I started interviewing (due to work "stress", which turned out to be burn-out in disguise). One recruiter informed me they have seen a spike - 70% of interviewees canceled scheduled interviews, post-screening. I almost cancelled mine due to being overw…
Re: Study finds 83% of software developers feel burnout
#255Recently I've been feeling more and more detached from work. I thought about taking annual leave but I'm covering for my boss who just went on paternity leave. My employer also coerced (forced) us to take a big chunk of our annual leave already because of concerns there will be a huge clash later in the year. So I feel reluctant to use even more of it and then suffer later in the year. I also thought about asking for…
Re: Study finds 83% of software developers feel burnout
#256Earlier quoted context omitted.
A lot of the COVID19 stuff only has a population size (NOT EVEN sample size) of like, 15. Ex: J&J's blood clot issue. People make conclusions off of tinier sample sizes. There are plenty of scientific studies with just 50 or 100 people, but you need to keep the small numbers in mind. As long as sampling was done correctly, then small sample size won't matter much !!
> There are plenty of scientific studies with just 50 or 100 people, but you need to keep the small numbers in mind. The converse also needs to be held in mind as well. Very large studies are more likely to display statistically significant findings. Your p-value is really just a test of your sample size, so as the study grows your p-value tends to shrink. Get a large enough sample and you'll almost always find stati…
> If you’re testing whether mean=0 and actually the truth is that mean=0.000000001, and if you have a large enough sample size, then YOU WILL GET A TINY P-VALUE.
Think about this for a second. Does it make sense that the results of my statistical test would change if I measured in terragrams vs. micrograms? No, it doesn't, because it's not the numerical difference between null hypothesis and sample mean, it's the difference relative to standard deviation.
If the effect size is 0.000000001 and the standard deviation is 0.0000000000000001 then this is an enormous, important effect. If the standard deviation is 0.1, then this is a really small and unimportant effect. It can still be observed correctly with a sufficient sample size though because you've stated it's a real effect to begin with.
You will absolutely not get a tiny p value just because your sample size is large. That is wrong. You will be able to detect very small effects, but if there is no difference, you are more likely to draw the correct conclusion with 1,000,000 samples than with 100 samples.
The point the author here is trying to convey but is not getting clearly, is that a large sample size can detect an effect that is practically useless and call it significant. p-values tell you if it seems probable, given your data, if there is a difference between the two datasets. They make no claims about whether the difference is material.
Example python code to do prove p values are not likelier to be smaller just due to large sample sizes, given no actual difference:
import random
from scipy.stats import ttest_ind
num_tests = 10000
res = []
for test in range(num_tests):
n = 100
a = [random.random() for _ in range(n)]
b = [random.random() for _ in range(n)]
res = res + [ttest_ind(a,b)[1] Re: Study finds 83% of software developers feel burnout
#257Earlier quoted context omitted.
I used to work in a company with 400+ developers and our team barely had any new features to add. We took care of one screen in the app. However getting anything done was an impossible task and required huge amounts of code because of the complexity of the architecture. I left after 6 months and a couple weeks because I couldn't bear the stress. Fred Brooks is still right in 2021. More people, more problems.
This is why FAANG pays so much. Imagine 1000+ engineers working on a single page in a web app. Your entire team is responsible for 1 widget on the page.
Re: Study finds 83% of software developers feel burnout
#258If you were a professional painter and every couple of months, someone snuck into your art studio, threw out all your brushes, paint, canvas, and then dumped a huge load of requests for you to "fix" your old paintings, I'm pretty sure you'd get burned out pretty quick.
It's more like you get scrawlings from toddlers on your desk every day and the boss says "these were supposed to be the blueprints to the next skyscraper. I paid a lot for this so you can't throw it away" and then you need to work with crayon scrawlings and somehow get the job done
Re: Study finds 83% of software developers feel burnout
#259Earlier quoted context omitted.
I wouldn't be so quick to discard it. From personal experience, the past 14+ months of working from home has been very stressful. I didn't think I was burnt out, until I started interviewing (due to work "stress", which turned out to be burn-out in disguise). One recruiter informed me they have seen a spike - 70% of interviewees canceled scheduled interviews, post-screening. I almost cancelled mine due to being overw…
There should be a study on studies being posted to HN. We must be at like 99% of them that get critiqued for being a bad study in one way or another. I would love to see some examples where everyone accepts the study.
Often, the statistical methods are inappropriate at best, or downright deceiving at worst; there is nominal, if any, understanding about what "controlling for a variable" means, among many other such things. Not to mention that often the methods themselves are highly questionable in their reproducibility, statistics aside.
(Another classic is to point out flaws in generalizations for mice studies and such, but that is, imo, independent of the study's methods. It's a fair critique, but I don't think it's what you're interested in measuring w.r.t. "bad" studies.)
Re: Study finds 83% of software developers feel burnout
#260Earlier quoted context omitted.
Ah, then I guess that's why so few "statistically significant" studies end up being contradicted by further research...
You make it sound like those studies were bad because the sample size was too small. In reality bad study design or having an unrepresentative sample are more common issues.