TLDR: The benefit is to the team, not the individual. When executing open concept, ensure communication is relevant to the team by not including unrelated teams in the same space. We have had private and open offices; we're much happier with the results of open offices.
Overview:
First, we believe the benefit of the open office space is derived at the organization/team level by an increase in communication and ambient awareness. The mental load of this awareness and communication negatively affects the productivity of individuals but results a net benefit to the whole. This is why many individuals working in an open space will have their complaints, but project managers/product owners will often sing the open concept's praises. I'm in the latter group but I work in our open office as well.
In Practice:
Here in our office, we've done it both ways. We're located in Texas and space is plentiful, we've got enough private offices to go around and we did it that way for quite some time. After years of struggling through project overrun, bugs, and misallocated resources we decided to try an open concept.
Doing it Wrong:
The difference in team productivity couldn't be more apparent, at first it was much worse... we combined QA, Engineering, and Customer Service in one open concept office. Everyone was unhappy; Customer Service was constantly talking and distracting others with their one-sided phone calls, QA was very collaborative and yet had to talk over Customer Service, and the poor engineering team was just annoyed by both and resorted to headphones and instant messaging each other.
Doing it Right (the second time):
We realized that the biggest benefit in better communication is to the teams of people who are directly working on building our new products and features and anything else in the room just distracts from those goals.
After reorganizing our teams we moved QA and Customer Service out of the room, and instead filled it with more engineering teams and added our design and product teams as well. By including the right teams in the open space, the ambient noise level is much lower, when conversations do occur they're often directly relevant to all other teams in the room. Overall everyone is much happier in the open space even though some miss the privacy of their own offices.
To address the needs of those who occasionally need to work in a quiet place, we've turned a few of our private offices into "break out rooms" where anyone can go work as they wish. They're a great place to take phone calls and they're communal and provide limited amenities so nobody is occupying them constantly and removing themselves from their core team.
Months Later / Conclusions:
Today, we have one large open office which includes web engineers, iOS engineers, Android engineers, OSX/Windows engineers, UX designers, quality assurance, and product managers. We have breakout rooms for private work and all other staff is in private offices just outside the open space.
After our second try at an open office we can confidently say that we made the right decision. We've launched two major products in one quarter of the time it took to previously launch one. We've launched numerous bug fixes, updates, and features. And most importantly, our metrics not only show the real world results of that work, but everyone on the team is aware of them in their everyday workflow. The pace of development is not only faster, but we're making better engineering and design decisions. And, as an added benefit, our sense of team solidarity and moral is way up as well. People (who previously had private offices) still occasionally complain about being distracted, and sometimes they choose to work in a breakout room, but overall everyone has adjusted now and feels like we're kicking butt.
Of course, I'm simplifying months of thoughts and work down as much as I can here, and there's a lot more in the details of executing a good open office. I'm happy to expand on anything if this is at all helpful to anyone.