I don't believe there's any such thing as authority by expertise for any meaningfully sized group of people. It works right up until there's a conflict between the security expert and the UI expert, or any other expert. Then whoever "wins" ends up being whoever has the most unspoken political power or the more forceful personality. And maybe the best choice ends up getting made for the product, and maybe it doesn't. And maybe it was one of those cases where the expert was just plain wrong but all of the other non-experts weren't empowered to push back, because there's no organization and it's supposedly a Platonic ideal of a meritocracy.
Every specialist faces a lifelong moral hazard/cognitive bias in which they overvalue the thing that's important to them (because it's literally their job to value that thing above all other considerations, and their value to their employer is based on a perception that the thing they know about is really really important), and they undervalue everything else. The job of an organization is to provide an explicit framework to allow everyone to have their say, and enable an actionable consensus to be formed. And every single person involved might think that that consensus is fatally flawed because it doesn't go quite far enough with respect to whatever the thing they care most about is. But really that's just the organization minimizing everyone's individual biases, so whatever gets produced is more likely to be closer to what's ideal for the majority of users.
The flat meritocracy ideal also ignores the fact that there are very few objectively correct answers to anything in software design. Get three different specialists in the same area and you might get three different answers over what needs to be done, and three different weightings of all the different pros and cons in every decision that affects other parts of the product. Most likely, none of them are wrong, but more importantly none of them are likely to be the One True Correct Answer, but it's human nature for each one of them to feel like that's what they're offering so of course they're going to argue for it. Most likely there is no One True Correct Answer to begin with, only sets of compromises that are more or less desirable. A true flat meritocracy would only work if there were correct answers that don't involve compromising anything else, and the specialists knew them all the time.