Buildroot is definitely much simpler, and more limited. That's a plus and a negative, depending on what you're doing with it.
Do you have one board that you need to bring up in isolation? Buildroot is great. It's easy and fast. Do you want to share configuration across multiple designs? Buildroot hits its limits pretty quickly there. And Yocto really starts to shine.
I worked at a company that used Buildroot to support 20 different boards, all in the same repo. The boards shared some level of common configuration across the whole repo, and groups of boards also shared some common configuration.
The entire thing was managed by the C preprocessor as a pre-Buildroot step, and it was a gordian knot of emergent complexity.
The catch is that Buildroot doesn't give you any tools to enable shared configuration, none at all. Buildroot doesn't know how to share build-products across multiple images, and doesn't even know how to _build_ multiple images or keep them separate without having to clean your workdir. It doesn't even track file dependencies, let alone configuration dependencies. It's terrible at knowing when a package is stale.
Yocto is much, much better than Buildroot for handling these kinds of use cases. Images are just recipes, like any other kind of recipe. You can build 8 different images by just asking for each of them by name. Yocto won't rebuild anything that can be shared, and your images can be subclasses of each other which just add or remove a few packages or a bit of configuration.
Yocto is also better at tracking/managing customizations. In Yocto, it's easy to express something like "use the upstream OpenSSH recipe, but do something custom for the config step, and add an extra post-build step, and add these 3 patches, but only add patch 3 on some designs, etc".
In Buildroot, you have to do this in a global namespace using macro-processing in GNU Make. You have no control over the parse order of included Makefiles relative to the upstream files. It's messy and difficult to modify anything, and you have to understand each recipe's quirks and hooks. And the only programming language you have is GNU Make.
In Yocto, there is an inheritance system that lets you use .bbappend files to override the behavior of any recipe you want. You can also stack multiple append files across different layers. If you need something exotic, you can express it as parse-time Python. Yocto is more complex up-front, but it pays off IMO.