"Whonix alone" is probably fine against browser exploits in the Tor browser (of which I generally assume there are many, because it's a browser of Very Much Interest to plenty of agencies). However, if you assume a "dirty host," with various bits of nastiness on it, if you're just using Virtualbox or something, it would be easy enough for a compromised Whonix workstation VM to chatter away with the host and have the host beacon out, or have the host modify the disk images for Whonix to add badness, or something of the sort. It's not a high risk, but if you're going to be doing something with Tor where failure of opsec puts you in prison for life (see DPR), it's something to consider.
Qubes adds a few more layers of isolation and security, because you now have a Type 1 hypervisor under everything (currently Xen), with your other isolation VMs separated out. Badness in another VM can't directly impact the Whonix VMs, unless it's compromised Dom0, at which point you've lost with Qubes anyway.
Both are at risk from a hypervisor escape as well, but I generally consider Xen to be a somewhat better inspected and harder to escape from target than Virtualbox or VMWare Workstation, just because there's less to Xen. It's a far smaller codebase, and when you're using hardware virtualization with paravirtualized devices (virtio-type interfaces), there's just not as much surface exposed for attack. It's not impossible, but I would generally consider VMWare/Virtualbox somewhat softer targets to escape from than Xen.
Again, does any of this matter for casual use? No. But if you're going to use Tor for things that have actual consequences, it may very well matter a lot, and at that point, fully understanding the various threats and how they've been used over the years may be a matter of your freedom.
For whatever it's worth, I try to add Tor traffic where I can, just to help with the noise factor.