That's not quite it. The power of a QC comes from modeling an exponentially large probabilistic state space using entanglement and superposition. The operations performed by a QC are also different, they can be analog (arbitrary rotations), but circuit depths (i.e. the number of operations) are still expected to be polynomial.
The difference is that the probabilistic state space uses probability amplitudes, which are complex valued and can be positive or negative, allowing for constructive and destructive interference over the probabilities tied to each state. Orchestrate the right kind of interference, and for some problems, you have an algorithm that outputs a solution to that problem with (relatively high probability) in time that, depending on the problem, may be exponentially faster. Examples of those problems include prime factorization/discrete logarithms (Shor's algorithm) and ones in quantum simulation (hence the interest in QC by chemists, physicists, etc.)