Another thing is that you can pipeline, for example, multiplies. So in a CPU multiply you give the CPU inputs, wait a couple cycles, and then get the result. In an FPGA you can build a pipelined multiply. It's built such that you can feed it input every cycle and get an output every cycle. The only caveat is that the outputs are delayed relative to the inputs. i.e. you may give it (2, 3) to multiply on one cycle, but you won't see the result (6) of that particular input on the output until a couple cycles later.
[Yup, "pipeline" here is the same term used to describe how modern CPUs get their performance. They, too, are pipelining their instruction execution so that many instructions can be in the process of executing at the same time. Though what I describe is a more extreme and specific kind of pipelining.]
This is kind of like having a bunch of multiplies in parallel. But it's useful for, for example, real-time calculations. You get to perform all the calculations you need every single cycle, no matter how complex; your results are just delayed by X cycles. Pipelines are usually also more efficient than straight parallelism (i.e. an 8 deep pipelined multiplier uses less silicon than 8 individual "serial" multiplier units).
Another interesting thing: In a previous life I built an FPGA based video processing device. It sat in an HDMI chain, so it had to be real-time. If that had been built with a GPU, most engineers would build the system to buffer up a frame, perform the processing, and then feed the processed frame out. That results in at least 1 frame of delay. In contrast, because we used an FPGA, it was simple to just pipeline the entire design and thus only needed to buffer up the few lines that we needed. This meant A) we needed no external memory (cheaper) and our latency was on the order of microseconds. In my travels with that job I ran into tons of other companies building video processing devices. They _all_ used frame buffers, which made their devices unacceptable for, e.g., gaming.