The soul-wrenching part depends a lot on the professor, I suppose. My Signals teacher was absolutely stellar and put a lot of effort into giving us a solid understanding of the theoretical underpinnings of her course, so it was pretty math-intensive. The exam, as I remember it, was pretty modest in terms of difficulty, but the course was pretty hard. I aced every homework, lab and exam, but that's largely because I loved that course. Everyone I knew struggled with it. It ate a good 4-6 hours off every week, in addition to the time spent on assignments & co. (sometimes more, but by my own making, i.e. because I'd hack on something alluded to, but not taught in that course. Like I said, I really liked that class).
My Circuits and Systems professor, on the other hand, wasn't too interested in this. He obviously cared about us knowing the math, but insisted that he was teaching a Circuits class, not a Mathematics course. His course seemed a lot easier as a consequence. I was never that good at math. I learned it because you can't do engineering without it, but it has always been my weak spot.
(Edit: I don't mean it in the "how much is 12% out of 45 USD? zomg I suck at math" way, that's 5.4 USD, I'm not an idiot. I mean it in the "vector calculus made sense, I think, but I could never have made a career out of it as I did with writing software and designing electronic devices" kind of way.)
My Systems professor was smart but had long stopped caring about that course and his students, so I learned most of the material by myself. At the time, I thought he was an idiot, but I've come to understand him in time. His course wasn't too torturous, largely because I skipped most of the classes.
Most of my colleagues (particularly those in CompEng) didn't take separate courses on Signals and Systems though. They had a single, bigger Signals & Systems course, that was taught by a couple of hardcore professors, both well into their sixties and both of them very draconian. One of them also taught an introductory Electronic Devices and Circuits course that I took, so I knew him firsthand: he was a very good professor, but he was about as flexible as a chopstick. An year when less than 1/3rd of his students failed the final exam was a really good year.