By far the hardest language I've ever tried to learn. In addition to pissing my Korean girlfriend off, it was so difficult I dropped the class after 2 weeks or so. Chinese is much much easier by an order of magnitude. Particularly if your focusing on speech and grammar vs writing. Not that Japanese writing is any easier. Hindsight is 20/20. Shoulda took Spanish...
I’ve learned both and found Chinese harder just because if you lose context then it’s hard to understand anything else being spoken. Japanese has more semantic repetition in the sentence structures. Both are equally logical grammatically though, and Chinese does have a much easier time learning character pronunciations.
I was an immigrant in the US as a teenager and it felt like that in English for me my first few years here. Eventually I got to a point where I can understand stuff without actively paying attention, and it got a lot easier to understand anything with or without context.
As an interesting aside, I speak Cantonese natively, and I barely speak Mandarin. Mandarin to me is like that as well. I have to pay 100% attention to keep up, and as soon as I lose context, all hope is lost in understanding anything in the conversation forward.