Earlier quoted context omitted.
You are correct that it's like learning anything else. If you put the time in and are motivated and go about it the right way you can learn it. But it's still "the most difficult language to learn". So it's worth taking that difficulty into account. For example one can learn Spanish, Swedish and Vietnamese in around the same amount of time it would take to learn Japanese.
What about Chinese? As someone who's learning Japanese already, Chinese looks more difficult to me. To me it seems very subjective, some people will be better at learning one language vs. another.
But, there's an important distinction. Japanese is singled out as "somewhat more difficult for native English speakers to learn than other languages in the same category."