I started learning Japanese around 8 years ago, and I've never been a fan of saying it's "the most difficult language to learn." It's like learning anything else, if you enjoy learning it and are motivated enough, you'll get better. Also I'm surprised when people assume the grammar is difficult - to me, the "flexible grammatical structures" as mentioned in the article are a good thing because it gives you more leeway…
You are correct that it's like learning anything else. If you put the time in and are motivated and go about it the right way you can learn it. But it's still "the most difficult language to learn". So it's worth taking that difficulty into account. For example one can learn Spanish, Swedish and Vietnamese in around the same amount of time it would take to learn Japanese.
Spoken Japanese, though, is easy. Well, at least from a mechanical standpoint. Japanese grammar is dirt simple and minimalistic, and I found it shockingly easy to learn. However, as someone else mentioned elsewhere in the post, Japanese is very big on inferring as much as possible from context and only speaking what can't be inferred. It can honestly be a chore just to determine whether a sentence is in the first person, the second person, or the third person, because Japanese is so aggressively pro-drop that most of the time, that information has to be inferred from context. And if you intend on speaking Japanese yourself, there's a lot of subtlety when it comes to picking the appropriate register to speak in. Use the wrong verb endings or the wrong pronouns (in this case, I'm glad Japanese is so aggressively pro-drop), and you'll stick your foot in your mouth.
[0] Most people use it as a dictionary, but it also has a really awesome glossing mode: http://www.edrdg.org/cgi-bin/wwwjdic/wwwjdic?9T