It really depends. This reminds me of when I took a bunch of writing courses. I found that some people didn't "get" my stories, but the feedback they gave actually means that they experienced what I wanted them to experience when they read the stories. They didn't like it, but it's what I intended, so... it was a win, right?
So it depends on what you're going for. If you intended it to be really hard, and people say it's hard, you win. Now, if they are also giving it really bad reviews (few stars) and that affects you in some way, then you need to somehow integrate the difficulty into the game in a way that makes it clear to people.
You need to present the game as a challenge that not everyone is up to (maybe you already do, I haven't checked yet) but this should help make things more clear to people up front.
At the end of the day, people who type into text areas on the Internet are fickle and don't care about you, so don't take it too hard.
edit: just read your description. I think you should try changing it to a challenge, rather than a caution sign. Make it a call to action of sorts, see if people are up to the challenge rather than saying "you aren't up for it, don't bother."