I have been following someone's critical chapter-by-chapter criticism of HPMOR for some time now, which might be illustrative. It is admittedly quite long:
http://su3su2u1.tumblr.com/tagged/Hariezer-Yudotter/chronoAmong other things, I've often seen people try to sell HPMOR as, "Harry Potter fanfiction in which Harry applies the scientific method to the magical world of Harry Potter!" And that shows up a little [^1] but is by and large not the content of the work. Harry does very few experiments to verify his hypotheses and is actually a broadly incurious character. For example: Harry is nominally interested in eliminating death, but never once investigates the many magical mechanisms which seem to eliminate death, preferring to just talk about it instead. Similarly, he comes up with hypotheses as to how magic works, but never bothers to investigate them beyond speculation.
Additionally, a lot of the "solutions" to problems Harry has are relatively unsatisfying: Harry often circumvents a problem by "clever" applications of the rules, but because the rules of magic as given both in Rowling and in Yudkowsky are ill-defined and even self-contradictory, this seems less clever and more like arbitrary author fiat. Harry is also given a small time machine early on in the plot, and so more often than not, the solution to his problems is, "Use the time-turner again."
Finally, I personally found the writing style to be generally in need of editing—not terrible but certainly not polished—but I consider that to be a smaller problem than the above plot-related issues. EDIT: All this doesn't mean that you shouldn't like it! These are just problems that I had with the work.
[^1]: The bit where Harry tries to experimentally prove that P=NP is probably my favorite part of the entire thing.