I know that when I complain about it, it's because it's blatant or otherwise poorly done. It's sort of the point of fiction to manipulate you without letting on that it's doing so, such that good fiction doesn't even seem manipulative. Thus, if I'm not really thinking about it, complaining that something in fiction is manipulative doesn't sound contradictory.