I don't think your English is to blame here; I got the same impression. I think it's a great compliment if I've done something to make someone enjoy a character they dislike. So long as they don't go on to deliberately imply something like "you have bad taste."