There's got to be more to this story. Lara Croft was literally designed to be hot, to the point where there were complaints about the movie because the actress who played Lara Croft "wasn't hot enough." (Before it was released. After it was released, there were other reasons to complain about it.) It's not like you're lusting after Wendy Koopa.

Either this person doesn't like you and would have reacted like that to anything you said, or there was something offensive about the joke or where/to whom it was made.