I don't see anything creepy about that at all. You might pleasantly surprise someone who wasn't even aware fanfic for your fandom existed. Even if there aren't many other writers or readers, I don't think there's anything to be ashamed of. You like what you like, and just because there isn't much of a fandom doesn't mean other people don't like it too.