I'm somewhat similar. I'm kind of face-blind, and I don't really tend to imagine characters physically much at all. They're more shapes to me, bundles of impressions and size and movement.

Possibly by an odd coincidence, then, that meant that my default impression of Shadow was dark, but for a completely different reason. Between his name, the fact that he's kind of gloomy, the very noir style of the book, and possibly the fact that I may have partially registered the question of his race early on, there were times when I was imagining him almost as a literal shadow, a dark human shape at the center of the narrative. I think I would actually have been surprised if he was white, because by about half-way in I really did not associate him with light colours at all.

Which, given who he turns out to be, is mildly ironic, really.