IIRC, Le Guin has said that most Earthsea dwellers would look close to Hawai'ians in Earth terms, so definitely what we'd call people of colour or non-white, but maybe not quite black?

Not that it actually matters when it comes to the whitewashing, which is indeed infuriating.

I'm glad the Swedish edition of the translated A Wizard of Earthsea which was my first exposure to Le Guin left him brownskinned on the cover (with dramtic long black hair and intriguing scars, too). ;)