I do not think it is passive-agressive in itsef, though some passive-agressive people use correcting as masked criticism that is basically malicious.

If you feel neither of you are, then chances are that the person calling you that might be recognizing it as part of a pattern of someone else they know, even if your motivations are completely different. If that makes sense.