Yup, that it is a funny anecdote. Most ninjas were dressed /disguised in common, civilian clothes to do their trade. And black is actually not the best color to hide (when you stand in front of a light wall, you would stood out in low light conditions wearing black).