I agree with the "colors are like letters" argument, and think it's a nice way of framing it. Although it's sometimes questionable practice to take a word from a person's piece, it's not the same as a whole line. Sometimes, though not often, something is done exactly in a way which compliments your ideas. Of course, it's still better to paraphrase than to simply parrot, but that's more a personal issue again than a larger one.
Anyway, colors themselves are far less important than the way they are used.