I was really struck by the following line from a poem by Tupper.

"Heed only this, -- not whether those be swine,
But whether these be pearls, precious and pure."

So my take on this is, that we need be less concerned about the faults in others, and make sure what we call the pearls we have to give, are truly pearls.