Where Did the Female and Male Symbols Originate?

Funny enough, these symbols were used to classify plants before classifying humans and animals.