Overall legume allergies affect 1-5% of the population, this is all legumes combined including peanuts etc. Soy allergies are 0.3 - 0.5%, not sure what you qualify as pretty common. For the record I have a fairly serious peanut allergy and abstain from meat and dairy with zero issues.
FWIW, I have pretty bad reactions to most foods outside of eggs and ruminant meat... so whatever the limited number of the population; across billions, you're still talking about millions of people. And it's pretty bad when you're one of those people, and there's others literally want to outlaw your food supply.