Even many agricultural societies still had seasonal nomadic tendencies during which they hunted and gathered. Many sedentary societies lived and worked communally without much social stratification. The reason we associate agriculture with centralization is that centralized agricultural societies conquered the others, not that centralization is inherent to agriculture.
To elaborate on this point: without agriculture centralization wasn't advantageous (or possible) but with agriculture centralization is advantageous--so centralized societies outcompete and are "selected for" over non-centralized societies and we end up with a "genotype" of societies having the trait of centralization.
