It would be enough for a friend having bragged about how they hallucinated about certain things. Words can convey very complex imagery like that and these experiences are encoded everywhere (in scripture, folk wisdom, idioms, songs, jokes, tales); in a web of distributed representations of the noosphere that we share via language. It is pretty much impossible to escape it and it seems conceivable that it might strongly bias our experiences under influence of hallucinogens because the mind is basically always concerned with interpreting inputs by the most suitable explanation (which will likely stem from stories and memories). Of course, we are only talking about a certain extent to which these experiences are determined by shared concept spaces, because there is no indication this is the only sensible hypothesis. The similarity of these experiences by different people may as well be explained by our shared cognitive architecture (e.g. modules for recognition of agency or venomous bugs) as other comments in this thread have suggested.