I can relate at least a bit, I think, but it's complicated....
Just being around people might be not-draining (might even be helpful for preserving energy, in some circumstances where they are acting as a sort of buffer between me and the world or infusing my existance with the sustenance of joy and comfort in other ways).....It's more the direct interaction that's draining; And with the direct interaction, it depends on what type of interaction it is, in what context.
Playing a game of cards with people I love and know well (or doing some other activity that requires minimal verbal interaction or no verbal interaction), not having to talk to anyone or follow anything but the game, in a low-key sensory environment is not super draining -- might not be draining at all.
Being in the same room with people I know and love well, in a low-key sensory environment with no expectation of direct interaction, each doing our own thing, them not being loud or doing anything that is overwhelming to my sensory system, that would not be draining at all.
Having a quiet conversation in a quiet/low-key environment, back and forth, with one person I know well and love, eventually that will wear me out -- I might be delighted and enjoy the conversation, but at some point it would exhaust me. I can't say how long it would take, because I've never timed it, and I really don't have any sense of time; If the conversation is shorter, I might not feel tired or overwhelmed by the processing demands. (Another factor is the nature of the conversation....things like subject matter, intensity, communication style of the other person.)
A group conversation with loved/familiar people in a quiet environment is always overwhelming and draining to some degree, no matter who the people are or whether I'm expected to speak or just to listen. (If I don't try to follow along and am just sort of there then it isn't draining.....sitting still doing nothing is extremely draining because it's very hard to do, but that is its own thing.)
Any attempt to interact with any number of loved/familiar people in an environment with too many sounds, smells, movement, bright lights, etc (or while I am distracted by tactile discomfort)...well, I'm already overwhelmed so the demands of interaction just add to that. Verbal interaction is more exhausting than nonverbal interaction, and the stress and drain on inner resources increases with each additional person as well as with the number and severity of sensory stressors.