I wish it weren't true, but yeah, how we look / dress promotes ourselves like a billboard to everyone. It's an instinct of humans (with sight, of course) to judge each other as such. No matter your style of dress, though, hygiene is more of how we also instinctively judge each other immediately.
Bear with me.... I have given several classes on doing film fx and gore make up, and folks have always been surprised to find out that what gives them apprehension or "scares" them about another human being to wind up such simple things as - gross/unkempt/odd looking fingers/hands, eyes/around eye sockets, teeth/mouth area. For whatever reason, those three areas are where we focus, first, and if anything is off to us...we definitely give pause at the least. Most people just want to leave the situation entirely. It's an instinctual part of us. It just is. Clothes almost never get judged as scrutinous, first up, but they are likely secondary judgement for many.
Think about it this way, too: People talk to you, they look at you and people are also touch friendly to degrees (shaking hands or fist bumps - some are repeat offender huggers, haha). In all three areas, even with our mind types being who don't give great eye contact, don't talk as much and don't touch as much, usually....still....we glance enough at those areas of others to pass judgement of comfort / trust (those words I always use...once again). So, you have to keep in mind that the majority of folks (NT out there) definitely pay even more attention to the same things.
I hope that wasn't word salad. Oops.