I'm adopted, but I was adopted as a much older child (I was a young teenager) after being in foster/residential care for most of my life. I was incredibly traumatized and it took me a long time to get used to my adoptive family.
But I still think of them as my family, whether I like some of them or not.
I have always been very open about being adopted and I don't see a reason not to tell people, especially because it explains why I have complex PTSD, which is something that people should know about me if they're going to be a close person in my life.
From an autistic perspective, I do understand being very attached to your adoptive family and wanting people (and yourself) to feel like they are your "real" family. Sometimes I ignore the fact that my parents are not my biological parents, because they are still my parents. I personally wouldn't get defensive about it if I was asked, but I don't want my parents to feel like I don't think of them as my real parents.
I guess I can see it from the perspective of not wanting your parents to feel like they are not your "real" parents, and not wanting them to feel like imposters or something. I do feel bad for my parents when people point out that they are not my biological parents, and people often point out that we are not the same ethnicity which is really annoying and irrelevant. Biological or not, they are family.
Hope that made sense. Thought you might like to hear this from someone who is actually adopted.